A Grad-CAM and YOLO based approach to extracting a human passageway ROI

ABSTRACT

We propose a computer vision based solution for the use of public video feeds to monitor crowd congestion with a focus on full automation as a potential scalable solution to address crowd statistics extraction needs amplified by the COVID-19 pandemic. The novelty is the provision of a fully autonomous solution that is able to generate a region of interest (ROI) upon initial feed registration with a self-refinement algorithm that perfects the ROI over time. Five classes were used from the Places 2 dataset. The root model of the hierarchy was used to classify between a beach, fast-food restaurant, train station, lawn and market with an overall accuracy of 95.58% and F1-Score of 88.94%. The market and beach class were then split into two sub-classes each. The 'beach' model was further explored using a Grad-CAM based post-processing technique to better understand what the model bases the classification on. The novelty is the use of the same technique to generate a human passageway region of interest based on the localisation of the Grad-CAM in several live beach footages. These were also inferred using a YOLOv5 based human tracking approach. The Grad-CAM based ROI was then evaluated for each footage on the YOLOv5 generated ROI. © 2021 IEEE.