DNN and Model Combined Passive Localization and Social Distancing with Partial Inertial Aiding

ABSTRACT

At the outbreak of COVID-19, researchers worldwide are seeking approaches to containing this disease. It is necessary to monitor social distance in enclosed public areas, such as subways or shopping malls. Passive localization, such as surveillance cameras, is a natural candidate for this issue, which is meaningful for rapid response to finding the infected suspect. However, the latest surveillance camera system is rotatable, even movable. And it is impossible for professionals to regularly calibrate the extrinsic parameters in a large-scale application, like COVID-19 suspect monitoring. We propose an inertial-aided passive localization method using surveillance camera for social distance measurement without the necessity to obtain extrinsic parameters. Moreover, the hardware modification cost of the off-the-shelf commercial camera is low, which suits the immediate application. The method uses SGBM (Semi-Global Block Matching) for 3D reconstruction and combines YOLOv3 and Gaussian Mixture Model (GMM) clustering algorithm to extract pedestrian point clouds in real time. Combining the 2D DNN-based and model-based methods makes a better balance between the computational load and the detection accuracy than end-to-end 3D DNN-based method. The inertial sensor provides an extra observation for the coordinate transformation from the camera frame into the world ground frame. Results show we can get a decimeter-level social distancing accuracy under noisy background and foreground environments at a low cost, which is promising for urgent COVID-19 public area monitoring. © 2023, The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.