Enhancing Mall Security Based on Augmented Reality in the Post-pandemic World

ABSTRACT

COVID-19 has shocked the retail industry, customers’ concerns for their health and safety are taking business away from shopping malls. Mall owners are thinking of finding new ways to bring their business back in this post-pandemic world. How to protect customers from COVID-19 in shopping malls has been a problem. This study presents an enhanced mall security system to help enforce wearing masks and proper social distancing in shopping malls using augmented reality (AR). We created a novel visualization way Radar vision to display detected people in the perspective of mall guards, to help guards react quickly to violations, and better enforce the mandatory rules. When the mall guards wearing hololens activate the radar vision function, they can see all people violating wearing masks or social distance mandates through the wall. Mall guards can use gaze to select the target person and then use the voice command to activate the navigation arrow to help them quickly go to the scene. In addition to helping mall guards to enforce mandates, the system also provides assisted functions to protect customers. When the violation situation appears around a customer, the system will alert them to avoid and show an avoidance arrow until the user goes in the correct direction. We demonstrated a preliminary system with four surveillance cameras in our school building area. The pilot study shows that our system can effectively detect and display radar images, increasing the efficiency of mall guards and reducing customer safety concerns. © 2022, The Author(s), under exclusive license to Springer Nature Switzerland AG.