Deep Learning-Based Multi-modal COVID-19 Screening by Socially Assistive Robots Using Cough and Breathing Symptoms

ABSTRACT

In this paper, we present the development of a novel autonomous social robot deep learning architecture capable of real-time COVID-19 screening during human-robot interactions. The architecture allows for autonomous preliminary multi-modal COVID-19 detection of cough and breathing symptoms using a VGG16 deep learning framework. We train and validate our VGG16 network using existing COVID datasets. We then perform real-time non-contact preliminary COVID-19 screening experiments with the Pepper robot. The results for our deep learning architecture demonstrate 1) an average computation time of 4.57 s for detection, and 2) an accuracy of 84.4% with respect to self-reported COVID symptoms. © 2022, The Author(s), under exclusive license to Springer Nature Switzerland AG.