Interpretable Passive Multi-Modal Sensor Fusion for Human Identification and Activity Recognition.

Human monitoring applications in indoor environments depend on accurate human identification and activity recognition (HIAR). Single modality sensor systems have shown to be accurate for HIAR, but there are some shortcomings to these systems, such as privacy, intrusion, and costs. To combat these shortcomings for a long-term monitoring solution, an interpretable, passive, multi-modal, sensor fusion system PRF-PIR is proposed in this work. PRF-PIR is composed of one software-defined radio (SDR) device and one novel passive infrared (PIR) sensor system. A recurrent neural network (RNN) is built as the HIAR model for this proposed solution to handle the temporal dependence of passive information captured by both modalities. We validate our proposed PRF-PIR system for a potential human monitoring system through the data collection of eleven activities from twelve human subjects in an academic office environment. From our data collection, the efficacy of the sensor fusion system is proven via an accuracy of 0.9866 for human identification and an accuracy of 0.9623 for activity recognition. The results of the system are supported with explainable artificial intelligence (XAI) methodologies to serve as a validation for sensor fusion over the deployment of single sensor solutions. PRF-PIR provides a passive, non-intrusive, and highly accurate system that allows for robustness in uncertain, highly similar, and complex at-home activities performed by a variety of human subjects.

Human Occupancy Detection via Passive Cognitive Radio.

Human occupancy detection (HOD) in an enclosed space, such as indoors or inside of a vehicle, via passive cognitive radio (CR) is a new and challenging research area. Part of the difficulty arises from the fact that a human subject cannot easily be detected due to spectrum variation. In this paper, we present an advanced HOD system that dynamically reconfigures a CR to collect passive radio frequency (RF) signals at different places of interest. Principal component analysis (PCA) and recursive feature elimination with logistic regression (RFE-LR) algorithms are applied to find the frequency bands sensitive to human occupancy when the baseline spectrum changes with locations. With the dynamically collected passive RF signals, four machine learning (ML) classifiers are applied to detect human occupancy, including support vector machine (SVM), k-nearest neighbors (KNN), decision tree (DT), and linear SVM with stochastic gradient descent (SGD) training. The experimental results show that the proposed system can accurately detect human subjects-not only in residential rooms-but also in commercial vehicles, demonstrating that passive CR is a viable technique for HOD. More specifically, the RFE-LR with SGD achieves the best results with a limited number of frequency bands. The proposed adaptive spectrum sensing method has not only enabled robust detection performance in various environments, but also improved the efficiency of the CR system in terms of speed and power consumption.