Human Action Recognition in Smart Living Services and Applications: Context Awareness, Data Availability, Personalization, and Privacy.

Smart living, an increasingly prominent concept, entails incorporating sophisticated technologies in homes and urban environments to elevate the quality of life for citizens. A critical success factor for smart living services and applications, from energy management to healthcare and transportation, is the efficacy of human action recognition (HAR). HAR, rooted in computer vision, seeks to identify human actions and activities using visual data and various sensor modalities. This paper extensively reviews the literature on HAR in smart living services and applications, amalgamating key contributions and challenges while providing insights into future research directions. The review delves into the essential aspects of smart living, the state of the art in HAR, and the potential societal implications of this technology. Moreover, the paper meticulously examines the primary application sectors in smart living that stand to gain from HAR, such as smart homes, smart healthcare, and smart cities. By underscoring the significance of the four dimensions of context awareness, data availability, personalization, and privacy in HAR, this paper offers a comprehensive resource for researchers and practitioners striving to advance smart living services and applications. The methodology for this literature review involved conducting targeted Scopus queries to ensure a comprehensive coverage of relevant publications in the field. Efforts have been made to thoroughly evaluate the existing literature, identify research gaps, and propose future research directions. The comparative advantages of this review lie in its comprehensive coverage of the dimensions essential for smart living services and applications, addressing the limitations of previous reviews and offering valuable insights for researchers and practitioners in the field.

Review on Human Action Recognition in Smart Living: Sensing Technology, Multimodality, Real-Time Processing, Interoperability, and Resource-Constrained Processing.

Smart living, a concept that has gained increasing attention in recent years, revolves around integrating advanced technologies in homes and cities to enhance the quality of life for citizens. Sensing and human action recognition are crucial aspects of this concept. Smart living applications span various domains, such as energy consumption, healthcare, transportation, and education, which greatly benefit from effective human action recognition. This field, originating from computer vision, seeks to recognize human actions and activities using not only visual data but also many other sensor modalities. This paper comprehensively reviews the literature on human action recognition in smart living environments, synthesizing the main contributions, challenges, and future research directions. This review selects five key domains, i.e., Sensing Technology, Multimodality, Real-time Processing, Interoperability, and Resource-Constrained Processing, as they encompass the critical aspects required for successfully deploying human action recognition in smart living. These domains highlight the essential role that sensing and human action recognition play in successfully developing and implementing smart living solutions. This paper serves as a valuable resource for researchers and practitioners seeking to further explore and advance the field of human action recognition in smart living.

Integrating Abnormal Gait Detection with Activities of Daily Living Monitoring in Ambient Assisted Living: A 3D Vision Approach.

Gait analysis plays a crucial role in detecting and monitoring various neurological and musculoskeletal disorders early. This paper presents a comprehensive study of the automatic detection of abnormal gait using 3D vision, with a focus on non-invasive and practical data acquisition methods suitable for everyday environments. We explore various configurations, including multi-camera setups placed at different distances and angles, as well as performing daily activities in different directions. An integral component of our study involves combining gait analysis with the monitoring of activities of daily living (ADLs), given the paramount relevance of this integration in the context of Ambient Assisted Living. To achieve this, we investigate cutting-edge Deep Neural Network approaches, such as the Temporal Convolutional Network, Gated Recurrent Unit, and Long Short-Term Memory Autoencoder. Additionally, we scrutinize different data representation formats, including Euclidean-based representations, angular adjacency matrices, and rotation matrices. Our system's performance evaluation leverages both publicly available datasets and data we collected ourselves while accounting for individual variations and environmental factors. The results underscore the effectiveness of our proposed configurations in accurately classifying abnormal gait, thus shedding light on the optimal setup for non-invasive and efficient data collection.

Ambient and Wearable Sensor Technologies for Energy Expenditure Quantification of Ageing Adults.

COVID-19 has affected daily life in unprecedented ways, with dramatic changes in mental health, sleep time and level of physical activity. These changes have been especially relevant in the elderly population, with important health-related consequences. In this work, two different sensor technologies were used to quantify the energy expenditure of ageing adults. To this end, a technological platform based on Raspberry Pi 4, as an elaboration unit, was designed and implemented. It integrates an ambient sensor node, a wearable sensor node and a coordinator node that uses the information provided by the two sensor technologies in a combined manner. Ambient and wearable sensors are used for the real-time recognition of four human postures (standing, sitting, bending and lying down), walking activity and for energy expenditure quantification. An important first aim of this work was to realize a platform with a high level of user acceptability. In fact, through the use of two unobtrusive sensors and a low-cost processing unit, the solution is easily accessible and usable in the domestic environment; moreover, it is versatile since it can be used by end-users who accept being monitored by a specific sensor. Another added value of the platform is the ability to abstract from sensing technologies, as the use of human posture and walking activity for energy expenditure quantification enables the integration of a wide set of devices, provided that they can reproduce the same set of features. The obtained results showed the ability of the proposed platform to automatically quantify energy expenditure, both with each sensing technology and with the combined version. Specifically, for posture and walking activity classification, an average accuracy of 93.8% and 93.3% was obtained, respectively, with the wearable and ambient sensor, whereas an improvement of approximately 4% was reached using data fusion. Consequently, the estimated energy expenditure quantification always had a relative error of less than 3.2% for each end-user involved in the experimentation stage, classifying the high level information (postures and walking activities) with the combined version of the platform, justifying the proposed overall architecture from a hardware and software point of view.

Behavioral Change Prediction from Physiological Signals Using Deep Learned Features.

Predicting change from multivariate time series has relevant applications ranging from the medical to engineering fields. Multisensory stimulation therapy in patients with dementia aims to change the patient's behavioral state. For example, patients who exhibit a baseline of agitation may be paced to change their behavioral state to relaxed. This study aimed to predict changes in one's behavioral state from the analysis of the physiological and neurovegetative parameters to support the therapist during the stimulation session. In order to extract valuable indicators for predicting changes, both handcrafted and learned features were evaluated and compared. The handcrafted features were defined starting from the CATCH22 feature collection, while the learned ones were extracted using a temporal convolutional network, and the behavioral state was predicted through bidirectional long short-term memory auto-encoder, operating jointly. From the comparison with the state of the art, the learned features-based approach exhibits superior performance with accuracy rates of up to 99.42% with a time window of 70 seconds and up to 98.44% with a time window of 10 seconds.

Remaining Useful Life Prediction from 3D Scan Data with Genetically Optimized Convolutional Neural Networks.

In the current industrial landscape, increasingly pervaded by technological innovations, the adoption of optimized strategies for asset management is becoming a critical key success factor. Among the various strategies available, the "Prognostics and Health Management" strategy is able to support maintenance management decisions more accurately, through continuous monitoring of equipment health and "Remaining Useful Life" forecasting. In the present study, convolutional neural network-based deep neural network techniques are investigated for the remaining useful life prediction of a punch tool, whose degradation is caused by working surface deformations during the machining process. Surface deformation is determined using a 3D scanning sensor capable of returning point clouds with micrometric accuracy during the operation of the punching machine, avoiding both downtime and human intervention. The 3D point clouds thus obtained are transformed into bidimensional image-type maps, i.e., maps of depths and normal vectors, to fully exploit the potential of convolutional neural networks for extracting features. Such maps are then processed by comparing 15 genetically optimized architectures with the transfer learning of 19 pretrained models, using a classic machine learning approach, i.e., support vector regression, as a benchmark. The achieved results clearly show that, in this specific case, optimized architectures provide performance far superior (MAPE = 0.058) to that of transfer learning, which, instead, remains at a lower or slightly higher level (MAPE = 0.416) than support vector regression (MAPE = 0.857).

AI-Based Early Change Detection in Smart Living Environments.

In the smart environments we live today, a great variety of heterogeneous sensors are being increasingly deployed with the aim of providing more and more value-added services. This huge availability of sensor data, together with emerging Artificial Intelligence (AI) methods for Big Data analytics, can yield a wide array of actionable insights to help older adults continue to live independently with minimal support of caregivers. In this regard, there is a growing demand for technological solutions able to monitor human activities and vital signs in order to early detect abnormal conditions, avoiding the caregivers' daily check of the care recipient. The aim of this study is to compare state-of-the-art machine and deep learning techniques suitable for detecting early changes in human behavior. At this purpose, specific synthetic data are generated, including activities of daily living, home locations in which such activities take place, and vital signs. The achieved results demonstrate the superiority of unsupervised deep-learning techniques over traditional supervised/semi-supervised ones in terms of detection accuracy and lead-time of prediction.

A Radar-Based Smart Sensor for Unobtrusive Elderly Monitoring in Ambient Assisted Living Applications.

Continuous in-home monitoring of older adults living alone aims to improve their quality of life and independence, by detecting early signs of illness and functional decline or emergency conditions. To meet requirements for technology acceptance by seniors (unobtrusiveness, non-intrusiveness, and privacy-preservation), this study presents and discusses a new smart sensor system for the detection of abnormalities during daily activities, based on ultra-wideband radar providing rich, not privacy-sensitive, information useful for sensing both cardiorespiratory and body movements, regardless of ambient lighting conditions and physical obstructions (through-wall sensing). The radar sensing is a very promising technology, enabling the measurement of vital signs and body movements at a distance, and thus meeting both requirements of unobtrusiveness and accuracy. In particular, impulse-radio ultra-wideband radar has attracted considerable attention in recent years thanks to many properties that make it useful for assisted living purposes. The proposed sensing system, evaluated in meaningful assisted living scenarios by involving 30 participants, exhibited the ability to detect vital signs, to discriminate among dangerous situations and activities of daily living, and to accommodate individual physical characteristics and habits. The reported results show that vital signs can be detected also while carrying out daily activities or after a fall event (post-fall phase), with accuracy varying according to the level of movements, reaching up to 95% and 91% in detecting respiration and heart rates, respectively. Similarly, good results were achieved in fall detection by using the micro-motion signature and unsupervised learning, with sensitivity and specificity greater than 97% and 90%, respectively.

In-home hierarchical posture classification with a time-of-flight 3D sensor.

A non-invasive technique for posture classification suitable to be used in several in-home scenarios is proposed and preliminary validation results are presented. 3D point cloud sequences were acquired using a single time-of-flight sensor working in a privacy preserving modality and they were processed with a low power embedded PC. In order to satisfy different application requirements (e.g. covered distance range, processing speed and discrimination capabilities), a twofold discrimination approach was investigated in which features were hierarchically arranged from coarse to fine by exploiting both topological and volumetric representations. The topological representation encoded the intrinsic topology of the body's shape using a skeleton-based structure, thus guaranteeing invariance to scale, rotations and postural changes and achieving a high level of detail with a moderate computational cost. On the other hand, using the volumetric representation features were described in terms of 3D cylindrical histograms working within a wider range of distances in a faster way and also guaranteeing good invariance properties. The discrimination capabilities were evaluated in four different real-home scenarios related with the fields of ambient assisted living and homecare, namely "dangerous event detection", "anomalous behaviour detection", "activities recognition" and "natural human-ambient interaction". For each mentioned scenario, the discrimination capabilities were evaluated in terms of invariance to viewpoint changes, representation capabilities and classification performance, achieving promising results. The two feature representation approaches exhibited complementary characteristics showing high reliability with classification rates greater than 97%.

Detecting falls with 3D range camera in ambient assisted living applications: a preliminary study.

In recent years several world-wide ambient assisted living (AAL) programs have been activated in order to improve the quality of life of older people, and to strengthen the industrial base through the use of information and communication technologies. An important issue is extending the time that older people can live in their home environment, by increasing their autonomy and helping them to carry out activities of daily living (ADLs). Research in the automatic detection of falls has received a lot of attention, with the object of enhancing safety, emergency response and independence of the elderly, at the same time comparing the social and economic costs related to fall accidents. In this work, an algorithmic framework to detect falls by using a 3D time-of-flight vision technology is presented. The proposed system presented complementary working requirements with respect to traditional worn and non-worn fall-detection devices. The vision system used a state-of-the-art 3D range camera for elderly movement measurement and detection of critical events, such as falls. The depth images provided by the active sensor allowed reliable segmentation and tracking of elderly movements, by using well-established imaging methods. Moreover, the range camera provided 3D metric information in all illumination conditions (even night vision), allowing the overcoming of some typical limitations of passive vision (shadows, camouflage, occlusions, brightness fluctuations, perspective ambiguity). A self-calibration algorithm guarantees different setup mountings of the range camera by non-technical users. A large dataset of simulated fall events and ADLs in real dwellings was collected and the proposed fall-detection system demonstrated high performance in terms of sensitivity and specificity.