Azure Kinect performance evaluation for human motion and upper limb biomechanical analysis.

Human motion tracking is a valuable task for many medical applications. Marker-based optoelectronic systems are considered the gold standard in human motion tracking. However, their use is not always feasible in clinics and industrial environments. On the other hand, marker-less sensors became valuable tools, as they are inexpensive, noninvasive and easy to use. However, their accuracy can depend on many factors including sensor positioning, light conditions and body occlusions. In this study, following previous works on the feasibility of marker-less systems for human motion monitoring, we investigate the performance of the Microsoft Azure Kinect sensor in computing kinematic and dynamic measurements of static postures and dynamic movements. According to our knowledge, it is the first time that this sensor is compared with a Vicon marker-based system to assess the best camera positioning while observing the upper body part movements of people performing several tasks. Twenty-five healthy volunteers were monitored to evaluate the effects of the several testing conditions, including the Azure Kinect positions, the light conditions, and lower limbs occlusions, on the tracking accuracy of kinematic, dynamic, and motor control parameters. From the statistical analysis of the performed measurements, the camera in the frontal position was the most reliable, the lighting conditions had almost no effects on the tracking accuracy, while the lower limbs occlusion worsened the accuracy of the upper limbs. The assessment of human static postures and dynamic movements based on experimental data proves the feasibility of applying the Azure Kinect to the biomechanical monitoring of human motion in several fields.

The HA4M dataset: Multi-Modal Monitoring of an assembly task for Human Action recognition in Manufacturing.

This paper introduces the Human Action Multi-Modal Monitoring in Manufacturing (HA4M) dataset, a collection of multi-modal data relative to actions performed by different subjects building an Epicyclic Gear Train (EGT). In particular, 41 subjects executed several trials of the assembly task, which consists of 12 actions. Data were collected in a laboratory scenario using a Microsoft® Azure Kinect which integrates a depth camera, an RGB camera, and InfraRed (IR) emitters. To the best of authors' knowledge, the HA4M dataset is the first multi-modal dataset about an assembly task containing six types of data: RGB images, Depth maps, IR images, RGB-to-Depth-Aligned images, Point Clouds and Skeleton data. These data represent a good foundation to develop and test advanced action recognition systems in several fields, including Computer Vision and Machine Learning, and application domains such as smart manufacturing and human-robot collaboration.

In-Field Automatic Identification of Pomegranates Using a Farmer Robot.

Ground vehicles equipped with vision-based perception systems can provide a rich source of information for precision agriculture tasks in orchards, including fruit detection and counting, phenotyping, plant growth and health monitoring. This paper presents a semi-supervised deep learning framework for automatic pomegranate detection using a farmer robot equipped with a consumer-grade camera. In contrast to standard deep-learning methods that require time-consuming and labor-intensive image labeling, the proposed system relies on a novel multi-stage transfer learning approach, whereby a pre-trained network is fine-tuned for the target task using images of fruits in controlled conditions, and then it is progressively extended to more complex scenarios towards accurate and efficient segmentation of field images. Results of experimental tests, performed in a commercial pomegranate orchard in southern Italy, are presented using the DeepLabv3+ (Resnet18) architecture, and they are compared with those that were obtained based on conventional manual image annotation. The proposed framework allows for accurate segmentation results, achieving an F1-score of 86.42% and IoU of 97.94%, while relieving the burden of manual labeling.

Microsoft Azure Kinect Calibration for Three-Dimensional Dense Point Clouds and Reliable Skeletons.

Nowadays, the need for reliable and low-cost multi-camera systems is increasing for many potential applications, such as localization and mapping, human activity recognition, hand and gesture analysis, and object detection and localization. However, a precise camera calibration approach is mandatory for enabling further applications that require high precision. This paper analyzes the available two-camera calibration approaches to propose a guideline for calibrating multiple Azure Kinect RGB-D sensors to achieve the best alignment of point clouds in both color and infrared resolutions, and skeletal joints returned by the Microsoft Azure Body Tracking library. Different calibration methodologies using 2D and 3D approaches, all exploiting the functionalities within the Azure Kinect devices, are presented. Experiments demonstrate that the best results are returned by applying 3D calibration procedures, which give an average distance between all couples of corresponding points of point clouds in color or an infrared resolution of 21.426 mm and 9.872 mm for a static experiment and of 20.868 mm and 7.429 mm while framing a dynamic scene. At the same time, the best results in body joint alignment are achieved by three-dimensional procedures on images captured by the infrared sensors, resulting in an average error of 35.410 mm.

Human Gait Analysis in Neurodegenerative Diseases: A Review.

This paper reviews the recent literature on technologies and methodologies for quantitative human gait analysis in the context of neurodegenerative diseases. The use of technological instruments can be of great support in both clinical diagnosis and severity assessment of these pathologies. In this paper, sensors, features and processing methodologies have been reviewed in order to provide a highly consistent work that explores the issues related to gait analysis. First, the phases of the human gait cycle are briefly explained, along with some non-normal gait patterns (gait abnormalities) typical of some neurodegenerative diseases. Then the paper reports the most common processing techniques for both feature selection and extraction and for classification and clustering. Finally, a conclusive discussion on current open problems and future directions is outlined.

Ambient Assisted Living: A Review of Technologies, Methodologies and Future Perspectives for Healthy Aging of Population.

Over the last decade, there has been considerable and increasing interest in the development of Active and Assisted Living (AAL) systems to support independent living. The demographic change towards an aging population has introduced new challenges to today's society from both an economic and societal standpoint. AAL can provide an arrary of solutions for improving the quality of life of individuals, for allowing people to live healthier and independently for longer, for helping people with disabilities, and for supporting caregivers and medical staff. A vast amount of literature exists on this topic, so this paper aims to provide a survey of the research and skills related to AAL systems. A comprehensive analysis is presented that addresses the main trends towards the development of AAL systems both from technological and methodological points of view and highlights the main issues that are worthy of further investigation.

Internet of Robotic Things in Smart Domains: Applications and Challenges.

With the advent of the Fourth Industrial Revolution, Internet of Things (IoT) and robotic systems are closely cooperating, reshaping their relations and managing to develop new-generation devices. Such disruptive technology corresponds to the backbone of the so-called Industry 4.0. The integration of robotic agents and IoT leads to the concept of the Internet of Robotic Things, in which innovation in digital systems is drawing new possibilities in both industrial and research fields, covering several domains such as manufacturing, agriculture, health, surveillance, and education, to name but a few. In this manuscript, the state-of-the-art of IoRT applications is outlined, aiming to mark their impact on several research fields, and focusing on the main open challenges of the integration of robotic technologies into smart spaces. IoRT technologies and applications are also discussed to underline their influence in everyday life, inducing the need for more research into remote and automated applications.

A compact 3D omnidirectional range sensor of high resolution for robust reconstruction of environments.

In this paper, an accurate range sensor for the three-dimensional reconstruction of environments is designed and developed. Following the principles of laser profilometry, the device exploits a set of optical transmitters able to project a laser line on the environment. A high-resolution and high-frame-rate camera assisted by a telecentric lens collects the laser light reflected by a parabolic mirror, whose shape is designed ad hoc to achieve a maximum measurement error of 10 mm when the target is placed 3 m away from the laser source. Measurements are derived by means of an analytical model, whose parameters are estimated during a preliminary calibration phase. Geometrical parameters, analytical modeling and image processing steps are validated through several experiments, which indicate the capability of the proposed device to recover the shape of a target with high accuracy. Experimental measurements show Gaussian statistics, having standard deviation of 1.74 mm within the measurable range. Results prove that the presented range sensor is a good candidate for environmental inspections and measurements.

Experimental demonstration of a novel bio-sensing platform via plasmonic band gap formation in gold nano-patch arrays.

In this paper we discuss the possibility of implementing a novel bio-sensing platform based on the observation of the shift of the leaky surface plasmon mode that occurs at the edge of the plasmonic band gap of metal gratings, when an analyte is deposited on top of the metallic structure. We report numerical calculations, fabrication and experimental measurements to prove the sensing capability of a two-dimensional array of gold nano-patches in the detection of a small quantity of Isopropyl Alcohol (IPA) deposited on top of sensor surface. The calculated sensitivity of our device approaches a value of 1000 nm/RIU with a corresponding Figure of Merit (FOM) of 222 RIU(-1). The presence of IPA can also be visually estimated by observing a color variation in the diffracted field. We show that color brightness and intensity variations can be ascribed to a change in the aperture size, keeping the periodicity constant, and to different types of analyte deposited on the sample, respectively. Moreover, we demonstrate that unavoidable fabrication imperfections revealed by the presence of rounded corners and surface roughness do not significantly affect device performance.

Plasmonic bandgap formation in two-dimensional periodic arrangements of gold patches with subwavelength gaps.

We report on the formation of plasmonic bandgaps in two-dimensional periodic arrangements of gold patches. Orthogonal arrays of subwavelength slits with different periodicities have been studied by means of a three-dimensional finite-difference time-domain (FDTD) code, changing incident polarization and geometrical parameters. Spectral response of gold patches having different a form factor and surrounded by different media have been also investigated and compared in order to give a full description of bandgap shifts paving the way for the design of polarization-sensitive devices.

High quality heart and lung auscultation system for diagnostic use on remote patients in real time.

We propose a medical electronic-computerized platform for diagnostic use, which allows doctors to carry out a complete cardio-respiratory control on remote patients in real time. In the context of telemedicine the proposed system can be considered as a really innovative product in which all the most advanced technologies of biomedical engineering converge to guarantee an efficient and reliable home assistance that allows the patient a highly better quality of life in terms of prophylaxis, treatment and reduction of discomfort connected to periodic patient controls and/or hospitalization. Moreover the system has been equipped to be employed also to real-time rescue in case of emergency without the necessity for data to be constantly monitored by a medical centre. In fact, when an emergency sign is detected through the real-time diagnosing system, it sends a warning message to people able to arrange for his/her rescue. A Global Positioning System (GPS) also provides the patient coordinates. The proposed system, in its version for diagnostic use, has been verified by the heart specialists of the Institute of Cardiology in the General Hospital (Polyclinic) of the University of Bari, Italy.