Pulse rate variability estimation method based on imaging-photoplethysmography and application to telepsychiatry.

The worldwide adoption of telehealth services may benefit people who otherwise would not be able to access mental health support. In this paper, we present a novel algorithm to obtain reliable pulse and respiration signals from non-contact facial image sequence analysis. The proposed algorithm involved a skin pixel extraction method in the image processing part and signal reconstruction using the spectral information of RGB signal in the signal processing part. The algorithm was tested on 15 healthy subjects in a laboratory setting. The results show that the proposed algorithm can accurately monitor respiration rate (RR), pulse rate (PR), and pulse rate variability (PRV) in rest conditions.Clinical Relevance- The main achievement of this study is enabling non-contact PR and RR signal extraction from facial image sequences, which has potential for future use and support for psychiatrists in telepsychiatry.

Remote sensing of vital signs by medical radar time-series signal using cardiac peak extraction and adaptive peak detection algorithm: Performance validation on healthy adults and application to neonatal monitoring at an NICU.

BACKGROUND AND OBJECTIVE: Continuous monitoring of vital signs plays a pivotal role in neonatal intensive care units (NICUs). In this paper, we present a system for monitoring fully non-contact medical radar-based vital signs to measure the respiratory rate (RR), heart rate (HR), I:E ratio, and heart rate variability (HRV). In addition, we evaluated its performance in a physiological laboratory and examined its adaptability in an NICU. METHODS: A non-contact medical radar-based vital sign monitoring system that includes 24 GHz radar installed in an incubator was developed. To enable reliable monitoring, an advanced signal processing algorithm (i.e., a nonlinear filter to separate respiration and heartbeat signals from the output of radar), template matching to extract cardiac peaks, and an adaptive peak detection algorithm to estimate cardiac peaks in time-series were proposed and implemented in the system. Nine healthy subjects comprising five males and four females (24 ± 5 years) participated in the laboratory test. To evaluate the adaptability of the system in an NICU setting, we tested it with three hospitalized infants, including two neonates. RESULTS: The results indicate strong agreement in healthy subjects between the non-contact system and reference contact devices for RR, HR, and inter-beat interval (IBI) measurement, with correlation coefficients of 0.83, 0.96, and 0.94, respectively. As anticipated, the template matching and adaptive peak detection algorithms outperformed the conventional approach. These showed a more accurate IBI close to the reference Bland-Altman analysis (proposed: bias of -3 ms, and 95% limits of agreement ranging from -73 to 67 ms; conventional: bias of -11 ms, and 95% limits of agreement ranging from -229 to 207 ms). Moreover, in the NICU clinical setting, the IBI correlation coefficient and 95% limit of agreement in the conventional method are 0.31 and 91 ms. The corresponding values obtained using the proposed method are 0.93 and 21 ms. CONCLUSION: The proposed system introduces a novel approach for NICU monitoring using a non-contact medical radar sensor. The signal processing method combining cardiac peak extraction algorithm with the adaptive peak detection algorithm shows high adaptability in detecting IBI the time series in various application settings.

An mHealth App for the Non-contact Measurement of Pulmonary Function Using the Smartphone's Built-in Depth Sensor.

The use of smartphones in clinical practice is referred to as mobile health (mHealth). This has attracted great interest in both academia and industry because of its potential to augment healthcare. In this study, we developed an mHealth app for the non-contact measurement of chest-wall movements using the iPhone ' s built-in depth sensor, thereby enabling a pulmonary self-monitoring function for personal use. The depth sensor provides depth values for each pixel and 2D mapping of the chest-wall movements. To extract respiratory signals from the right and left thoracic regions and abdomen, a 2D-depth image-segmentation method was implemented. The method was based on the anatomy and physiology of chest-wall movements, assuming differences in the anterior displacement in the thoracic and abdominal regions. It was observed that the differences were significant in the segmented regions of interest (ROIs) of the right and left thoracic region and abdomen. Respiratory signals extracted from each ROI were compared with the contact bio-impedance signals, which were highly correlated (r=0.94).

Proof-of-principle Experiment on 24 GHz Medical Radar for Non-contact Vital Signs Measurement.

Medical radar for non-contact vital signs measurement exhibits great potential in both clinical and home healthcare settings. Especially during the corona virus spreading time, non-contact sensing more clearly shows the advantages. Many previous studies have concentrated on medical radar-based healthcare applications, but pay less attention to the working principles. A clear understanding of medical radars at both the mathematical and physical levels is critically important for developing application-specific signal processing algorithms. Therefore, this study aims to re-define the operating principle of radar, and a proof-of-principle experiment was performed on both actuator and human subjects using 24 GHz Doppler radar system. Experimental results indicate that there is a difference in the radar output signals between the two cases, where the displacement is greater than and less than half of the wavelength. For the former situation, the displacement x = n.λ/2 (n ≥ 1), one peak of radar signals corresponds to n peaks of baseband signals. By contrast, for the latter situation, the displacement x < λ/2, one peak of radar signals corresponds to one peak of baseband signals. Strikingly, with human measurement on the dorsal side, the the number of respiration peaks are seen from the radar raw signals.

Design and Evaluation of Digital Filters for Non-Contact Measuring of HRV using Medical Radar and Its Application in Bedside Patient Monitoring System.

A non-contact bedside monitoring system using medical radar is expected to be applied to clinical fields. Our previous studies have developed a monitoring system based on medical radar for measuring respiratory rate (RR) and heart rate (HR). Heart rate variability (HRV), which is essentially implemented in advanced monitoring system, such as prognosis prediction, is a more challenging biological information than the RR and HR. In this study, we designed a HRV measurement filter and proposed a method to evaluate the optimal cardiac signal extraction filter for HRV measurement. Because the cardiac component in the radar signal is much smaller than the respiratory component, it is necessary to extract the cardiac element from the radar output signal using digital filters. It depends on the characteristics of the filter whether the HRV information is kept in the extracted cardiac signal or not. A cardiac signal extraction filter that is not distorted in the time domain and does not miss the cardiac component must be adopted. Therefore, we focused on evaluating the interval between the R-peak of the electrocardiogram (ECG) and the radar-cardio peak of the cardiac signal measured by radar (R-radar interval). This is based on the fact that the time between heart depolarization and ventricular contraction is measured as the R-radar interval. A band-pass filter (BPF) with several bandwidths and a nonlinear filter, locally projective adaptive signal separation (LoPASS), were analyzed and compared. The optimal filter was quantitatively evaluated by analyzing the distribution and standard deviation of the R-radar intervals. The performance of this monitoring system was evaluated in elderly patient at the Yokohama Hospital, Japan.

Non-contact Measurement of Pulse Rate Variability Using a Webcam and Application to Mental Illness Screening System.

The COVID-19 pandemic is a global health crisis. Mental health is critical in such uncertain situations, particularly when people are required to significantly restrict their movements and change their lifestyles. Under these conditions, many countries have turned to telemedicine to strengthen and expand mental health services. Our research group previously developed a mental illness screening system based on heart rate variability (HRV) analysis, enabling an objective and easy mental health self-check. This screening system cannot be used for telemedicine because it uses electrocardiography (ECG) and contact photoplethysmography (PPG), that are not widely available outside of a clinical setting. The purpose of this study is to enable the extension of the aforementioned system to telemedicine by the application of non-contact PPG using an RGB webcam, also called imaging- photoplethysmography (iPPG). The iPPG measurement errors occur due to changes in the relative position between the camera and the target, and due to changes in light. Conventionally, in image processing, the pixel value of the entire face region is used. We propose skin pixel extraction to eliminate blinks, eye movements, and changes in light and shadow. In signal processing, the green channel signal is conventionally used as a pulse wave owing to the absorption characteristics of blood flow. Taking advantage of the fact that the red and blue channels contain noise, we propose a signal reconstruction method for removing noise and strengthening the signal in the pulse rate variability (PRV) frequency band by weighting the three signals of the RGB camera. We conducted an experiment with 13 healthy subjects, and showed that the PRV index and pulse rate (PR) errors estimated by the proposed method were smaller than those of the conventional method. The correlation coefficients between estimated values by the proposed method and reference values of LF, HF, and PR were 0.86, 0.69, and 0.96, respectively.

A Non-contact Spirometer with Time-of-Flight Sensor for Assessment of Pulmonary Function.

Assessment of pulmonary function is vital for early detection of chronic diseases such as chronic obstructive pulmonary disease (COPD) in home healthcare. However, monitoring of pulmonary function is often omitted owing to the heavy burden that the use of specific medical devices places on the patients. In this study, we developed a non-contact spirometer using a time-of-flight sensor that measures very small displacements caused by chest wall motion during breathing. However, this sensor occasionally failed when estimating the values from breathing waveforms because their shape depends on the subject test experience. As a result, further measurements were required to address motion artifacts. To accomplish high accuracy estimation in the face of these factors, we developed methods to estimate parameters from a part of the waveform and remove outliers from multiple-region measurements. According to laboratory experiments, the proposed system achieved an absolute error of 5.26 % and a correlation coefficient of 0.88. This study also addressed the limitations of depth sensor measurements, thereby contributing to the implementation of high-accuracy COPD screening.

Contactless Vital Signs Measurement System Using RGB-Thermal Image Sensors and Its Clinical Screening Test on Patients with Seasonal Influenza.

Background: In the last two decades, infrared thermography (IRT) has been applied in quarantine stations for the screening of patients with suspected infectious disease. However, the fever-based screening procedure employing IRT suffers from low sensitivity, because monitoring body temperature alone is insufficient for detecting infected patients. To overcome the drawbacks of fever-based screening, this study aims to develop and evaluate a multiple vital sign (i.e., body temperature, heart rate and respiration rate) measurement system using RGB-thermal image sensors. Methods: The RGB camera measures blood volume pulse (BVP) through variations in the light absorption from human facial areas. IRT is used to estimate the respiration rate by measuring the change in temperature near the nostrils or mouth accompanying respiration. To enable a stable and reliable system, the following image and signal processing methods were proposed and implemented: (1) an RGB-thermal image fusion approach to achieve highly reliable facial region-of-interest tracking, (2) a heart rate estimation method including a tapered window for reducing noise caused by the face tracker, reconstruction of a BVP signal with three RGB channels to optimize a linear function, thereby improving the signal-to-noise ratio and multiple signal classification (MUSIC) algorithm for estimating the pseudo-spectrum from limited time-domain BVP signals within 15 s and (3) a respiration rate estimation method implementing nasal or oral breathing signal selection based on signal quality index for stable measurement and MUSIC algorithm for rapid measurement. We tested the system on 22 healthy subjects and 28 patients with seasonal influenza, using the support vector machine (SVM) classification method. Results: The body temperature, heart rate and respiration rate measured in a non-contact manner were highly similarity to those measured via contact-type reference devices (i.e., thermometer, ECG and respiration belt), with Pearson correlation coefficients of 0.71, 0.87 and 0.87, respectively. Moreover, the optimized SVM model with three vital signs yielded sensitivity and specificity values of 85.7% and 90.1%, respectively. Conclusion: For contactless vital sign measurement, the system achieved a performance similar to that of the reference devices. The multiple vital sign-based screening achieved higher sensitivity than fever-based screening. Thus, this system represents a promising alternative for further quarantine procedures to prevent the spread of infectious diseases.

Non-contact monitoring of heart rate variability using medical radar for the evaluation of dynamic changes in autonomic nervous activity during a head-up tilt test.

Electrocardiography (ECG) is a mandatory standard for monitoring electrical activity of the heart in many clinical settings such as intensive care and emergency units. However, in situations wherein the skin is damaged, such as acute burn injuries, it is impossible to efficiently attach electrodes to the skin. In this study, we developed a non-contact cardiac monitoring system using a 24-GHz medical radar for directly measuring the beat-to-beat heart mechanical activity at a distance from a subject. The heart rate variability (HRV) was analysed using an autoregressive model (AR) from the measured beat-to-beat intervals during a head-up tilt test. To investigate the feasibility of the proposed system, we compared medical radar and ECG recording by using Lin's correlation coefficient and Bland-Altman analysis, which showed a negligible mean difference from the substantial agreement of Lin's correlation coefficient of 0.9 between the radar and ECG. The non-contact radar clearly monitored dynamic changes in HRV indices induced by the head-up tilt test. This type of non-contact HRV-sensing technique as an alternative approach has significant potential for advancing personal healthcare in both clinical and out-of-hospital settings.

Extracting Cardiac Information From Medical Radar Using Locally Projective Adaptive Signal Separation.

Electrocardiography is the gold standard for electrical heartbeat activity, but offers no direct measurement of mechanical activity. Mechanical cardiac activity can be assessed non-invasively using, e.g., ballistocardiography and recently, medical radar has emerged as a contactless alternative modality. However, all modalities for measuring the mechanical cardiac activity are affected by respiratory movements, requiring a signal separation step before higher-level analysis can be performed. This paper adapts a non-linear filter for separating the respiratory and cardiac signal components of radar recordings. In addition, we present an adaptive algorithm for estimating the parameters for the non-linear filter. The novelty of our method lies in the combination of the non-linear signal separation method with a novel, adaptive parameter estimation method specifically designed for the non-linear signal separation method, eliminating the need for manual intervention and resulting in a fully adaptive algorithm. Using the two benchmark applications of (i) cardiac template extraction from radar and (ii) peak timing analysis, we demonstrate that the non-linear filter combined with adaptive parameter estimation delivers superior results compared to linear filtering. The results show that using locally projective adaptive signal separation (LoPASS), we are able to reduce the mean standard deviation of the cardiac template by at least a factor of 2 across all subjects. In addition, using LoPASS, 9 out of 10 subjects show significant (at a confidence level of 2.5%) correlation between the R-T-interval and the R-radar-interval, while using linear filters this ratio drops to 6 out of 10. Our analysis suggests that the improvement is due to better preservation of the cardiac signal morphology by the non-linear signal separation method. Hence, we expect that the non-linear signal separation method introduced in this paper will mostly benefit analysis methods investigating the cardiac radar signal morphology on a beat-to-beat basis.

Twenty-four-hour continuous and remote monitoring of respiratory rate using a medical radar system for the early detection of pneumonia in symptomatic elderly bedridden hospitalized patients.

Respiratory rate is often measured manually and discontinuously by counting of chest wall movements in routine clinical practice. We introduce a novel approach to investigate respiration dynamics using a noncontact medical radar system for identifying patient at risk of infection. The system enables early detection of pneumonia in bedridden hospitalized patients.

Infection Screening System Using Thermography and CCD Camera with Good Stability and Swiftness for Non-contact Vital-Signs Measurement by Feature Matching and MUSIC Algorithm.

Screening systems for infectious diseases based on fever have been implemented at international airports to prevent the spread of infection for over a decade. Currently, only Infrared Thermography (IRT) is used for screening and measuring facial skin temperature, which is one of clinical indicators of potential infection. Aiming at higher accuracy in screening, our group adopted heart rate (HR) and respiration rate (RR) for the first time as the new screening parameters. In our previous study, we proposed a screening system based on dual image sensors, which include IRT and a charged-coupled devices (CCD) camera. The sensors can measure three vital signs simultaneously, namely HR, RR, and facial skin temperature. For the measurement of RR in this system, stability and swiftness must be applied for application in airports. In this study, we introduce feature matching and multiple signal classification (MUSIC) algorithm in this system. Feature matching between thermal images and RGB images captured by a CCD camera and IRT, respectively, is used to detect the nose and mouth in IRT, which helps extract respiration signals corresponding to airflow from breathing. In addition, the MUSIC algorithm improves the accuracy of RR frequency estimations in limited time respiration signal and achieves swiftness. The proposed method improves stability by simultaneously detecting the nose and mouth in thermal images, and enhances the accuracy of estimated RR using the MUSIC algorithm. By using this system, we evaluate the accuracy of the estimated vital signs. The performance of this screening system was evaluated using data obtained from 12 influenza patients and 13 healthy subjects at a clinical facility in Fukushima, Japan.

Stable Contactless Sensing of Vital Signs Using RGB-Thermal Image Fusion System with Facial Tracking for Infection Screening.

Infrared thermography (IRT) has been used to screen febrile passengers in international airports for over a decade. However, fever-based infection screening using IRT suffered from low sensitivity because measurements can be affected by ambient temperature, humidity, etc. In our previous study, we proposed an RGB-thermal image fusion system to measure vital signs i.e., the RGB camera detects tiny changes in color from facial skin, associated with blood flow, to estimate heart rate, and IRT senses temperature changes around the nasal area, caused by respiration, to measure respiratory rate). The inclusion of heart and respiratory rates lead to increased screening accuracy. In the present study, to promote the widespread use of our system in real-world settings, a face detection and tracking method was developed and implemented into the system, thereby enabling the accurate and stable measurement of vital signs. We assessed heart and respiratory rate estimation via an RGB-thermal image fusion system using Bland-Altman plots and statistical analysis.

Simple and objective screening of major depressive disorder by heart rate variability analysis during paced respiration and mental task conditions.

Since objective biomarkers for major depressive disorder (MDD) are not readily available, clinical psychiatrists diagnose patients with MDD subjectively based on clinical interviews and diagnostic criteria. It often raises various concerns, including false responses by patients, subjective factors, and inexperience of the attendants leading to incorrect diagnosis. Here, we developed a self-monitoring system for simple and objective screening of MDD using a photoplethysmography (PPG) sensor and a 24-GHz microwave radar, which was based on the analysis of heart rate variability (HRV) during paced respiration and mental task conditions. In our previous study, we assessed the reactivity of HRV measurements during a mental task (random number generation) condition in patients with MDD and healthy control subjects. The HRV indices are less reactive in patients with MDD compared to healthy subjects during the mental task, which enabled us to identify the patients at risk for depression. In this study, the reactivity of HRV was measured not only in the mental task but also during paced respiration (i.e., 5-s inhalation and 5-s exhalation) conditions, thereby assessing more detailed autonomic nervous system (ANS) activity via HRV indices. To investigate the effect of paced respiration on MDD screening, we compared the ANS activity via HRV indices in with/without paced respiration conditions in 28 drug-naïve patients with MDD and 27 healthy control subjects. The result showed that ANS significantly activated during the paced respiration condition (p<;0.05). The sensitivity in detecting patients with MDD was 86% under paced respiration and mental task conditions, which was higher than the sensitivity (68%) under mental task condition alone.

Remote sensing of multiple vital signs using a CMOS camera-equipped infrared thermography system and its clinical application in rapidly screening patients with suspected infectious diseases.

BACKGROUND: Infrared thermography (IRT) is used to screen febrile passengers at international airports, but it suffers from low sensitivity. This study explored the application of a combined visible and thermal image processing approach that uses a CMOS camera equipped with IRT to remotely sense multiple vital signs and screen patients with suspected infectious diseases. METHODS: An IRT system that produced visible and thermal images was used for image acquisition. The subjects' respiration rates were measured by monitoring temperature changes around the nasal areas on thermal images; facial skin temperatures were measured simultaneously. Facial blood circulation causes tiny color changes in visible facial images that enable the determination of the heart rate. A logistic regression discriminant function predicted the likelihood of infection within 10s, based on the measured vital signs. Sixteen patients with an influenza-like illness and 22 control subjects participated in a clinical test at a clinic in Fukushima, Japan. RESULTS: The vital-sign-based IRT screening system had a sensitivity of 87.5% and a negative predictive value of 91.7%; these values are higher than those of conventional fever-based screening approaches. CONCLUSIONS: Multiple vital-sign-based screening efficiently detected patients with suspected infectious diseases. It offers a promising alternative to conventional fever-based screening.

An Objective Screening Method for Major Depressive Disorder Using Logistic Regression Analysis of Heart Rate Variability Data Obtained in a Mental Task Paradigm.

BACKGROUND AND OBJECTIVES: Heart rate variability (HRV) has been intensively studied as a promising biological marker of major depressive disorder (MDD). Our previous study confirmed that autonomic activity and reactivity in depression revealed by HRV during rest and mental task (MT) conditions can be used as diagnostic measures and in clinical evaluation. In this study, logistic regression analysis (LRA) was utilized for the classification and prediction of MDD based on HRV data obtained in an MT paradigm. METHODS: Power spectral analysis of HRV on R-R intervals before, during, and after an MT (random number generation) was performed in 44 drug-naïve patients with MDD and 47 healthy control subjects at Department of Psychiatry in Shizuoka Saiseikai General Hospital. Logit scores of LRA determined by HRV indices and heart rates discriminated patients with MDD from healthy subjects. The high frequency (HF) component of HRV and the ratio of the low frequency (LF) component to the HF component (LF/HF) correspond to parasympathetic and sympathovagal balance, respectively. RESULTS: The LRA achieved a sensitivity and specificity of 80.0 and 79.0%, respectively, at an optimum cutoff logit score (0.28). Misclassifications occurred only when the logit score was close to the cutoff score. Logit scores also correlated significantly with subjective self-rating depression scale scores (p < 0.05). CONCLUSION: HRV indices recorded during a MT may be an objective tool for screening patients with MDD in psychiatric practice. The proposed method appears promising for not only objective and rapid MDD screening but also evaluation of its severity.