Predicting daily cognition and lifestyle behaviors for older adults using smart home data and ecological momentary assessment.

OBJECTIVE: Extraction of digital markers from passive sensors placed in homes is a promising method for understanding real-world behaviors. In this study, machine learning (ML) and multilevel modeling (MLM) are used to examine types of digital markers and whether smart home sensors can predict cognitive functioning, lifestyle behaviors, and contextual factors measured through ecological momentary assessment (EMA). METHOD: Smart home sensors were installed in the homes of 44 community-dwelling midlife and older adults for 3-4 months. Sensor data were categorized into eight digital markers. Participants responded to iPad-delivered EMA prompts 4×/day for 2 wk. Prompts included an n-back task and survey on recent (past 2 h) lifestyle and contextual factors. RESULTS: ML marker rankings revealed that sensor counts (indicating increased activity) and time outside the home were among the most influential markers for all survey questions. Additionally, MLM revealed for every 1000 sensor counts, mental sharpness, social, physical, and cognitive EMA responses increased by 0.134-0.155 points on a 5-point scale. For every additional 30-minutes spent outside home, social, physical, and cognitive EMA responses increased by 0.596, 0.472, and 0.157 points. Advanced ML joint classification/regression significantly predicted EMA responses from smart home digital markers with error of 0.370 on a 5-point scale, and n-back performance with a normalized error of 0.040. CONCLUSION: Results from ML and MLM were complimentary and comparable, suggesting that machine learning may be used to develop generalized models to predict everyday cognition and track lifestyle behaviors and contextual factors that impact health outcomes using smart home sensor data.

CALDA: Improving Multi-Source Time Series Domain Adaptation With Contrastive Adversarial Learning.

Unsupervised domain adaptation (UDA) provides a strategy for improving machine learning performance in data-rich (target) domains where ground truth labels are inaccessible but can be found in related (source) domains. In cases where meta-domain information such as label distributions is available, weak supervision can further boost performance. We propose a novel framework, CALDA, to tackle these two problems. CALDA synergistically combines the principles of contrastive learning and adversarial learning to robustly support multi-source UDA (MS-UDA) for time series data. Similar to prior methods, CALDA utilizes adversarial learning to align source and target feature representations. Unlike prior approaches, CALDA additionally leverages cross-source label information across domains. CALDA pulls examples with the same label close to each other, while pushing apart examples with different labels, reshaping the space through contrastive learning. Unlike prior contrastive adaptation methods, CALDA requires neither data augmentation nor pseudo labeling, which may be more challenging for time series. We empirically validate our proposed approach. Based on results from human activity recognition, electromyography, and synthetic datasets, we find utilizing cross-source information improves performance over prior time series and contrastive methods. Weak supervision further improves performance, even in the presence of noise, allowing CALDA to offer generalizable strategies for MS-UDA.

The Indoor Predictability of Human Mobility: Estimating Mobility with Smart Home Sensors.

Analyzing human mobility patterns is valuable for understanding human behavior and providing location-anticipating services. In this work, we theoretically estimate the predictability of human movement for indoor settings, a problem that has not yet been tackled by the community. To validate the model, we utilize location data collected by ambient sensors in residential settings. The data support the model and allow us to contrast the predictability of various groups, including single-resident homes, homes with multiple residents, and homes with pets.

But will they use it? Predictors of adoption of an electronic memory aid in individuals with amnestic mild cognitive impairment.

OBJECTIVE: Electronic memory aids are being researched and developed widely to assist the everyday functioning of individuals experiencing cognitive decline. Although development studies show promise in the initial use of electronic memory aids, little is known about the factors that influence adoption of these aids after training ends. METHOD: We analyzed the baseline characteristics (e.g., demographics, cognitive performance) and training usage (e.g., frequency and pattern of use) of 32 older adults experiencing amnestic mild cognitive impairment who participated in a pilot clinical trial with an electronic memory and management aid (EMMA) tablet application. Sixteen participants who were still using EMMA at 3-months posttraining were defined as "adopters," whereas the 16 participants who were not using EMMA at 3-months posttraining were defined as "nonadopters." RESULTS: Adopters scored higher on baseline delayed memory (Cohen's d = .87) and language (Cohen's d = .82) index scores than nonadopters. Adopters also interacted with EMMA more frequently (Cohen's d = 1.34) and in greater quantities (Cohen's d > .87) than nonadopters by Week 2 of training. Stepwise logistic regression revealed that higher baseline language score and increased frequency of use during training significantly predicted classification of adopters at 3-months posttraining. CONCLUSIONS: Adoption of this electronic memory aid was enhanced by teaching the aid to individuals who demonstrated average-level language abilities and who used the aid on average eight times per day during training. Encouraging individuals to use the aid early and often during training can increase adoption of electronic memory aids. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Associations of device-measured physical activity and sedentary time with quality of life and fatigue in newly diagnosed breast cancer patients: Baseline results from the AMBER cohort study.

BACKGROUND: This study examined associations of device-measured physical activity and sedentary time with quality of life (QOL) and fatigue in newly diagnosed breast cancer patients in the Alberta Moving Beyond Breast Cancer (AMBER) cohort study. METHODS: After diagnosis, 1409 participants completed the SF-36 version 2 and the Fatigue Scale, wore an ActiGraph device on their right hip to measure physical activity, and an activPAL device on their thigh to measure sedentary time (sitting/lying) and steps. ActiGraph data was analyzed using a hybrid machine learning method (R Sojourn package, Soj3x) and activPAL data were analyzed using activPAL algorithms (PAL Software version 8). Quantile regression was used to examine cross-sectional associations of QOL and fatigue with steps, physical activity, and sedentary hours at the 25th, 50th, and 75th percentiles of the QOL and fatigue distributions. RESULTS: Total daily moderate and vigorous physical activity (MVPA) hours was positively associated with better physical QOL at the 25th (ß = 2.14, p = <.001), 50th (ß = 1.98, p = <.001), and 75th percentiles (ß = 1.25, p = .003); better mental QOL at the 25th (ß = 1.73, p = .05) and 50th percentiles (ß = 1.07, p = .03); and less fatigue at the 25th (ß = 4.44, p < .001), 50th (ß = 3.08, p = <.001), and 75th percentiles (ß = 1.51, p = <.001). Similar patterns of associations were observed for daily steps. Total sedentary hours was associated with worse fatigue at the 25th (ß = -0.58, p = .05), 50th (ß = -0.39, p = .06), and 75th percentiles (ß = -0.24, p = .02). Sedentary hours were not associated with physical or mental QOL. CONCLUSIONS: MVPA and steps were associated with better physical and mental QOL and less fatigue in newly diagnosed breast cancer patients. Higher sedentary time was associated with greater fatigue symptoms.

Piloting Smart Home Sensors to Detect Overnight Respiratory and Withdrawal Symptoms in Adults Prescribed Opioids.

BACKGROUND: Novel strategies are needed to curb the opioid overdose epidemic. Smart home sensors have been successfully deployed as digital biomarkers to monitor health conditions, yet they have not been used to assess symptoms important to opioid use and overdose risks. AIM: This study piloted smart home sensors and investigated their ability to accurately detect clinically pertinent symptoms indicative of opioid withdrawal or respiratory depression in adults prescribed methadone. METHODS: Participants (n = 4; 3 completed) were adults with opioid use disorder exhibiting moderate levels of pain intensity, withdrawal symptoms, and sleep disturbance. Participants were invited to two 8-hour nighttime sleep opportunities to be recorded in a sleep research laboratory, using observed polysomnography and ambient smart home sensors attached to lab bedroom walls. Measures of feasibility included completeness of data captured. Accuracy was determined by comparing polysomnographic data of sleep/wake and respiratory status assessments with time and event sensor data. RESULTS: Smart home sensors captured overnight data on 48 out of 64 hours (75% completeness). Sensors detected sleep/wake patterns in alignment with observed sleep episodes captured by polysomnography 89.4% of the time. Apnea events (n = 118) were only detected with smart home sensors in two episodes where oxygen desaturations were less severe (>80%). CONCLUSIONS: Smart home technology could serve as a less invasive substitute for biologic monitoring for adults with pain, sleep disturbances, and opioid withdrawal symptoms. Supplemental sensors should be added to detect apnea events. Such innovations could provide a step forward in assessing overnight symptoms important to populations taking opioids.

Multimodal Fusion of Smart Home and Text-based Behavior Markers for Clinical Assessment Prediction.

New modes of technology are offering unprecedented opportunities to unobtrusively collect data about people's behavior. While there are many use cases for such information, we explore its utility for predicting multiple clinical assessment scores. Because clinical assessments are typically used as screening tools for impairment and disease, such as mild cognitive impairment (MCI), automatically mapping behavioral data to assessment scores can help detect changes in health and behavior across time. In this paper, we aim to extract behavior markers from two modalities, a smart home environment and a custom digital memory notebook app, for mapping to ten clinical assessments that are relevant for monitoring MCI onset and changes in cognitive health. Smart home-based behavior markers reflect hourly, daily, and weekly activity patterns, while app-based behavior markers reflect app usage and writing content/style derived from free-form journal entries. We describe machine learning techniques for fusing these multimodal behavior markers and utilizing joint prediction. We evaluate our approach using three regression algorithms and data from 14 participants with MCI living in a smart home environment. We observed moderate to large correlations between predicted and ground-truth assessment scores, ranging from r = 0.601 to r = 0.871 for each clinical assessment.

Automated Cognitive Health Assessment Using Partially Complete Time Series Sensor Data.

BACKGROUND: Behavior and health are inextricably linked. As a result, continuous wearable sensor data offer the potential to predict clinical measures. However, interruptions in the data collection occur, which create a need for strategic data imputation. OBJECTIVE: The objective of this work is to adapt a data generation algorithm to impute multivariate time series data. This will allow us to create digital behavior markers that can predict clinical health measures. METHODS: We created a bidirectional time series generative adversarial network to impute missing sensor readings. Values are imputed based on relationships between multiple fields and multiple points in time, for single time points or larger time gaps. From the complete data, digital behavior markers are extracted and are mapped to predicted clinical measures. RESULTS: We validate our approach using continuous smartwatch data for n = 14 participants. When reconstructing omitted data, we observe an average normalized mean absolute error of 0.0197. We then create machine learning models to predict clinical measures from the reconstructed, complete data with correlations ranging from r = 0.1230 to r = 0.7623. This work indicates that wearable sensor data collected in the wild can be used to offer insights on a person's health in natural settings.

Detecting Smartwatch-based Behavior Change in Response to a Multi-domain Brain Health Intervention.

In this study, we introduce and validate a computational method to detect lifestyle change that occurs in response to a multi-domain healthy brain aging intervention. To detect behavior change, digital behavior markers (DM) are extracted from smartwatch sensor data and a Permutation-based Change Detection (PCD) algorithm quantifies the change in marker-based behavior from a pre-intervention, one-week baseline. To validate the method, we verify that changes are successfully detected from synthetic data with known pattern differences. Next, we employ this method to detect overall behavior change for n=28 BHI subjects and n=17 age-matched control subjects. For these individuals, we observe a monotonic increase in behavior change from the baseline week with a slope of 0.7460 for the intervention group and a slope of 0.0230 for the control group. Finally, we utilize a random forest algorithm to perform leave-one-subject-out prediction of intervention versus control subjects based on digital marker delta values. The random forest predicts whether the subject is in the intervention or control group with an accuracy of 0.87. This work has implications for capturing objective, continuous data to inform our understanding of intervention adoption and impact.

PIP: Pictorial Interpretable Prototype Learning for Time Series Classification.

Time series classifiers are not only challenging to design, but they are also notoriously difficult to deploy for critical applications because end users may not understand or trust black-box models. Despite new efforts, explanations generated by other interpretable time series models are complicated for non-engineers to understand. The goal of PIP is to provide time series explanations that are tailored toward specific end users. To address the challenge, this paper introduces PIP, a novel deep learning architecture that jointly learns classification models and meaningful visual class prototypes. PIP allows users to train the model on their choice of class illustrations. Thus, PIP can create a user-friendly explanation by leaning on end-users definitions. We hypothesize that a pictorial description is an effective way to communicate a learned concept to non-expert users. Based on an end-user experiment with participants from multiple backgrounds, PIP offers an improved combination of accuracy and interpretability over baseline methods for time series classification.

Multi-Person Activity Recognition in Continuously Monitored Smart Homes.

Activity recognizers are challenging to design for continuous, in-home settings. However, they are notoriously difficult to create when there is more than one resident in the home. Despite recent efforts, there remains a need for an algorithm that can estimate the number of residents in the house, split a time series stream into separate substreams, and accurately identify each resident's activities. To address this challenge, we introduce Gamut. This novel unsupervised method jointly estimates the number of residents and associates sensor readings with those residents, based on a multi-target Gaussian mixture probability hypothesis density filter. We hypothesize that the proposed method will offer robust recognition for homes with two or more residents. In experiments with labeled data collected from 50 single-resident and 11 multi-resident homes, we observe that Gamut outperforms previous unsupervised and supervised methods, offering a robust strategy to track behavioral routines in complex settings.

Nurse-in-the-loop smart home detection of health events associated with diagnosed chronic conditions: A case-event series.

Background: Telehealth and home-based care options significantly expanded during the SARS-CoV2 pandemic. Sophisticated, remote monitoring technologies now exist that support at-home care. Advances in the research of smart homes for health monitoring have shown these technologies are capable of recognizing and predicting health changes in near-real time. However, few nurses are familiar enough with this technology to use smart homes for optimizing patient care or expanding their reach into the home between healthcare touch points. Objective: The objective of this work is to explore a partnership between nurses and smart homes for automated remote monitoring and assessing of patient health. We present a series of health event cases to demonstrate how this partnership may be harnessed to effectively detect and report on clinically relevant health events that can be automatically detected by smart homes. Participants: 25 participants with multiple chronic health conditions. Methods: Ambient sensors were installed in the homes of 25 participants with multiple chronic health conditions. Motion, light, temperature, and door usage data were continuously collected from participants' homes. Descriptions of health events and participants' associated behaviors were captured via weekly nursing telehealth visits with study participants and used to analyze sensor data representing health events. Two cases of participants with congestive heart failure exacerbations, one case of urinary tract infection, two cases of bowel inflammation flares, and four cases of participants with sleep interruption were explored. Results: For each case, clinically relevant health events aligned with changes from baseline in behavior data patterns derived from sensors installed in the participant's home. In some cases, the detected event was precipitated by additional behavior patterns that could be used to predict the event. Conclusions: We found evidence in this case series that continuous sensor-based monitoring of patient behavior in home settings may be used to provide automated detection of health events. Nursing insights into smart home sensor data could be used to initiate preventive strategies and provide timely intervention. Tweetable abstract: Nurses partnered with smart homes could detect exacerbations of health conditions at home leading to early intervention.

The Alberta moving beyond breast cancer (AMBER) cohort study: baseline description of the full cohort.

PURPOSE: The Alberta Moving Beyond Breast Cancer (AMBER) Study is an ongoing prospective cohort study investigating how direct measures of physical activity (PA), sedentary behavior (SB), and health-related fitness (HRF) are associated with survival after breast cancer. METHODS: Women in Alberta with newly diagnosed stage I (≥ T1c) to IIIc breast cancer were recruited between 2012 and 2019. Baseline assessments were completed within 90 days of surgery. Measurements included accelerometers to measure PA and SB; a graded treadmill test with gas exchange analysis to measure cardiorespiratory fitness (VO2peak); upper and lower body muscular strength and endurance; dual-X-ray absorptiometry to measure body composition; and questionnaires to measure self-reported PA and SB. RESULTS: At baseline, the 1528 participants' mean age was 56 ± 11 years, 59% were post-menopausal, 62% had overweight/obesity, and 55% were diagnosed with stage II or III disease. Based on device measurements, study participants spent 8.9 ± 1.7 h/day sedentary, 4.4 ± 1.2 h/day in light-intensity activity, 0.9 ± 0.5 h/day in moderate-intensity activity, and 0.2 ± 0.2 h/day in vigorous-intensity activity. For those participants who reached VO2peak, the average aerobic fitness level was 26.6 ± 6 ml/kg/min. Average body fat was 43 ± 7.1%. CONCLUSION: We have established a unique cohort of breast cancer survivors with a wealth of data on PA, SB, and HRF obtained through both direct and self-reported measurements. Study participants are being followed for at least ten years to assess all outcomes after breast cancer. These data will inform clinical and public health guidelines on PA, SB, and HRF for improving breast cancer outcomes.

Memory-Aware Active Learning in Mobile Sensing Systems.

We propose a novel active learning framework for activity recognition using wearable sensors. Our work is unique in that it takes limitations of the oracle into account when selecting sensor data for annotation by the oracle. Our approach is inspired by human-beings' limited capacity to respond to prompts on their mobile device. This capacity constraint is manifested not only in the number of queries that a person can respond to in a given time-frame but also in the time lag between the query issuance and the oracle response. We introduce the notion of mindful active learning and propose a computational framework, called EMMA, to maximize the active learning performance taking informativeness of sensor data, query budget, and human memory into account. We formulate this optimization problem, propose an approach to model memory retention, discuss the complexity of the problem, and propose a greedy heuristic to solve the optimization problem. Additionally, we design an approach to perform mindful active learning in batch where multiple sensor observations are selected simultaneously for querying the oracle. We demonstrate the effectiveness of our approach using three publicly available activity datasets and by simulating oracles with various memory strengths. We show that the activity recognition accuracy ranges from 21% to 97% depending on memory strength, query budget, and difficulty of the machine learning task. Our results also indicate that EMMA achieves an accuracy level that is, on average, 13.5% higher than the case when only informativeness of the sensor data is considered for active learning. Moreover, we show that the performance of our approach is at most 20% less than the experimental upper-bound and up to 80% higher than the experimental lower-bound. To evaluate the performance of EMMA for batch active learning, we design two instantiations of EMMA to perform active learning in batch mode. We show that these algorithms improve the algorithm training time at the cost of a reduced accuracy in performance. Another finding in our work is that integrating clustering into the process of selecting sensor observations for batch active learning improves the activity learning performance by 11.1% on average, mainly due to reducing the redundancy among the selected sensor observations. We observe that mindful active learning is most beneficial when the query budget is small and/or the oracle's memory is weak. This observation emphasizes advantages of utilizing mindful active learning strategies in mobile health settings that involve interaction with older adults and other populations with cognitive impairments.

Use of machine learning to predict medication adherence in individuals at risk for atherosclerotic cardiovascular disease.

Background: Medication nonadherence is a critical problem with severe implications in individuals at risk for atherosclerotic cardiovascular disease. Many studies have attempted to predict medication adherence in this population, but few, if any, have been effective in prediction, sug-gesting that essential risk factors remain unidentified. Objective: This study's objective was to (1) establish an accurate prediction model of medi-cation adherence in individuals at risk for atherosclerotic cardiovascular disease and (2) identify significant contributing factors to the predictive accuracy of medication adherence. In particular, we aimed to use only the baseline questionnaire data to assess medication adherence prediction feasibility. Methods: A sample of 40 individuals at risk for atherosclerotic cardiovascular disease was recruited for an eight-week feasibility study. After collecting baseline data, we recorded data from a pillbox that sent events to a cloud-based server. Health measures and medication use events were analyzed using machine learning algorithms to identify variables that best predict medication adherence. Results: Our adherence prediction model, based on only the ten most relevant variables, achieved an average error rate of 12.9%. Medication adherence was closely correlated with being encouraged to play an active role in their treatment, having confidence about what to do in an emergency, knowledge about their medications, and having a special person in their life. Conclusions: Our results showed the significance of clinical and psychosocial factors for predicting medication adherence in people at risk for atherosclerotic cardiovascular diseases. Clini-cians and researchers can use these factors to stratify individuals to make evidence-based decisions to reduce the risks.

Partnering a Compensatory Application with Activity-Aware Prompting to Improve Use in Individuals with Amnestic Mild Cognitive Impairment: A Randomized Controlled Pilot Clinical Trial.

BACKGROUND: Compensatory aids can help mitigate the impact of progressive cognitive impairment on daily living. OBJECTIVE: We evaluate whether the learning and sustained use of an Electronic Memory and Management Aid (EMMA) application can be augmented through a partnership with real-time, activity-aware transition-based prompting delivered by a smart home. METHODS: Thirty-two adults who met criteria for amnestic mild cognitive impairment (aMCI) were randomized to learn to use the EMMA app on its own (N = 17) or when partnered with smart home prompting (N = 15). The four-week, five-session manualized EMMA training was conducted individually in participant homes by trained clinicians. Monthly questionnaires were completed by phone with trained personnel blind to study hypotheses. EMMA data metrics were collected continuously for four months. For the partnered condition, activity-aware prompting was on during training and post-training months 1 and 3, and off during post-training month 2. RESULTS: The analyzed aMCI sample included 15 EMMA-only and 14 partnered. Compared to the EMMA-only condition, by week four of training, participants in the partnered condition were engaging with EMMA more times daily and using more basic and advanced features. These advantages were maintained throughout the post-training phase with less loss of EMMA app use over time. There was little differential impact of the intervention on self-report primary (everyday functioning, quality of life) and secondary (coping, satisfaction with life) outcomes. CONCLUSION: Activity-aware prompting technology enhanced acquisition, habit formation and long-term use of a digital device by individuals with aMCI. (ClinicalTrials.gov NCT03453554).

Remote Monitoring of the Performance Status and Burden of Symptoms of Patients With Gastrointestinal Cancer Via a Consumer-Based Activity Tracker: Quantitative Cohort Study.

BACKGROUND: The number of older patients with gastrointestinal cancer is increasing due to an aging global population. Minimizing reliance on an in-clinic patient performance status test to determine a patient's prognosis and course of treatment can improve resource utilization. Further, current performance status measurements cannot capture patients' constant changes. These measurements also rely on self-reports, which are subjective and subject to bias. Real-time monitoring of patients' activities may allow for a more accurate assessment of patients' performance status while minimizing resource utilization. OBJECTIVE: This study investigates the validity of consumer-based activity trackers for monitoring the performance status of patients with gastrointestinal cancer. METHODS: A total of 27 consenting patients (63% male, median age 58 years) wore a consumer-based activity tracker 7 days before chemotherapy and 14 days after receiving their first treatment. The provider assessed patients using the Eastern Cooperative Oncology Group Performance Status (ECOG-PS) scale and Memorial Symptom Assessment Scale-Short Form (MSAS-SF) before and after chemotherapy visits. The statistical correlations between ECOG-PS and MSAS-SF scores and patients' daily step counts were assessed. RESULTS: The daily step counts yielded the highest correlation with the patients' ECOG-PS scores after chemotherapy (P<.001). The patients with higher ECOG-PS scores experienced a higher fluctuation in their step counts. The patients who walked more prechemotherapy (mean 6071 steps per day) and postchemotherapy (mean 5930 steps per day) had a lower MSAS-SF score (lower burden of symptoms) compared to patients who walked less prechemotherapy (mean 5205 steps per day) and postchemotherapy (mean 4437 steps per day). CONCLUSIONS: This study demonstrates the feasibility of using inexpensive, consumer-based activity trackers for the remote monitoring of performance status in the gastrointestinal cancer population. The findings need to be validated in a larger population for generalizability.

A Survey of Deep Network Techniques All Classifiers Can Adopt.

Deep neural networks (DNNs) have introduced novel and useful tools to the machine learning community. Other types of classifiers can potentially make use of these tools as well to improve their performance and generality. This paper reviews the current state of the art for deep learning classifier technologies that are being used outside of deep neural networks. Non-neural network classifiers can employ many components found in DNN architectures. In this paper, we review the feature learning, optimization, and regularization methods that form a core of deep network technologies. We then survey non-neural network learning algorithms that make innovative use of these methods to improve classification performance. Because many opportunities and challenges still exist, we discuss directions that can be pursued to expand the area of deep learning for a variety of classification algorithms.

Indirectly-Supervised Anomaly Detection of Clinically-Meaningful Health Events from Smart Home Data.

Anomaly detection techniques can extract a wealth of information about unusual events. Unfortunately, these methods yield an abundance of findings that are not of interest, obscuring relevant anomalies. In this work, we improve upon traditional anomaly detection methods by introducing Isudra, an Indirectly-Supervised Detector of Relevant Anomalies from time series data. Isudra employs Bayesian optimization to select time scales, features, base detector algorithms, and algorithm hyperparameters that increase true positive and decrease false positive detection. This optimization is driven by a small amount of example anomalies, driving an indirectly-supervised approach to anomaly detection. Additionally, we enhance the approach by introducing a warm start method that reduces optimization time between similar problems. We validate the feasibility of Isudra to detect clinically-relevant behavior anomalies from over 2 million sensor readings collected in 5 smart homes, reflecting 26 health events. Results indicate that indirectly-supervised anomaly detection outperforms both supervised and unsupervised algorithms at detecting instances of health-related anomalies such as falls, nocturia, depression, and weakness.

Fusing Ambient and Mobile Sensor Features Into a Behaviorome for Predicting Clinical Health Scores.

Advances in machine learning and low-cost, ubiquitous sensors offer a practical method for understanding the predictive relationship between behavior and health. In this study, we analyze this relationship by building a behaviorome, or set of digital behavior markers, from a fusion of data collected from ambient and wearable sensors. We then use the behaviorome to predict clinical scores for a sample of n = 21 participants based on continuous data collected from smart homes and smartwatches and automatically labeled with corresponding activity and location types. To further investigate the relationship between domains, including participant demographics, self-report and external observation-based health scores, and behavior markers, we propose a joint inference technique that improves predictive performance for these types of high-dimensional spaces. For our participant sample, we observe correlations ranging from small to large for the clinical scores. We also observe an improvement in predictive performance when multiple sensor modalities are used and when joint inference is employed.