A systems approach towards remote health-monitoring in older adults: Introducing a zero-interaction digital exhaust.

Using connected sensing devices to remotely monitor health is a promising way to help transition healthcare from a rather reactive to a more precision medicine oriented proactive approach, which could be particularly relevant in the face of rapid population ageing and the challenges it poses to healthcare systems. Sensor derived digital measures of health, such as digital biomarkers or digital clinical outcome assessments, may be used to monitor health status or the risk of adverse events like falls. Current research around such digital measures has largely focused on exploring the use of few individual measures obtained through mobile devices. However, especially for long-term applications in older adults, this choice of technology may not be ideal and could further add to the digital divide. Moreover, large-scale systems biology approaches, like genomics, have already proven beneficial in precision medicine, making it plausible that the same could also hold for remote-health monitoring. In this context, we introduce and describe a zero-interaction digital exhaust: a set of 1268 digital measures that cover large parts of a person's activity, behavior and physiology. Making this approach more inclusive of older adults, we base this set entirely on contactless, zero-interaction sensing technologies. Applying the resulting digital exhaust to real-world data, we then demonstrate the possibility to create multiple ageing relevant digital clinical outcome assessments. Paired with modern machine learning, we find these assessments to be surprisingly powerful and often on-par with mobile approaches. Lastly, we highlight the possibility to discover novel digital biomarkers based on this large-scale approach.

A Sensor-Driven Visit Detection System in Older Adults' Homes: Towards Digital Late-Life Depression Marker Extraction.

Modern sensor technology is increasingly used in older adults to not only provide additional safety but also to monitor health status, often by means of sensor derived digital measures or biomarkers. Social isolation is a known risk factor for late-life depression, and a potential component of social-isolation is the lack of home visits. Therefore, home visits may serve as a digital measure for social isolation and late-life depression. Late-life depression is a common mental and emotional disorder in the growing population of older adults. The disorder, if untreated, can significantly decrease quality of life and, amongst other effects, leads to increased mortality. Late-life depression often goes undiagnosed due to associated stigma and the incorrect assumption that it is a normal part of ageing. In this work, we propose a visit detection system that generalizes well to previously unseen apartments - which may differ largely in layout, sensor placement, and size from apartments found in the semi-annotated training dataset. We find that by using a self-training-based domain adaptation strategy, a robust system to extract home visit information can be built (ROC AUC = 0.773). We further show that the resulting visit information correlates well with the common geriatric depression scale screening tool ( ρ = -0.87, p = 0.001), providing further support for the idea of utilizing the extracted information as a potential digital measure or even as a digital biomarker to monitor the risk of late-life depression.

Contactless Sleep Monitoring for Early Detection of Health Deteriorations in Community-Dwelling Older Adults: Exploratory Study.

BACKGROUND: Population aging is posing multiple social and economic challenges to society. One such challenge is the social and economic burden related to increased health care expenditure caused by early institutionalizations. The use of modern pervasive computing technology makes it possible to continuously monitor the health status of community-dwelling older adults at home. Early detection of health issues through these technologies may allow for reduced treatment costs and initiation of targeted preventive measures leading to better health outcomes. Sleep is a key factor when it comes to overall health and many health issues manifest themselves with associated sleep deteriorations. Sleep quality and sleep disorders such as sleep apnea syndrome have been extensively studied using various wearable devices at home or in the setting of sleep laboratories. However, little research has been conducted evaluating the potential of contactless and continuous sleep monitoring in detecting early signs of health problems in community-dwelling older adults. OBJECTIVE: In this work we aim to evaluate which contactlessly measurable sleep parameter is best suited to monitor perceived and actual health status changes in older adults. METHODS: We analyzed real-world longitudinal (up to 1 year) data from 37 community-dwelling older adults including more than 6000 nights of measured sleep. Sleep parameters were recorded by a pressure sensor placed beneath the mattress, and corresponding health status information was acquired through weekly questionnaires and reports by health care personnel. A total of 20 sleep parameters were analyzed, including common sleep metrics such as sleep efficiency, sleep onset delay, and sleep stages but also vital signs in the form of heart and breathing rate as well as movements in bed. Association with self-reported health, evaluated by EuroQol visual analog scale (EQ-VAS) ratings, were quantitatively evaluated using individual linear mixed-effects models. Translation to objective, real-world health incidents was investigated through manual retrospective case-by-case analysis. RESULTS: Using EQ-VAS rating based self-reported perceived health, we identified body movements in bed-measured by the number toss-and-turn events-as the most predictive sleep parameter (t score=-0.435, P value [adj]=<.001). Case-by-case analysis further substantiated this finding, showing that increases in number of body movements could often be explained by reported health incidents. Real world incidents included heart failure, hypertension, abdominal tumor, seasonal flu, gastrointestinal problems, and urinary tract infection. CONCLUSIONS: Our results suggest that nightly body movements in bed could potentially be a highly relevant as well as easy to interpret and derive digital biomarker to monitor a wide range of health deteriorations in older adults. As such, it could help in detecting health deteriorations early on and provide timelier, more personalized, and precise treatment options.

Ten seconds of my nights: Exploring methods to measure brightness, loudness and attendance and their associations with alcohol use from video clips.

INTRODUCTION: Most evidence on associations between alcohol use behaviors and the characteristics of its social and physical context is based on self-reports from study participants and, thus, only account for their subjective impressions of the situation. This study explores the feasibility of obtaining alternative measures of loudness, brightness, and attendance (number of people) using 10-second video clips of real-life drinking occasions rated by human annotators and computer algorithms, and explores the associations of these measures with participants' choice to drink alcohol or not. METHODS: Using a custom-built smartphone application, 215 16-25-year-olds documented characteristics of 2,380 weekend night drinking events using questionnaires and videos. Ratings of loudness, brightness, and attendance were obtained from three sources, namely in-situ participants' ratings, video-based annotator ratings, and video-based computer algorithm ratings. Bivariate statistics explored differences in ratings across sources. Multilevel logistic regressions assessed the associations of contextual characteristics with alcohol use. Finally, model fit indices and cross-validation were used to assess the ability of each set of contextual measures to predict participants' alcohol use. RESULTS: Raw ratings of brightness, loudness and attendance differed slightly across sources, but were all correlated (r = .21 to .82, all p < .001). Participants rated bars/pubs as being louder (Cohen's d = 0.50 [95%-CI: 0.07-0.92]), and annotators rated private places as darker (d = 1.21 [95%-CI: 0.99-1.43]) when alcohol was consumed than when alcohol was not consumed. Multilevel logistic regressions showed that drinking in private places was more likely in louder (ORparticipants = 1.74 [CI: 1.31-2.32]; ORannotators = 3.22 [CI: 2.06-5.03]; ORalgorithm = 2.62 [CI: 1.83-3.76]), more attended (ORparticipants = 1.10 [CI: 1.03-1.18]; ORalgorithm = 1.19 [CI: 1.07-1.32]) and darker (OR = 0.64 [CI: 0.44-0.94]) situations. In commercial venues, drinking was more likely in darker (ORparticipants = 0.67 [CI: 0.47-0.94]; ORannotators = 0.53 [CI: 0.33-0.85]; ORalgorithm = 0.58 [CI: 0.37-0.88]) and louder (ORparticipants = 1.40 [CI: 1.02-1.92]; ORalgorithm = 2.45 [CI: 1.25-4.80]) places. Higher inference accuracies were found for the models based on the annotators' ratings (80% to 84%) and the algorithms' ratings (76% to 86%) than on the participants' ratings (69% to 71%). CONCLUSIONS: Several contextual characteristics are associated with increased odds of drinking in private and commercial settings, and might serve as a basis for the development of prevention measures. Regarding assessment of contextual characteristics, annotators and algorithms might serve as appropriate substitutes of participants' in-situ impressions for correlational and regression analyses despite differences in raw ratings. Collecting contextual data by means of sensors or media files is recommended for future research.

Shooting shots: Estimating alcoholic drink sizes in real life using event-level reports and annotations of close-up pictures.

INTRODUCTION AND AIMS: Drinks consumed in real life are diverse, in terms of beverage type, container size and alcohol by volume. To date, most ecological momentary assessment studies have assessed drinking amounts with 'standard' drinks, although their event-level design allows for more advanced assessment schemes. The purpose of this empirical study is to compare participants' estimates of alcoholic drink characteristics, assessed using drink-specific questions, with estimates generated by annotators based on pictures of the same drinks. DESIGN AND METHODS: On weekend nights, 186 young adults took 1484 close-up pictures of their drinks using a custom-built smartphone application. Participants reported the beverage type, drink size and alcohol by volume. Annotators described the beverage type, container size and filling level. Correspondence between participants' and annotators' estimates was explored using descriptive statistics, difference tests and correlations. RESULTS: Annotators were unable to precisely identify the beverage types in most pictures of liqueurs, spirits and mixed drinks. Participants' drink size estimates converged with annotators' estimates of the container size for beer (41 cl corresponding to 16 g of pure alcohol) and mixed drinks (28 cl/35 g), and of the content size for wine (10 cl/9 g). However, annotators estimated larger sizes for liqueur/fortified wine (12 cl/14 g vs. 7 cl/9 g) and spirits (8 cl/26 g vs. 4 cl/10 g) than participants. DISCUSSION AND CONCLUSIONS: Annotations of pictures should be considered as a complement to participants' reports rather than a substitute. Except for wine, real-life drinks vary largely and often exceed 10 g 'standard' drinks.

Evaluation of 1-Year in-Home Monitoring Technology by Home-Dwelling Older Adults, Family Caregivers, and Nurses.

Introduction: Population aging is increasing the needs and costs of healthcare. Both frailty and the chronic diseases affecting older people reduce their ability to live independently. However, most older people prefer to age in their own homes. New development of in-home monitoring can play a role in staying independent, active, and healthy for older people. This 12-month observational study aimed to evaluate a new in-home monitoring system among home-dwelling older adults (OA), their family caregivers (FC), and nurses for the support of home care. Methods: The in-home monitoring system evaluated in this study continuously monitored OA's daily activities (e.g., mobility, sleep habits, fridge visits, door events) by ambient sensor system (DomoCare®) and health-related events by wearable sensors (Activity tracker, ECG). In the case of deviations in daily activities, alerts were transmitted to nurses via email. Using specific questionnaires, the opinions of 13 OA, 13 FC, and 20 nurses were collected at the end of 12-months follow-up focusing on user experience and the impact of in-home monitoring on home care services. Results: The majority of OA, FC, and nurses considered that in-home sensors can help with staying at home, improving home care and quality of life, preventing domestic accidents, and reducing family stress. The opinion tended to be more frequently favorable toward ambient sensors (76%; 95% CI: 61-87%) than toward wearable sensors (Activity tracker: 65%; 95% CI: 50-79%); ECG: 60%; 95% CI: 45-75%). On average, OA (74%; 95% CI: 46-95%) and FC (70%; 95% CI: 39-91%) tended to be more enthusiastic than nurses (60%; 95% CI: 36-81%). Some barriers reported by nurses were a fear of weakening of the relationship with OA and lack of time. Discussion/Conclusion: Overall, the opinions of OA, FC, and nurses were positively related to in-home sensors, with nurses being less enthusiastic about their use in clinical practice.

Validity of pervasive computing based continuous physical activity assessment in community-dwelling old and oldest-old.

In older adults, physical activity is crucial for healthy aging and associated with numerous health indicators and outcomes. Regular assessments of physical activity can help detect early health-related changes and manage physical activity targeted interventions. The quantification of physical activity, however, is difficult as commonly used self-reported measures are biased and rather unprecise point in time measurements. Modern alternatives are commonly based on wearable technologies which are accurate but suffer from usability and compliance issues. In this study, we assessed the potential of an unobtrusive ambient-sensor based system for continuous, long-term physical activity quantification. Towards this goal, we analysed one year of longitudinal sensor- and medical-records stemming from thirteen community-dwelling old and oldest old subjects. Based on the sensor data the daily number of room-transitions as well as the raw sensor activity were calculated. We did find the number of room-transitions, and to some degree also the raw sensor activity, to capture numerous known associations of physical activity with cognitive, well-being and motor health indicators and outcomes. The results of this study indicate that such low-cost unobtrusive ambient-sensor systems can provide an adequate approximation of older adults' overall physical activity, sufficient to capture relevant associations with health indicators and outcomes.

Emergent Power Hierarchies and Group Performance.

In newly formed groups, informal hierarchies emerge automatically and readily. In this study, we argue that emergent group hierarchies enhance group performance (Hypothesis 1) and we assume that the more the power hierarchy within a group corresponds to the task-competence differences of the individual group members, the better the group performs (Hypothesis 2). Twelve three-person groups and 28 four-person groups were investigated while solving the Winter Survival Task. Results show that emerging power hierarchies positively impact group performance but the alignment between task-competence and power hierarchy did not affect group performance. Thus, emergent power hierarchies are beneficial for group performance and although they were on average created around individual group members' competence, this correspondence was not a prerequisite for better group performance.

Tracking the visual focus of attention for a varying number of wandering people.

We define and address the problem of finding the visual focus of attention for a varying number of wandering people (VFOA-W), determining where the people's movement is unconstrained. VFOA-W estimation is a new and important problem with mplications for behavior understanding and cognitive science, as well as real-world applications. One such application, which we present in this article, monitors the attention passers-by pay to an outdoor advertisement. Our approach to the VFOA-W problem proposes a multi-person tracking solution based on a dynamic Bayesian network that simultaneously infers the (variable) number of people in a scene, their body locations, their head locations, and their head pose. For efficient inference in the resulting large variable-dimensional state-space we propose a Reversible Jump Markov Chain Monte Carlo (RJMCMC) sampling scheme, as well as a novel global observation model which determines the number of people in the scene and localizes them. We propose a Gaussian Mixture Model (GMM) and Hidden Markov Model (HMM)-based VFOA-W model which use head pose and location information to determine people's focus state. Our models are evaluated for tracking performance and ability to recognize people looking at an outdoor advertisement, with results indicating good performance on sequences where a moderate number of people pass in front of an advertisement.

Modeling semantic aspects for cross-media image indexing.

To go beyond the query-by-example paradigm in image retrieval, there is a need for semantic indexing of large image collections for intuitive text-based image search. Different models have been proposed to learn the dependencies between the visual content of an image set and the associated text captions, then allowing for the automatic creation of semantic indices for unannotated images. The task, however, remains unsolved. In this paper, we present three alternatives to learn a Probabilistic Latent Semantic Analysis model (PLSA) for annotated images, and evaluate their respective performance for automatic image indexing. Under the PLSA assumptions, an image is modeled as a mixture of latent aspects that generates both image features and text captions, and we investigate three ways to learn the mixture of aspects. We also propose a more discriminative image representation than the traditional Blob histogram, concatenating quantized local color information and quantized local texture descriptors. The first learning procedure of a PLSA model for annotated images is a standard EM algorithm, which implicitly assumes that the visual and the textual modalities can be treated equivalently. The other two models are based on an asymmetric PLSA learning, allowing to constrain the definition of the latent space on the visual or on the textual modality. We demonstrate that the textual modality is more appropriate to learn a semantically meaningful latent space, which translates into improved annotation performance. A comparison of our learning algorithms with respect to recent methods on a standard dataset is presented, and a detailed evaluation of the performance shows the validity of our framework.

A thousand words in a scene.

This paper presents a novel approach for visual scene modeling and classification, investigating the combined use of text modeling methods and local invariant features. Our work attempts to elucidate (1) whether a text-like bag-of-visterms representation (histogram of quantized local visual features) is suitable for scene (rather than object) classification, (2) whether some analogies between discrete scene representations and text documents exist, and (3) whether unsupervised, latent space models can be used both as feature extractors for the classification task and to discover patterns of visual co-occurrence. Using several data sets, we validate our approach, presenting and discussing experiments on each of these issues. We first show, with extensive experiments on binary and multi-class scene classification tasks using a 9,500-image data set, that the bag-of-visterms representation consistently outperforms classical scene classification approaches. In other data sets we show that our approach competes with or outperforms other recent, more complex, methods. We also show that Probabilistic Latent Semantic Analysis (PLSA) generates a compact scene representation, discriminative for accurate classification, and more robust than the bag-of-visterms representation when less labeled training data is available. Finally, through aspect-based image ranking experiments, we show the ability of PLSA to automatically extract visually meaningful scene patterns, making such representation useful for browsing image collections.

Embedding motion in model-based stochastic tracking.

Particle filtering is now established as one of the most popular methods for visual tracking. Within this framework, there are two important considerations. The first one refers to the generic assumption that the observations are temporally independent given the sequence of object states. The second consideration, often made in the literature, uses the transition prior as the proposal distribution. Thus, the current observations are not taken into account, requiring the noise process of this prior to be large enough to handle abrupt trajectory changes. As a result, many particles are either wasted in low likelihood regions of the state space, resulting in low sampling efficiency, or more importantly, propagated to distractor regions of the image, resulting in tracking failures. In this paper, we propose to handle both considerations using motion. We first argue that, in general, observations are conditionally correlated, and propose a new model to account for this correlation, allowing for the natural introduction of implicit and/or explicit motion measurements in the likelihood term. Second, explicit motion measurements are used to drive the sampling process towards the most likely regions of the state space. Overall, the proposed model handles abrupt motion changes and filters out visual distractors, when tracking objects with generic models based on shape or color distribution. Results were obtained on head tracking experiments using several sequences with moving camera involving large dynamics. When compared against the Condensation Algorithm, they have demonstrated the superior tracking performance of our approach.

Automatic analysis of multimodal group actions in meetings.

This paper investigates the recognition of group actions in meetings. A framework is employed in which group actions result from the interactions of the individual participants. The group actions are modeled using different HMM-based approaches, where the observations are provided by a set of audiovisual features monitoring the actions of individuals. Experiments demonstrate the importance of taking interactions into account in modeling the group actions. It is also shown that the visual modality contains useful information, even for predominantly audio-based events, motivating a multimodal approach to meeting analysis.