Quantifying impairment and disease severity using AI models trained on healthy subjects.

Automatic assessment of impairment and disease severity is a key challenge in data-driven medicine. We propose a framework to address this challenge, which leverages AI models trained exclusively on healthy individuals. The COnfidence-Based chaRacterization of Anomalies (COBRA) score exploits the decrease in confidence of these models when presented with impaired or diseased patients to quantify their deviation from the healthy population. We applied the COBRA score to address a key limitation of current clinical evaluation of upper-body impairment in stroke patients. The gold-standard Fugl-Meyer Assessment (FMA) requires in-person administration by a trained assessor for 30-45 minutes, which restricts monitoring frequency and precludes physicians from adapting rehabilitation protocols to the progress of each patient. The COBRA score, computed automatically in under one minute, is shown to be strongly correlated with the FMA on an independent test cohort for two different data modalities: wearable sensors (ρ = 0.814, 95% CI [0.700,0.888]) and video (ρ = 0.736, 95% C.I [0.584, 0.838]). To demonstrate the generalizability of the approach to other conditions, the COBRA score was also applied to quantify severity of knee osteoarthritis from magnetic-resonance imaging scans, again achieving significant correlation with an independent clinical assessment (ρ = 0.644, 95% C.I [0.585,0.696]).

Feasibility of a Self-directed Upper Extremity Training Program to Promote Actual Arm Use for Individuals Living in the Community With Chronic Stroke.

Objective: To determine the feasibility of a self-directed training protocol to promote actual arm use in everyday life. The secondary aim was to explore the initial efficacy on upper extremity (UE) outcome measures. Design: Feasibility study using multiple methods. Setting: Home and outpatient research lab. Participants: Fifteen adults (6 women, 9 men, mean age=53.08 years) with chronic stroke living in the community. There was wide range of UE functional levels, ranging from dependent stabilizer (limited function) to functional assist (high function). Intervention: Use My Arm-Remote protocol. Phase 1 consisted of clinician training on motivational interviewing (MI). Phase 2 consisted of MI sessions with participants to determine participant generated goals, training activities, and training schedules. Phase 3 consisted of UE task-oriented training (60 minutes/day, 5 days/week, for 4 weeks). Participants received daily surveys through an app to monitor arm training behavior and weekly virtual check-ins with clinicians to problem-solve challenges and adjust treatment plans. Outcome Measures: Primary outcome measures were feasibility domains after intervention, measured by quantitative study data and qualitative semi-structured interviews. Secondary outcomes included the Canadian Occupational Performance Measure (COPM), Motor Activity Log (MAL), Fugl-Meyer Assessment (FMA), and accelerometry-based duration of use metric measured at baseline, discharge, and 4-week follow-up. Results: The UMA-R was feasible in the following domains: recruitment rate, retention rate, intervention acceptance, intervention delivery, adherence frequency, and safety. Adherence to duration of daily practice did not meet our criteria. Improvements in UE outcomes were achieved at discharge and maintained at follow-up as measured by COPM-Performance subscale (F[1.42, 19.83]=17.72, P<.001) and COPM-Satisfaction subscale (F[2, 28]=14.73, P<.001), MAL (F[1.31, 18.30]=12.05, P<.01) and the FMA (F[2, 28]=16.62, P<.001). Conclusion: The UMA-R was feasible and safe to implement for individuals living in the community with chronic stroke. Adherence duration was identified as area of refinement. Participants demonstrated improvements in standardized UE outcomes to support initial efficacy of the UMA-R. Shared decision-making and behavior change frameworks can support the implementation of UE self-directed rehabilitation. Our results warrant the refinement and further testing of the UMA-R.

Quantifying Impairment and Disease Severity Using AI Models Trained on Healthy Subjects.

Automatic assessment of impairment and disease severity is a key challenge in data-driven medicine. We propose a novel framework to address this challenge, which leverages AI models trained exclusively on healthy individuals. The COnfidence-Based chaRacterization of Anomalies (COBRA) score exploits the decrease in confidence of these models when presented with impaired or diseased patients to quantify their deviation from the healthy population. We applied the COBRA score to address a key limitation of current clinical evaluation of upper-body impairment in stroke patients. The gold-standard Fugl-Meyer Assessment (FMA) requires in-person administration by a trained assessor for 30-45 minutes, which restricts monitoring frequency and precludes physicians from adapting rehabilitation protocols to the progress of each patient. The COBRA score, computed automatically in under one minute, is shown to be strongly correlated with the FMA on an independent test cohort for two different data modalities: wearable sensors ($\rho = 0.845$, 95% CI [0.743,0.908]) and video ($\rho = 0.746$, 95% C.I [0.594, 0.847]). To demonstrate the generalizability of the approach to other conditions, the COBRA score was also applied to quantify severity of knee osteoarthritis from magnetic-resonance imaging scans, again achieving significant correlation with an independent clinical assessment ($\rho = 0.644$, 95% C.I [0.585,0.696]).

Data-Driven Quantitation of Movement Abnormality after Stroke.

Stroke commonly affects the ability of the upper extremities (UEs) to move normally. In clinical settings, identifying and measuring movement abnormality is challenging due to the imprecision and impracticality of available assessments. These challenges interfere with therapeutic tracking, communication, and treatment. We thus sought to develop an approach that blends precision and pragmatism, combining high-dimensional motion capture with out-of-distribution (OOD) detection. We used an array of wearable inertial measurement units to capture upper body motion in healthy and chronic stroke subjects performing a semi-structured, unconstrained 3D tabletop task. After data were labeled by human coders, we trained two deep learning models exclusively on healthy subject data to classify elemental movements (functional primitives). We tested these healthy subject-trained models on previously unseen healthy and stroke motion data. We found that model confidence, indexed by prediction probabilities, was generally high for healthy test data but significantly dropped when encountering OOD stroke data. Prediction probabilities worsened with more severe motor impairment categories and were directly correlated with individual impairment scores. Data inputs from the paretic UE, rather than trunk, most strongly influenced model confidence. We demonstrate for the first time that using OOD detection with high-dimensional motion data can reveal clinically meaningful movement abnormality in subjects with chronic stroke.

Dual-Tasking in Daily Activities Among Adults With and Without Stroke.

IMPORTANCE: In laboratory settings, dual-tasking is a performance strategy affected by dominance and stroke. However, the volitional use of dual-tasking has not been examined during naturalistic performance of activities of daily living (ADLs). OBJECTIVE: To examine dual-tasking in the context of ADLs and identify whether dominance and stroke influence its use. DESIGN: Cross-sectional, observational. SETTING: Academic medical center. PARTICIPANTS: Forty-three participants with chronic stroke and upper extremity (UE) motor impairment and 19 control participants without stroke. OUTCOMES AND MEASURES: We identified dual-tasking as the performance of dual-object primitives (DOPs), a functional strategy to manage two objects simultaneously. We videotaped participants performing feeding and toothbrushing tasks and identified the initiation and frequency of DOPs. We assessed whether these outcomes were influenced by UE dominance or paresis and whether among participants with stroke these outcomes were influenced by motor impairment (using the Fugl-Meyer Assessment) or cognitive impairment (using the Montreal Cognitive Assessment). RESULTS: DOP initiation was reduced on the nondominant side of control UEs and in the paretic UE of participants with stroke. After DOPs were initiated, however, their frequency was not significantly related to dominance or paresis. Among participants with stroke, DOP initiation but not DOP frequency was influenced by motor impairment, and neither were influenced by cognitive impairment. CONCLUSIONS AND RELEVANCE: The initiation of dual-tasking is curtailed in the nondominant and paretic UEs, extending previous laboratory-based findings to a more naturalistic setting. These results may reflect a demand on neural resources that is exceeded when these limbs are used. What This Article Adds: DOPs, a functional strategy to simultaneously engage two objects during ADLs, could serve as a behavioral marker of dual-tasking in real-world activities, supporting their investigation more broadly. Practicing DOPs in rehabilitation could also train the integration of dual-tasking strategies in activity execution.

PrimSeq: A deep learning-based pipeline to quantitate rehabilitation training.

Stroke rehabilitation seeks to accelerate motor recovery by training functional activities, but may have minimal impact because of insufficient training doses. In animals, training hundreds of functional motions in the first weeks after stroke can substantially boost upper extremity recovery. The optimal quantity of functional motions to boost recovery in humans is currently unknown, however, because no practical tools exist to measure them during rehabilitation training. Here, we present PrimSeq, a pipeline to classify and count functional motions trained in stroke rehabilitation. Our approach integrates wearable sensors to capture upper-body motion, a deep learning model to predict motion sequences, and an algorithm to tally motions. The trained model accurately decomposes rehabilitation activities into elemental functional motions, outperforming competitive machine learning methods. PrimSeq furthermore quantifies these motions at a fraction of the time and labor costs of human experts. We demonstrate the capabilities of PrimSeq in previously unseen stroke patients with a range of upper extremity motor impairment. We expect that our methodological advances will support the rigorous measurement required for quantitative dosing trials in stroke rehabilitation.

The use of wearable sensors to assess and treat the upper extremity after stroke: a scoping review.

PURPOSE: To address the gap in the literature and clarify the expanding role of wearable sensor data in stroke rehabilitation, we summarized the methods for upper extremity (UE) sensor-based assessment and sensor-based treatment. MATERIALS AND METHODS: The guideline outlined by the preferred reporting items for systematic reviews and meta-analysis extension for scoping reviews was used to complete this scoping review. Information pertaining to participant demographics, sensory information, data collection, data processing, data analysis, and study results were extracted from the studies for analysis and synthesis. RESULTS: We included 43 articles in the final review. We organized the results into assessment and treatment categories. The included articles used wearable sensors to identify UE functional motion, categorize motor impairment/activity limitation, and quantify real-world use. Wearable sensors were also used to augment UE training by triggering sensory cues or providing instructional feedback about the affected UE. CONCLUSIONS: Sensors have the potential to greatly expand assessment and treatment beyond traditional clinic-based approaches. This capability could support the quantification of rehabilitation dose, the nuanced assessment of impairment and activity limitation, the characterization of daily UE use patterns in real-world settings, and augment UE training adherence for home-based rehabilitation.IMPLICATIONS FOR REHABILITATIONSensor data have been used to assess UE functional motion, motor impairment/activity limitation, and real-world use.Sensor-assisted treatment approaches are emerging, and may be a promising tool to augment UE adherence in home-based rehabilitation.Wearable sensors may extend our ability to objectively assess UE motion beyond supervised clinical settings, and into home and community settings.

StrokeRehab: A Benchmark Dataset for Sub-second Action Identification.

Automatic action identification from video and kinematic data is an important machine learning problem with applications ranging from robotics to smart health. Most existing works focus on identifying coarse actions such as running, climbing, or cutting vegetables, which have relatively long durations and a complex series of motions. This is an important limitation for applications that require identification of more elemental motions at high temporal resolution. For example, in the rehabilitation of arm impairment after stroke, quantifying the training dose (number of repetitions) requires differentiating motions with sub-second durations. Our goal is to bridge this gap. To this end, we introduce a large-scale, multimodal dataset, StrokeRehab, as a new action-recognition benchmark that includes elemental short-duration actions labeled at a high temporal resolution. StrokeRehab consists of high-quality inertial measurement unit sensor and video data of 51 stroke-impaired patients and 20 healthy subjects performing activities of daily living like feeding, brushing teeth, etc. Because it contains data from both healthy and impaired individuals, StrokeRehab can be used to study the influence of distribution shift in action-recognition tasks. When evaluated on StrokeRehab, current state-of-the-art models for action segmentation produce noisy predictions, which reduces their accuracy in identifying the corresponding sequence of actions. To address this, we propose a novel approach for high-resolution action identification, inspired by speech-recognition techniques, which is based on a sequence-to-sequence model that directly predicts the sequence of actions. This approach outperforms current state-of-the-art methods on StrokeRehab, as well as on the standard benchmark datasets 50Salads, Breakfast, and Jigsaws.

NE-Motion: Visual Analysis of Stroke Patients Using Motion Sensor Networks.

A large number of stroke survivors suffer from a significant decrease in upper extremity (UE) function, requiring rehabilitation therapy to boost recovery of UE motion. Assessing the efficacy of treatment strategies is a challenging problem in this context, and is typically accomplished by observing the performance of patients during their execution of daily activities. A more detailed assessment of UE impairment can be undertaken with a clinical bedside test, the UE Fugl-Meyer Assessment, but it fails to examine compensatory movements of functioning body segments that are used to bypass impairment. In this work, we use a graph learning method to build a visualization tool tailored to support the analysis of stroke patients. Called NE-Motion, or Network Environment for Motion Capture Data Analysis, the proposed analytic tool handles a set of time series captured by motion sensors worn by patients so as to enable visual analytic resources to identify abnormalities in movement patterns. Developed in close collaboration with domain experts, NE-Motion is capable of uncovering important phenomena, such as compensation while revealing differences between stroke patients and healthy individuals. The effectiveness of NE-Motion is shown in two case studies designed to analyze particular patients and to compare groups of subjects.

Towards data-driven stroke rehabilitation via wearable sensors and deep learning.

Recovery after stroke is often incomplete, but rehabilitation training may potentiate recovery by engaging endogenous neuroplasticity. In preclinical models of stroke, high doses of rehabilitation training are required to restore functional movement to the affected limbs of animals. In humans, however, the necessary dose of training to potentiate recovery is not known. This ignorance stems from the lack of objective, pragmatic approaches for measuring training doses in rehabilitation activities. Here, to develop a measurement approach, we took the critical first step of automatically identifying functional primitives, the basic building block of activities. Forty-eight individuals with chronic stroke performed a variety of rehabilitation activities while wearing inertial measurement units (IMUs) to capture upper body motion. Primitives were identified by human labelers, who labeled and segmented the associated IMU data. We performed automatic classification of these primitives using machine learning. We designed a convolutional neural network model that outperformed existing methods. The model includes an initial module to compute separate embeddings of different physical quantities in the sensor data. In addition, it replaces batch normalization (which performs normalization based on statistics computed from the training data) with instance normalization (which uses statistics computed from the test data). This increases robustness to possible distributional shifts when applying the method to new patients. With this approach, we attained an average classification accuracy of 70%. Thus, using a combination of IMU-based motion capture and deep learning, we were able to identify primitives automatically. This approach builds towards objectively-measured rehabilitation training, enabling the identification and counting of functional primitives that accrues to a training dose.

The Pragmatic Classification of Upper Extremity Motion in Neurological Patients: A Primer.

Recent advances in wearable sensor technology and machine learning (ML) have allowed for the seamless and objective study of human motion in clinical applications, including Parkinson's disease, and stroke. Using ML to identify salient patterns in sensor data has the potential for widespread application in neurological disorders, so understanding how to develop this approach for one's area of inquiry is vital. We previously proposed an approach that combined wearable inertial measurement units (IMUs) and ML to classify motions made by stroke patients. However, our approach had computational and practical limitations. We address these limitations here in the form of a primer, presenting how to optimize a sensor-ML approach for clinical implementation. First, we demonstrate how to identify the ML algorithm that maximizes classification performance and pragmatic implementation. Second, we demonstrate how to identify the motion capture approach that maximizes classification performance but reduces cost. We used previously collected motion data from chronic stroke patients wearing off-the-shelf IMUs during a rehabilitation-like activity. To identify the optimal ML algorithm, we compared the classification performance, computational complexity, and tuning requirements of four off-the-shelf algorithms. To identify the optimal motion capture approach, we compared the classification performance of various sensor configurations (number and location on the body) and sensor type (IMUs vs. accelerometers). Of the algorithms tested, linear discriminant analysis had the highest classification performance, low computational complexity, and modest tuning requirements. Of the sensor configurations tested, seven sensors on the paretic arm and trunk led to the highest classification performance, and IMUs outperformed accelerometers. Overall, we present a refined sensor-ML approach that maximizes both classification performance and pragmatic implementation. In addition, with this primer, we showcase important considerations for appraising off-the-shelf algorithms and sensors for quantitative motion assessment.

A Taxonomy of Functional Upper Extremity Motion.

Background: Functional upper extremity (UE) motion enables humans to execute activities of daily living (ADLs). There currently exists no universal language to systematically characterize this type of motion or its fundamental building blocks, called functional primitives. Without a standardized classification approach, pooling mechanistic knowledge and unpacking rehabilitation content will remain challenging. Methods: We created a taxonomy to characterize functional UE motions occurring during ADLs, classifying them by motion presence, temporal cyclicity, upper body effector, and contact type. We identified five functional primitives by their phenotype and purpose: reach, reposition, transport, stabilize, and idle. The taxonomy was assessed for its validity and interrater reliability in right-paretic chronic stroke patients performing a selection of ADL tasks. We applied the taxonomy to identify the primitive content and motion characteristics of these tasks, and to evaluate the influence of impairment level on these outcomes. Results: The taxonomy could account for all motions in the sampled activities. Interrater reliability was high for primitive identification (Cohen's kappa = 0.95-0.99). Using the taxonomy, the ADL tasks were found to be composed primarily of transport and stabilize primitives mainly executed with discrete, proximal motions. Compared to mildly impaired patients, moderately impaired patients used more repeated reaches and axial-proximal UE motion to execute the tasks. Conclusions: The proposed taxonomy yields objective, quantitative data on human functional UE motion. This new method could facilitate the decomposition and quantification of UE rehabilitation, the characterization of functional abnormality after stroke, and the mechanistic examination of shared behavior in motor studies.

BioPad: Leveraging off-the-Shelf Video Games for Stress Self-Regulation.

This paper presents an approach to use commercial videogames for biofeedback training. It consists of intercepting signals from the game controller and adapting them in real-time based on physiological measurements from the player. We present three sample implementations and a case study for teaching stress self-regulation via an immersive car racing game. We use a crossover gaming device to manipulate controller signals, and a respiratory sensor to monitor the players' breathing rate. We then alter the speed of the car to encourage slow deep breathing, in this way, allowing players to reduce their arousal while playing the game. We evaluate the approach against an alternative form of biofeedback that uses a graphic overlay to convey physiological information, and a control condition (playing the game without biofeedback). Experimental results show that our approach can promote deep breathing during gameplay, and also during a subsequent task, once biofeedback is removed. Our results also indicate that delivering biofeedback through subtle changes in gameplay can be as effective as delivering them directly through a visual display. These results open the possibility to develop low-cost and engaging biofeedback interventions using a variety of commercial videogames to promote adherence.

Physiological Modalities for Relaxation Skill Transfer in Biofeedback Games.

We present an adaptive biofeedback game for teaching self-regulation of stress. Our approach consists of monitoring the user's physiology during gameplay and adapting the game using a positive feedback loop that rewards relaxing behaviors and penalizes states of high arousal. We evaluate the approach using a casual game under three biofeedback modalities: electrodermal activity, heart rate variability, and breathing rate. The three biosignals can be measured noninvasively with wearable sensors, and represent different degrees of voluntary control and selectivity toward arousal. We conducted an experiment trial with 25 participants to compare the three modalities against a standard treatment (deep breathing) and a control condition (the game without biofeedback). Our results indicate that breathing-based game biofeedback is more effective in inducing relaxation during treatment than the other four groups. Participants in this group also showed greater retention of the relaxation skills (without biofeedback) during a subsequent stressor.

Motor function assessment using wearable inertial sensors.

We present an approach to wearable sensor-based assessment of motor function in individuals post stroke. We make use of one on-body inertial measurement unit (IMU) to automate the functional ability (FA) scoring of the Wolf Motor Function Test (WMFT). WMFT is an assessment instrument used to determine the functional motor capabilities of individuals post stroke. It is comprised of 17 tasks, 15 of which are rated according to performance time and quality of motion. We present signal processing and machine learning tools to estimate the WMFT FA scores of the 15 tasks using IMU data. We treat this as a classification problem in multidimensional feature space and use a supervised learning approach.