Toward Real-Time Muscle Force Inference and Device Control via Optical-Flow-Tracked Muscle Deformation.

Despite the utility of musculoskeletal dynamics modeling, there exists no safe, noninvasive method of measuring in vivo muscle output force in real time - limiting both biomechanical insight into dexterous motion and intuitive control of assistive devices. In this paper, we demonstrate that muscle deformation constitutes a promising, yet unexplored signal from which to 1) infer such forces and 2) build novel device control schemes. Through a case study of the elbow joint on a preliminary cohort of 10 subjects, we show that muscle deformation (specifically, thickness change of the brachioradialis, as measured via ultrasound and tracked via optical flow) correlates well with elbow output force to an extent comparable with standard surface electromyography (sEMG) activation during varied isometric elbow contraction. We then show that, given real-time visual feedback, subjects can readily perform a trajectory tracking task using this deformation signal, and that they largely prefer this method to a comparable sEMG-based control scheme and perform the tracking task with similar accuracy. Together, these contributions illustrate muscle deformation's potential utility for both biomechanical study of individual muscle dynamics and device control, in a manner that - thanks to, unlike sEMG, the localized nature of the signal and its tight mechanistic coupling to output force - is readily extensible to multiple muscles and device degrees of freedom. To enable such future extensions, all modeling, tracking, and visualization software described in this paper, as well as all raw and processed data, have been made available on SimTK as part of the Open-Arm project (https://simtk.org/projects/openarm) for general research use.

Design of a Passive, Variable Stiffness Exoskeleton for Triceps Deficiency Mitigation.

Individuals with neurological impairment, particularly those with cervical level spinal cord injuries (SCI), often have difficulty with daily tasks due to triceps weakness or total loss of function. More demanding tasks, such as sit-skiing, may be rendered impossible due to their extreme strength demands. Design of exoskeletons that address this issue by providing supplemental strength in arm extension is an active field of research but commercial devices are not yet available for use. Most current designs employ electric motors that necessitate the addition of bulky power sources and extraneous wiring, rendering the devices impractical in daily life. The possibility of powering an upper extremity exoskeleton passively has been explored, but to date, none have delivered sufficient function or strength to provide useful assistance for sit-skiing. We seek to rectify this with the design of a passively actuated exoskeletal arm brace capable of operating in two, adjustable-strength modes: one for low level gravity compensation to aid in active range of motion, and the other for more stringent weight bearing activities. The mechanism developed through this paper allows for an affordable, lightweight, modular device that can be adjusted and customized for the needs of each individual patient.

Reachable Workspace and Proximal Function Measures for Quantifying Upper Limb Motion.

There are a lack of quantitative measures for clinically assessing upper limb function. Conventional biomechanical performance measures are restricted to specialist labs due to hardware cost and complexity, while the resulting measurements require specialists for analysis. Depth cameras are low cost and portable systems that can track surrogate joint positions. However, these motions may not be biologically consistent, which can result in noisy, inaccurate movements. This paper introduces a rigid body modelling method to enforce biological feasibility of the recovered motions. This method is evaluated on an existing depth camera assessment: the reachable workspace (RW) measure for assessing gross shoulder function. As a rigid body model is used, position estimates of new proximal targets can be added, resulting in a proximal function (PF) measure for assessing a subject's ability to touch specific body landmarks. The accuracy, and repeatability of these measures is assessed on ten asymptomatic subjects, with and without rigid body constraints. This analysis is performed both on a low-cost depth camera system and a gold-standard active motion capture system. The addition of rigid body constraints was found to improve accuracy and concordance of the depth camera system, particularly in lateral reaching movements. Both RW and PF measures were found to be feasible candidates for clinical assessment, with future analysis needed to determine their ability to detect changes within specific patient populations.

Estimating Sit-to-Stand Dynamics Using a Single Depth Camera.

Kinetic and dynamic motion analysis provides quantitative, functional assessments of human ability that are unobtainable through static imaging methods or subjective surveys. While biomechanics facilities are equipped to perform this measurement and analysis, the clinical translation of these methods is limited by the specialized skills and equipment needed. This paper presents and validates a method for estimating dynamic effects such as joint torques and body momenta using a single depth camera. An allometrically scaled, sagittal plane dynamic model is used to estimate the joint torques at the ankles, knees, hips, and low back, as well as the torso momenta, and shear and normal loads at the L5-S1 disk. These dynamic metrics are applied to the sit-to-stand motion and validated against a gold-standard biomechanical system consisting of full-body active motion-capture and force-sensing systems. The metrics obtained from the proposed method were found to have excellent concordance with peak metrics that are consistent with prior biomechanical studies. This suggests the feasibility of using this system for rapid clinical assessment, with applications in diagnostics, longitudinal tracking, and quantifying patient recovery.

OpenArm 2.0: Automated Segmentation of 3D Tissue Structures for Multi-Subject Study of Muscle Deformation Dynamics.

We present a novel neural-network-based pipeline for segmentation of 3D muscle and bone structures from localized 2D ultrasound data of the human arm. Building from the U-Net [1] neural network framework, we examine various data augmentation techniques and training data sets to both optimize the network's performance on our data set and hypothesize strategies to better select training data, minimizing manual annotation time while maximizing performance. We then employ this pipeline to generate the OpenArm 2.0 data set, the first factorial set of multi-subject, multi-angle, multi-force scans of the arm with full volumetric annotation of the biceps and humerus. This data set has been made available on SimTK (https://simtk.org/projects/openarm) to enable future exploration of muscle force modeling, improved musculoskeletal graphics, and assistive device control.

Simple Spline Representation for Identifying Sit-to-Stand Strategies.

Standing from a seated position is an activity of daily living and a common clinical test of strength and balance. While this action is well-studied biomechanically, there remains a need for a clear modelling method for appropriately capturing performance and discriminating between standing strategies. This paper presents a simple framework for representing the rise from a chair as a set of splines. This formulation is inherently differentiable, defines a clear start and end point of the motion, and allows for secondary analysis of dynamic and energetic effects. This method is tested on two healthy subjects performing four different standing strategies. The spline method was found to accurately capture the standing action, with mean absolute errors of 1-2 cm for joint position, and 2-3 degrees angular error across the different standing strategies. Analysis of the spline trajectories revealed strategy-specific differences in kinematic, kinetic, and dynamic bio-markers. This suggests that low order splines can be used to accurately capture variations in sit-to-stand actions.

Kinematic and Kinetic Validation of an Improved Depth Camera Motion Assessment System Using Rigid Bodies.

The study of joint kinematics and dynamics has broad clinical applications, including the identification of pathological motions or compensation strategies and the analysis of dynamic stability. High-end motion capture systems, however, are expensive and require dedicated camera spaces with lengthy setup and data processing commitments. Depth cameras, such as the Microsoft Kinect, provide an inexpensive, marker-free alternative at the sacrifice of joint-position accuracy. In this work, we present a fast framework for adding biomechanical constraints to the joint estimates provided by a depth camera system. We also present a new model for the lower lumbar joint angle. We validate key joint position, angle, and velocity measurements against a gold standard active motion-capture system on ten healthy subjects performing sit to stand (STS). Our method showed significant improvement in mean absolute error and intraclass correlation coefficients for the recovered joint angles and position-based metrics. These improvements suggest that depth cameras can provide an accurate and clinically viable method of rapidly assessing the kinematics and kinetics of the STS action, providing data for further analysis using biomechanical or machine learning methods.

Tracking Kinematic and Kinetic Measures of Sit to Stand using an Instrumented Spine Orthosis.

Age related spinal deformity is becoming an increasingly prevalent problem, resulting in decreased quality of life. While spinal deformity can be corrected via surgical intervention, a large number of people with spinal fusions require follow-up surgery due to further degeneration. The identification of changes to a subjects kinematics and kinetics post-surgery are limited by a lack of methods to collect patient specific motion data over the course of surgical recovery. This paper introduces an Instrumented Spine Orthosis (ISO) that can capture the motions of the subjects torso without requiring the use of a control computer or other dedicated motion capture equipment. This system is used to collect the peak torso angles and velocities for a single subject performing sit-to-stand actions. The accuracy of the ISO is evaluated using motion capture, during different sit-to-stand protocols designed to highlight motion changes that have been seen in subjects with reduced mobility. This system was found to provide reliable measurements of these kinematic and kinetic torso measures across all tested motions, demonstrating the potential for the use of Instrumented Spine Orthotics to provide quantitative measures during the surgical recovery process.

A functional method for generating individualized spine models from motion-capture data.

A representative model is necessary for the analysis of spine kinematics and dynamics during motion. Existing models, based on stationary imaging or cadaveric data, may not be accurate through the full range of spinal motion or for clinical populations. In this paper, we propose a functional method for estimating subject-specific spinal joint centers, generating a one-joint or two-joint kinematic model of the spine. These models are driven by the motion of the thorax and pelvis as observed by eight surface landmarks. We apply this method to experimental data from ten subjects performing flexion/extension and sit-to-stand motions. The recovered functional models are assessed against an allometric model though the analysis of marker residuals. We found that the functional models provide lower residuals than the allometric methods. Between the functional models, the two-joint model provided lower residuals with less sensitivity to the training action, while the one-joint model should be trained on the motion of interest.

Fully Automated Echocardiogram Interpretation in Clinical Practice.

BACKGROUND: Automated cardiac image interpretation has the potential to transform clinical practice in multiple ways, including enabling serial assessment of cardiac function by nonexperts in primary care and rural settings. We hypothesized that advances in computer vision could enable building a fully automated, scalable analysis pipeline for echocardiogram interpretation, including (1) view identification, (2) image segmentation, (3) quantification of structure and function, and (4) disease detection. METHODS: Using 14 035 echocardiograms spanning a 10-year period, we trained and evaluated convolutional neural network models for multiple tasks, including automated identification of 23 viewpoints and segmentation of cardiac chambers across 5 common views. The segmentation output was used to quantify chamber volumes and left ventricular mass, determine ejection fraction, and facilitate automated determination of longitudinal strain through speckle tracking. Results were evaluated through comparison to manual segmentation and measurements from 8666 echocardiograms obtained during the routine clinical workflow. Finally, we developed models to detect 3 diseases: hypertrophic cardiomyopathy, cardiac amyloid, and pulmonary arterial hypertension. RESULTS: Convolutional neural networks accurately identified views (eg, 96% for parasternal long axis), including flagging partially obscured cardiac chambers, and enabled the segmentation of individual cardiac chambers. The resulting cardiac structure measurements agreed with study report values (eg, median absolute deviations of 15% to 17% of observed values for left ventricular mass, left ventricular diastolic volume, and left atrial volume). In terms of function, we computed automated ejection fraction and longitudinal strain measurements (within 2 cohorts), which agreed with commercial software-derived values (for ejection fraction, median absolute deviation=9.7% of observed, N=6407 studies; for strain, median absolute deviation=7.5%, n=419, and 9.0%, n=110) and demonstrated applicability to serial monitoring of patients with breast cancer for trastuzumab cardiotoxicity. Overall, we found automated measurements to be comparable or superior to manual measurements across 11 internal consistency metrics (eg, the correlation of left atrial and ventricular volumes). Finally, we trained convolutional neural networks to detect hypertrophic cardiomyopathy, cardiac amyloidosis, and pulmonary arterial hypertension with C statistics of 0.93, 0.87, and 0.85, respectively. CONCLUSIONS: Our pipeline lays the groundwork for using automated interpretation to support serial patient tracking and scalable analysis of millions of echocardiograms archived within healthcare systems.

Stability basin estimates fall risk from observed kinematics, demonstrated on the Sit-to-Stand task.

The ability to quantitatively measure stability is essential to ensuring the safety of locomoting systems. While the response to perturbation directly reflects the stability of a motion, this experimental method puts human subjects at risk. Unfortunately, existing indirect methods for estimating stability from unperturbed motion have been shown to have limited predictive power. This paper leverages recent advances in dynamical systems theory to accurately estimate the stability of human motion without requiring perturbation. This approach relies on kinematic observations of a nominal Sit-to-Stand motion to construct an individual-specific dynamic model, input bounds, and feedback control that are then used to compute the set of perturbations from which the model can recover. This set, referred to as the stability basin, was computed for 14 individuals, and was able to successfully differentiate between less and more stable Sit-to-Stand strategies for each individual with greater accuracy than existing methods.

The Feasibility and Usability of RunningCoach: A Remote Coaching System for Long-Distance Runners.

Studies have shown that about half of the injuries sustained during long-distance running involve the knee. Cadence (steps per minute) has been identified as a factor that is strongly associated with these running-related injuries, making it a worthwhile candidate for further study. As such, it is critical for long-distance runners to minimize their risk of injury by running at an appropriate running cadence. In this paper, we present the results of a study on the feasibility and usability of RunningCoach, a mobile health (mHealth) system that remotely monitors running cadence levels of runners in a continuous fashion, among other variables, and provides immediate feedback to runners in an effort to help them optimize their running cadence.

Revisiting active perception.

Despite the recent successes in robotics, artificial intelligence and computer vision, a complete artificial agent necessarily must include active perception. A multitude of ideas and methods for how to accomplish this have already appeared in the past, their broader utility perhaps impeded by insufficient computational power or costly hardware. The history of these ideas, perhaps selective due to our perspectives, is presented with the goal of organizing the past literature and highlighting the seminal contributions. We argue that those contributions are as relevant today as they were decades ago and, with the state of modern computational tools, are poised to find new life in the robotic perception systems of the next decade.

Fusing motion-capture and inertial measurements for improved joint state recovery: An application for sit-to-stand actions.

The estimate of joint angles, velocities, and accelerations is a key component of biomechanical modelling. The literature presents a variety of sensing modalities and algorithms to recover the full joint state, with tuning parameters varying between different applications, actions, and limbs. Comparisons between these methods are frequently limited to angles only, without comparison between the joint velocities and accelerations. This paper introduces an algorithm to fuse motion-capture and inertial measurements to recover the full state during a sit-to-stand task. This algorithm is then compared to three other methods: Kalman filtering on motion-capture or inertial measurements alone and the standard angular recovery/differentiation method. It is shown that the fusion of both optical and inertial measurements reduce the ripple and offset artefacts which become pronounced in high acceleration human motions.

Design and Evaluation of an Interactive Exercise Coaching System for Older Adults: Lessons Learned.

Although the positive effects of exercise on the well-being and quality of independent living for older adults are well accepted, many elderly individuals lack access to exercise facilities, or the skills and motivation to perform exercise at home. To provide a more engaging environment that promotes physical activity, various fitness applications have been proposed. Many of the available products, however, are geared toward a younger population and are not appropriate or engaging for an older population. To address these issues, we developed an automated interactive exercise coaching system using the Microsoft Kinect. The coaching system guides users through a series of video exercises, tracks and measures their movements, provides real-time feedback, and records their performance over time. Our system consists of exercises to improve balance, flexibility, strength, and endurance, with the aim of reducing fall risk and improving performance of daily activities. In this paper, we report on the development of the exercise system, discuss the results of our recent field pilot study with six independently living elderly individuals, and highlight the lessons learned relating to the in-home system setup, user tracking, feedback, and exercise performance evaluation.

Reachable workspace and performance of upper limb (PUL) in duchenne muscular dystrophy.

INTRODUCTION: The Kinect-based reachable workspace relative surface area (RSA) is compared with the performance of upper limb (PUL) assessment in Duchenne muscular dystrophy (DMD). METHODS: 29 individuals with DMD (ages: 7-23; Brooke: 1-5) underwent both Kinect-based reachable workspace RSA and PUL assessments. RSAs were also collected from 24 age-matched controls. Total and quadrant RSAs were compared with the PUL total, shoulder-, middle-, and distal-dimension scores. RESULTS: The total reachable workspace RSA correlated well with the total PUL score (Spearman ρ = -0.602; P < 0.001), and with each of the PUL dimensional scores: shoulder (ρ = -0.624; P < 0.001), middle (ρ = -0.564; P = 0.001), and distal (ρ = -0.630; P < 0.001). With quadrant RSA, reachability in a particular quadrant was closely associated with respective PUL dimensional-level function (lateral-upper quadrant for shoulder-, lateral-upper/lower quadrants for middle-, and lateral-lower quadrant for distal-level function). CONCLUSIONS: This study demonstrates concurrent validity of the reachable workspace outcome measure (RSA) with the DMD-specific upper extremity outcome measure (PUL).

Upper extremity 3-dimensional reachable workspace assessment in amyotrophic lateral sclerosis by Kinect sensor.

INTRODUCTION: Reachable workspace is a measure that provides clinically meaningful information regarding arm function. In this study, a Kinect sensor was used to determine the spectrum of 3-dimensional reachable workspace encountered in a cross-sectional cohort of individuals with amyotrophic lateral sclerosis (ALS). METHODS: Bilateral 3D reachable workspace was recorded from 10 subjects with ALS and 17 healthy controls. The data were normalized by each individual's arm length to obtain a reachable workspace relative surface area (RSA). Concurrent validity was assessed by correlation with scoring on the ALS Functional Rating Score-revised (ALSFRSr). RESULTS: The Kinect-measured reachable workspace RSA differed significantly between the ALS and control subjects (0.579 ± 0.226 vs. 0.786 ± 0.069; P < 0.001). The RSA demonstrated correlation with ALSFRSr upper extremity items (Spearman correlation ρ = 0.569; P = 0.009). With worsening upper extremity function, as categorized by the ALSFRSr, the reachable workspace also decreased progressively. CONCLUSIONS: This study demonstrates the feasibility and potential of using a novel Kinect-based reachable workspace outcome measure in ALS.

Generating physically realistic kinematic and dynamic models from small data sets: An application for sit-to-stand actions.

Kinematic and dynamic models are used to create simplified, yet accurate representations of reality. In application to biological systems, there is often a choice on what level of complexity is appropriate for the model. This paper introduces a structured method for obtaining an accurate model that can represent the sit-to-stand motion and reproduce the associated contact forces in the standing phase. These models are generated from small datasets, just five measured sit-to-stand actions, and result in simple, physically realisable dynamic models. The assumptions made apriori on the model are minimal, with the number of segments, axes of rotation, marker allocation and location, and dynamic model all determined from this small dataset. From this initial analysis, the use of a triple pendulum with a simple point mass at the centre of the torso was found to be representative. Through the generation of these simple, repeatable models, this work aims to develop a modelling framework that is suitable for the study of biological systems and clinical use.

Recognizing the intensity of strength training exercises with wearable sensors.

In this paper we propose a system based on a network of wearable accelerometers and an off-the-shelf smartphone to recognize the intensity of stationary activities, such as strength training exercises. The system uses a hierarchical algorithm, consisting of two layers of Support Vector Machines (SVMs), to first recognize the type of exercise being performed, followed by recognition of exercise intensity. The first layer uses a single SVM to recognize the type of the performed exercise. Based on the recognized type a corresponding intensity prediction SVM is selected on the second layer, specializing in intensity prediction for the recognized type of exercise. We evaluate the system for a set of upper-body exercises using different weight loads. Additionally, we compare the most important features for exercise and intensity recognition tasks and investigate how different sliding window combinations, sensor configurations and number of training subjects impact the algorithm performance. We perform all of the experiments for two different types of features to evaluate the feasibility of implementation on resource constrained hardware. The results show the algorithm is able to recognize exercise types with approximately 85% accuracy and 6% intensity prediction error. Furthermore, due to similar performance using different types of features, the algorithm offers potential for implementation on resource constrained hardware.

Validity and reliability of smartphone magnetometer-based goniometer evaluation of shoulder abduction--A pilot study.

BACKGROUND: Goniometers are commonly used by physical therapists to measure range-of-motion (ROM) in the musculoskeletal system. These measurements are used to assist in diagnosis and to help monitor treatment efficacy. With newly emerging technologies, smartphone-based applications are being explored for measuring joint angles and movement. OBJECTIVE: This pilot study investigates the intra- and inter-rater reliability as well as concurrent validity of a newly-developed smartphone magnetometer-based goniometer (MG) application for measuring passive shoulder abduction in both sitting and supine positions, and compare against the traditional universal goniometer (UG). DESIGN: This is a comparative study with repeated measurement design. METHODS: Three physical therapists utilized both the smartphone MG and a traditional UG to measure various angles of passive shoulder abduction in a healthy subject, whose shoulder was positioned in eight different positions with pre-determined degree of abduction while seated or supine. Each therapist was blinded to the measured angles. Concordance correlation coefficients (CCCs), Bland-Altman plotting methods, and Analysis of Variance (ANOVA) were used for statistical analyses. RESULTS: Both traditional UG and smartphone MG were reliable in repeated measures of standardized joint angle positions (average CCC > 0.997) with similar variability in both measurement tools (standard deviation (SD) ± 4°). Agreement between the UG and MG measurements was greater than 0.99 in all positions. CONCLUSION: Our results show that the smartphone MG has equivalent reliability compared to the traditional UG when measuring passive shoulder abduction ROM. With concordant measures and comparable reliability to the UG, the newly developed MG application shows potential as a useful tool to assess joint angles.