Toward Real-Time Muscle Force Inference and Device Control via Optical-Flow-Tracked Muscle Deformation.

Despite the utility of musculoskeletal dynamics modeling, there exists no safe, noninvasive method of measuring in vivo muscle output force in real time - limiting both biomechanical insight into dexterous motion and intuitive control of assistive devices. In this paper, we demonstrate that muscle deformation constitutes a promising, yet unexplored signal from which to 1) infer such forces and 2) build novel device control schemes. Through a case study of the elbow joint on a preliminary cohort of 10 subjects, we show that muscle deformation (specifically, thickness change of the brachioradialis, as measured via ultrasound and tracked via optical flow) correlates well with elbow output force to an extent comparable with standard surface electromyography (sEMG) activation during varied isometric elbow contraction. We then show that, given real-time visual feedback, subjects can readily perform a trajectory tracking task using this deformation signal, and that they largely prefer this method to a comparable sEMG-based control scheme and perform the tracking task with similar accuracy. Together, these contributions illustrate muscle deformation's potential utility for both biomechanical study of individual muscle dynamics and device control, in a manner that - thanks to, unlike sEMG, the localized nature of the signal and its tight mechanistic coupling to output force - is readily extensible to multiple muscles and device degrees of freedom. To enable such future extensions, all modeling, tracking, and visualization software described in this paper, as well as all raw and processed data, have been made available on SimTK as part of the Open-Arm project (https://simtk.org/projects/openarm) for general research use.

OpenArm 2.0: Automated Segmentation of 3D Tissue Structures for Multi-Subject Study of Muscle Deformation Dynamics.

We present a novel neural-network-based pipeline for segmentation of 3D muscle and bone structures from localized 2D ultrasound data of the human arm. Building from the U-Net [1] neural network framework, we examine various data augmentation techniques and training data sets to both optimize the network's performance on our data set and hypothesize strategies to better select training data, minimizing manual annotation time while maximizing performance. We then employ this pipeline to generate the OpenArm 2.0 data set, the first factorial set of multi-subject, multi-angle, multi-force scans of the arm with full volumetric annotation of the biceps and humerus. This data set has been made available on SimTK (https://simtk.org/projects/openarm) to enable future exploration of muscle force modeling, improved musculoskeletal graphics, and assistive device control.

Fully Automated Echocardiogram Interpretation in Clinical Practice.

BACKGROUND: Automated cardiac image interpretation has the potential to transform clinical practice in multiple ways, including enabling serial assessment of cardiac function by nonexperts in primary care and rural settings. We hypothesized that advances in computer vision could enable building a fully automated, scalable analysis pipeline for echocardiogram interpretation, including (1) view identification, (2) image segmentation, (3) quantification of structure and function, and (4) disease detection. METHODS: Using 14 035 echocardiograms spanning a 10-year period, we trained and evaluated convolutional neural network models for multiple tasks, including automated identification of 23 viewpoints and segmentation of cardiac chambers across 5 common views. The segmentation output was used to quantify chamber volumes and left ventricular mass, determine ejection fraction, and facilitate automated determination of longitudinal strain through speckle tracking. Results were evaluated through comparison to manual segmentation and measurements from 8666 echocardiograms obtained during the routine clinical workflow. Finally, we developed models to detect 3 diseases: hypertrophic cardiomyopathy, cardiac amyloid, and pulmonary arterial hypertension. RESULTS: Convolutional neural networks accurately identified views (eg, 96% for parasternal long axis), including flagging partially obscured cardiac chambers, and enabled the segmentation of individual cardiac chambers. The resulting cardiac structure measurements agreed with study report values (eg, median absolute deviations of 15% to 17% of observed values for left ventricular mass, left ventricular diastolic volume, and left atrial volume). In terms of function, we computed automated ejection fraction and longitudinal strain measurements (within 2 cohorts), which agreed with commercial software-derived values (for ejection fraction, median absolute deviation=9.7% of observed, N=6407 studies; for strain, median absolute deviation=7.5%, n=419, and 9.0%, n=110) and demonstrated applicability to serial monitoring of patients with breast cancer for trastuzumab cardiotoxicity. Overall, we found automated measurements to be comparable or superior to manual measurements across 11 internal consistency metrics (eg, the correlation of left atrial and ventricular volumes). Finally, we trained convolutional neural networks to detect hypertrophic cardiomyopathy, cardiac amyloidosis, and pulmonary arterial hypertension with C statistics of 0.93, 0.87, and 0.85, respectively. CONCLUSIONS: Our pipeline lays the groundwork for using automated interpretation to support serial patient tracking and scalable analysis of millions of echocardiograms archived within healthcare systems.