System Architecture for VR Yoga Therapy Platform with 6-DoF Whole-Body Avatar Tracking.

Chronic pain is a leading cause of morbidity among children and adolescents affecting 35% of the global population. Pediatric chronic pain management requires integrative health methods spanning physical and psychological subsystems through various mind-body interventions. Yoga therapy is one such method, known for its ability to improve the quality of life both physically and psychologically in chronic pain conditions. However, maintaining the clinical outcomes of personalized yoga therapy sessions at-home is challenging due to fear of movement, lack of motivation, and boredom. Virtual Reality (VR) has the potential to bridge the gap between the clinic and home by motivating engagement and mitigating pain-related anxiety or fear of movement. We developed a multi-modal algorithmic architecture for fusing real-time 3D human body pose estimation models with custom developed inverse kinematics models of physical movement to render biomechanically informed 6-DoF whole-body avatars capable of embodying an individual's real-time yoga poses within the VR environment. Experiments conducted among control participants demonstrated superior movement tracking accuracy over existing commercial off-the-shelf avatar tracking solutions, leading to successful embodiment and engagement. These findings demonstrate the feasibility of rendering virtual avatar movements that embody complex physical poses such as those encountered in yoga therapy. The impact of this work moves the field one step closer to an interactive system to facilitate at-home individual or group yoga therapy for children with chronic pain conditions.

A Virtual Reality Exergame: Clinician-Guided Breathing and Relaxation for Children with Muscular Dystrophy.

This study introduces a VR-based breathing and relaxation exergame tailored for individuals with Duchenne muscular dystrophy (DMD). DMD is a rare neuromuscular disease that leads to respiratory muscle dysfunction with anxiety being a common comorbidity. Clinical management requires frequent visits to rare disease specialists to manage symptom progression. Limited availability and/or proximity of rare disease experts present challenges to care and can lead to missed care opportunities and reduced quality of life. We propose a breathing and relaxation exergame with remote telehealth applicability that incorporates shared patient-clinician VR interaction, and physiological sensors that provide both real-time feedback to the patient and health analytics for the clinician. The game focuses on two key aspects of DMD clinical care that can be mediated through control of breathing: relaxation/mindfulness training and respiratory muscle exercise. The system was evaluated among 13 individuals, including 4 participants with DMD. Feedback surveys, interviews, and focus group discussions with participants, accompanying family members, and clinicians demonstrated the feasibility of this VR tool for telehealth or as part of a home exercise program.

Ability-Based Methods for Personalized Keyboard Generation.

This study introduces an ability-based method for personalized keyboard generation, wherein an individual's own movement and human-computer interaction data are used to automatically compute a personalized virtual keyboard layout. Our approach integrates a multidirectional point-select task to characterize cursor control over time, distance, and direction. The characterization is automatically employed to develop a computationally efficient keyboard layout that prioritizes each user's movement abilities through capturing directional constraints and preferences. We evaluated our approach in a study involving 16 participants using inertial sensing and facial electromyography as an access method, resulting in significantly increased communication rates using the personalized keyboard (52.0 bits/min) when compared to a generically optimized keyboard (47.9 bits/min). Our results demonstrate the ability to effectively characterize an individual's movement abilities to design a personalized keyboard for improved communication. This work underscores the importance of integrating a user's motor abilities when designing virtual interfaces.

Prediction of Voice Fundamental Frequency and Intensity from Surface Electromyographic Signals of the Face and Neck.

Silent speech interfaces (SSIs) enable speech recognition and synthesis in the absence of an acoustic signal. Yet, the archetypal SSI fails to convey the expressive attributes of prosody such as pitch and loudness, leading to lexical ambiguities. The aim of this study was to determine the efficacy of using surface electromyography (sEMG) as an approach for predicting continuous acoustic estimates of prosody. Ten participants performed a series of vocal tasks including sustained vowels, phrases, and monologues while acoustic data was recorded simultaneously with sEMG activity from muscles of the face and neck. A battery of time-, frequency-, and cepstral-domain features extracted from the sEMG signals were used to train deep regression neural networks to predict fundamental frequency and intensity contours from the acoustic signals. We achieved an average accuracy of 0.01 ST and precision of 0.56 ST for the estimation of fundamental frequency, and an average accuracy of 0.21 dB SPL and precision of 3.25 dB SPL for the estimation of intensity. This work highlights the importance of using sEMG as an alternative means of detecting prosody and shows promise for improving SSIs in future development.

Ability-based Keyboards for Augmentative and Alternative Communication: Understanding How Individuals' Movement Patterns Translate to More Efficient Keyboards: Methods to Generate Keyboards Tailored to User-specific Motor Abilities.

This study presents the evaluation of ability-based methods extended to keyboard generation for alternative communication in people with dexterity impairments due to motor disabilities. Our approach characterizes user-specific cursor control abilities from a multidirectional point-select task to configure letters on a virtual keyboard based on estimated time, distance, and direction of movement. These methods were evaluated in three individuals with motor disabilities against a generically optimized keyboard and the ubiquitous QWERTY keyboard. We highlight key observations relating to the heterogeneity of the manifestation of motor disabilities, perceived importance of communication technology, and quantitative improvements in communication performance when characterizing an individual's movement abilities to design personalized AAC interfaces.

Surface Electromyography-Based Recognition, Synthesis, and Perception of Prosodic Subvocal Speech.

Purpose This study aimed to evaluate a novel communication system designed to translate surface electromyographic (sEMG) signals from articulatory muscles into speech using a personalized, digital voice. The system was evaluated for word recognition, prosodic classification, and listener perception of synthesized speech. Method sEMG signals were recorded from the face and neck as speakers with (n = 4) and without (n = 4) laryngectomy subvocally recited (silently mouthed) a speech corpus comprising 750 phrases (150 phrases with variable phrase-level stress). Corpus tokens were then translated into speech via personalized voice synthesis (n = 8 synthetic voices) and compared against phrases produced by each speaker when using their typical mode of communication (n = 4 natural voices, n = 4 electrolaryngeal [EL] voices). Naïve listeners (n = 12) evaluated synthetic, natural, and EL speech for acceptability and intelligibility in a visual sort-and-rate task, as well as phrasal stress discriminability via a classification mechanism. Results Recorded sEMG signals were processed to translate sEMG muscle activity into lexical content and categorize variations in phrase-level stress, achieving a mean accuracy of 96.3% (SD = 3.10%) and 91.2% (SD = 4.46%), respectively. Synthetic speech was significantly higher in acceptability and intelligibility than EL speech, also leading to greater phrasal stress classification accuracy, whereas natural speech was rated as the most acceptable and intelligible, with the greatest phrasal stress classification accuracy. Conclusion This proof-of-concept study establishes the feasibility of using subvocal sEMG-based alternative communication not only for lexical recognition but also for prosodic communication in healthy individuals, as well as those living with vocal impairments and residual articulatory function. Supplemental Material https://doi.org/10.23641/asha.14558481.

Motor unit drive: a neural interface for real-time upper limb prosthetic control.

OBJECTIVE: Modern prosthetic limbs have made strident gains in recent years, incorporating terminal electromechanical devices that are capable of mimicking the human hand. However, access to these advanced control capabilities has been prevented by fundamental limitations of amplitude-based myoelectric neural interfaces, which have remained virtually unchanged for over four decades. Consequently, nearly 23% of adults and 32% of children with major traumatic or congenital upper-limb loss abandon regular use of their myoelectric prosthesis. To address this healthcare need, we have developed a noninvasive neural interface technology that maps natural motor unit increments of neural control and force into biomechanically informed signals for improved prosthetic control. APPROACH: Our technology, referred to as motor unit drive (MU Drive), utilizes real-time machine learning algorithms for directly measuring motor unit firings from surface electromyographic signals recorded from residual muscles of an amputated or congenitally missing limb. The extracted firings are transformed into biomechanically informed signals based on the force generating properties of individual motor units to provide a control source that represents the intended movement. MAIN RESULTS: We evaluated the characteristics of the MU Drive control signals and compared them to conventional amplitude-based myoelectric signals in healthy subjects as well as subjects with congenital or traumatic trans-radial limb-loss. Our analysis established a vital proof-of-concept: MU Drive provides a more responsive real-time signal with improved smoothness and more faithful replication of intended limb movement that overcomes the trade-off between performance and latency inherent to amplitude-based myoelectric methods. SIGNIFICANCE: MU Drive is the first neural interface for prosthetic control that provides noninvasive real-time access to the natural motor control mechanisms of the human nervous system. This new neural interface holds promise for improving prosthetic function by achieving advanced control that better reflects the user intent. Beyond the immediate advantages in the field of prosthetics, MU Drive provides an innovative alternative for advancing the control of exoskeletons, assistive devices, and other robotic rehabilitation applications.

Development of sEMG sensors and algorithms for silent speech recognition.

OBJECTIVE: Speech is among the most natural forms of human communication, thereby offering an attractive modality for human-machine interaction through automatic speech recognition (ASR). However, the limitations of ASR-including degradation in the presence of ambient noise, limited privacy and poor accessibility for those with significant speech disorders-have motivated the need for alternative non-acoustic modalities of subvocal or silent speech recognition (SSR). APPROACH: We have developed a new system of face- and neck-worn sensors and signal processing algorithms that are capable of recognizing silently mouthed words and phrases entirely from the surface electromyographic (sEMG) signals recorded from muscles of the face and neck that are involved in the production of speech. The algorithms were strategically developed by evolving speech recognition models: first for recognizing isolated words by extracting speech-related features from sEMG signals, then for recognizing sequences of words from patterns of sEMG signals using grammar models, and finally for recognizing a vocabulary of previously untrained words using phoneme-based models. The final recognition algorithms were integrated with specially designed multi-point, miniaturized sensors that can be arranged in flexible geometries to record high-fidelity sEMG signal measurements from small articulator muscles of the face and neck. MAIN RESULTS: We tested the system of sensors and algorithms during a series of subvocal speech experiments involving more than 1200 phrases generated from a 2200-word vocabulary and achieved an 8.9%-word error rate (91.1% recognition rate), far surpassing previous attempts in the field. SIGNIFICANCE: These results demonstrate the viability of our system as an alternative modality of communication for a multitude of applications including: persons with speech impairments following a laryngectomy; military personnel requiring hands-free covert communication; or the consumer in need of privacy while speaking on a mobile phone in public.

Contribution from motor unit firing adaptations and muscle coactivation during fatigue.

The control of motor unit firing behavior during fatigue is still debated in the literature. Most studies agree that the central nervous system increases the excitation to the motoneuron pool to compensate for decreased force contributions of individual motor units and sustain muscle force output during fatigue. However, some studies claim that motor units may decrease their firing rates despite increased excitation, contradicting the direct relationship between firing rates and excitation that governs the voluntary control of motor units. To investigate whether the control of motor units in fact changes with fatigue, we measured motor unit firing behavior during repeated contractions of the first dorsal interosseous (FDI) muscle while concurrently monitoring the activation of surrounding muscles, including the flexor carpi radialis, extensor carpi radialis, and pronator teres. Across all subjects, we observed an overall increase in FDI activation and motor unit firing rates by the end of the fatigue task. However, in some subjects we observed increases in FDI activation and motor unit firing rates only during the initial phase of the fatigue task, followed by subsequent decreases during the late phase of the fatigue task while the coactivation of surrounding muscles increased. These findings indicate that the strategy for sustaining force output may occasionally change, leading to increases in the relative activation of surrounding muscles while the excitation to the fatiguing muscle decreases. Importantly, irrespective of changes in the strategy for sustaining force output, the control properties regulating motor unit firing behavior remain unchanged during fatigue. NEW & NOTEWORTHY This work addresses sources of debate surrounding the manner in which motor unit firing behavior is controlled during fatigue. We found that decreases in the motor unit firing rates of the fatiguing muscle may occasionally be observed when the contribution of coactive muscles increases. Despite changes in the strategy employed to sustain the force output, the underlying control properties regulating motor unit firing behavior remain unchanged during muscle fatigue.

Silent Speech Recognition as an Alternative Communication Device for Persons with Laryngectomy.

Each year thousands of individuals require surgical removal of their larynx (voice box) due to trauma or disease, and thereby require an alternative voice source or assistive device to verbally communicate. Although natural voice is lost after laryngectomy, most muscles controlling speech articulation remain intact. Surface electromyographic (sEMG) activity of speech musculature can be recorded from the neck and face, and used for automatic speech recognition to provide speech-to-text or synthesized speech as an alternative means of communication. This is true even when speech is mouthed or spoken in a silent (subvocal) manner, making it an appropriate communication platform after laryngectomy. In this study, 8 individuals at least 6 months after total laryngectomy were recorded using 8 sEMG sensors on their face (4) and neck (4) while reading phrases constructed from a 2,500-word vocabulary. A unique set of phrases were used for training phoneme-based recognition models for each of the 39 commonly used phonemes in English, and the remaining phrases were used for testing word recognition of the models based on phoneme identification from running speech. Word error rates were on average 10.3% for the full 8-sensor set (averaging 9.5% for the top 4 participants), and 13.6% when reducing the sensor set to 4 locations per individual (n=7). This study provides a compelling proof-of-concept for sEMG-based alaryngeal speech recognition, with the strong potential to further improve recognition performance.

The compensatory interaction between motor unit firing behavior and muscle force during fatigue.

Throughout the literature, different observations of motor unit firing behavior during muscle fatigue have been reported and explained with varieties of conjectures. The disagreement amongst previous studies has resulted, in part, from the limited number of available motor units and from the misleading practice of grouping motor unit data across different subjects, contractions, and force levels. To establish a more clear understanding of motor unit control during fatigue, we investigated the firing behavior of motor units from the vastus lateralis muscle of individual subjects during a fatigue protocol of repeated voluntary constant force isometric contractions. Surface electromyographic decomposition technology provided the firings of 1,890 motor unit firing trains. These data revealed that to sustain the contraction force as the muscle fatigued, the following occurred: 1) motor unit firing rates increased; 2) new motor units were recruited; and 3) motor unit recruitment thresholds decreased. Although the degree of these adaptations was subject specific, the behavior was consistent in all subjects. When we compared our empirical observations with those obtained from simulation, we found that the fatigue-induced changes in motor unit firing behavior can be explained by increasing excitation to the motoneuron pool that compensates for the fatigue-induced decrease in muscle force twitch reported in empirical studies. Yet, the fundamental motor unit control scheme remains invariant throughout the development of fatigue. These findings indicate that the central nervous system regulates motor unit firing behavior by adjusting the operating point of the excitation to the motoneuron pool to sustain the contraction force as the muscle fatigues.

Synchronization of motor unit firings: an epiphenomenon of firing rate characteristics not common inputs.

Synchronous motor unit firing instances have been attributed to anatomical inputs shared by motoneurons. Yet, there is a lack of empirical evidence confirming the notion that common inputs elicit synchronization under voluntary conditions. We tested this notion by measuring synchronization between motor unit action potential trains (MUAPTs) as their firing rates progressed within a contraction from a relatively low force level to a higher one. On average, the degree of synchronization decreased as the force increased. The common input notion provides no empirically supported explanation for the observed synchronization behavior. Therefore, we investigated a more probable explanation for synchronization. Our data set of 17,546 paired MUAPTs revealed that the degree of synchronization varies as a function of two characteristics of the motor unit firing rate: the similarity and the slope as a function of force. Both are measures of the excitation of the motoneurons. As the force generated by the muscle increases, the firing rate slope decreases, and the synchronization correspondingly decreases. Different muscles have motor units with different firing rate characteristics and display different amounts of synchronization. Although this association is not proof of causality, it consistently explains our observations and strongly suggests further investigation. So viewed, synchronization is likely an epiphenomenon, subject to countless unknown neural interactions. As such, synchronous firing instances may not be the product of a specific design and may not serve a specific physiological purpose. Our explanation for synchronization has the advantage of being supported by empirical evidence, whereas the common input does not.

Decomposition of surface EMG signals from cyclic dynamic contractions.

Over the past 3 decades, various algorithms used to decompose the electromyographic (EMG) signal into its constituent motor unit action potentials (MUAPs) have been reported. All are limited to decomposing EMG signals from isometric contraction. In this report, we describe a successful approach to decomposing the surface EMG (sEMG) signal collected from cyclic (repeated concentric and eccentric) dynamic contractions during flexion/extension of the elbow and during gait. The increased signal complexity introduced by the changing shapes of the MUAPs due to relative movement of the electrodes and the lengthening/shortening of muscle fibers was managed by an incremental approach to enhancing our established algorithm for decomposing sEMG signals obtained from isometric contractions. We used machine-learning algorithms and time-varying MUAP shape discrimination to decompose the sEMG signal from an increasingly challenging sequence of pseudostatic and dynamic contractions. The accuracy of the decomposition results was assessed by two verification methods that have been independently evaluated. The firing instances of the motor units had an accuracy of ∼90% with a MUAP train yield as high as 25. Preliminary observations from the performance of motor units during cyclic contractions indicate that during repetitive dynamic contractions, the control of motor units is governed by the same rules as those evidenced during isometric contractions. Modifications in the control properties of motoneuron firings reported by previous studies were not confirmed. Instead, our data demonstrate that the common drive and hierarchical recruitment of motor units are preserved during concentric and eccentric contractions.

Statistically rigorous calculations do not support common input and long-term synchronization of motor-unit firings.

Over the past four decades, various methods have been implemented to measure synchronization of motor-unit firings. In this work, we provide evidence that prior reports of the existence of universal common inputs to all motoneurons and the presence of long-term synchronization are misleading, because they did not use sufficiently rigorous statistical tests to detect synchronization. We developed a statistically based method (SigMax) for computing synchronization and tested it with data from 17,736 motor-unit pairs containing 1,035,225 firing instances from the first dorsal interosseous and vastus lateralis muscles--a data set one order of magnitude greater than that reported in previous studies. Only firing data, obtained from surface electromyographic signal decomposition with >95% accuracy, were used in the study. The data were not subjectively selected in any manner. Because of the size of our data set and the statistical rigor inherent to SigMax, we have confidence that the synchronization values that we calculated provide an improved estimate of physiologically driven synchronization. Compared with three other commonly used techniques, ours revealed three types of discrepancies that result from failing to use sufficient statistical tests necessary to detect synchronization. 1) On average, the z-score method falsely detected synchronization at 16 separate latencies in each motor-unit pair. 2) The cumulative sum method missed one out of every four synchronization identifications found by SigMax. 3) The common input assumption method identified synchronization from 100% of motor-unit pairs studied. SigMax revealed that only 50% of motor-unit pairs actually manifested synchronization.

Error reduction in EMG signal decomposition.

Decomposition of the electromyographic (EMG) signal into constituent action potentials and the identification of individual firing instances of each motor unit in the presence of ambient noise are inherently probabilistic processes, whether performed manually or with automated algorithms. Consequently, they are subject to errors. We set out to classify and reduce these errors by analyzing 1,061 motor-unit action-potential trains (MUAPTs), obtained by decomposing surface EMG (sEMG) signals recorded during human voluntary contractions. Decomposition errors were classified into two general categories: location errors representing variability in the temporal localization of each motor-unit firing instance and identification errors consisting of falsely detected or missed firing instances. To mitigate these errors, we developed an error-reduction algorithm that combines multiple decomposition estimates to determine a more probable estimate of motor-unit firing instances with fewer errors. The performance of the algorithm is governed by a trade-off between the yield of MUAPTs obtained above a given accuracy level and the time required to perform the decomposition. When applied to a set of sEMG signals synthesized from real MUAPTs, the identification error was reduced by an average of 1.78%, improving the accuracy to 97.0%, and the location error was reduced by an average of 1.66 ms. The error-reduction algorithm in this study is not limited to any specific decomposition strategy. Rather, we propose it be used for other decomposition methods, especially when analyzing precise motor-unit firing instances, as occurs when measuring synchronization.

Transposed firing activation of motor units.

Muscles are composed of groups of muscle fibers, called motor units, each innervated by a single motoneuron originating in the spinal cord. During constant or linearly varying voluntary force contractions, motor units are activated in a hierarchical order, with the earlier-recruited motor units having greater firing rates than the later-recruited ones. We found that this normal pattern of firing activation can be altered during oscillatory contractions where the force oscillates at frequencies ≥2 Hz. During these high-frequency oscillations, the activation of the lower-threshold motor units effectively decreases and that of the higher-threshold motor units effectively increases. This transposition of firing activation provides means to activate higher-threshold motor units preferentially. Our results demonstrate that the hierarchical regulation of motor unit activation can be manipulated to activate specific motoneuron populations preferentially. This finding can be exploited to develop new forms of physical therapies and exercise programs that enhance muscle performance or that target the preferential atrophy of high-threshold motor units as a result of aging or motor disorders such as stroke and amyotrophic lateral sclerosis.