Computer Vision and Deep Learning for Environment-Adaptive Control of Robotic Lower-Limb Exoskeletons.

Robotic exoskeletons require human control and decision making to switch between different locomotion modes, which can be inconvenient and cognitively demanding. To support the development of automated locomotion mode recognition systems (i.e., intelligent high-level controllers), we designed an environment recognition system using computer vision and deep learning. Here we first reviewed the development of the "ExoNet" database - the largest and most diverse open-source dataset of wearable camera images of indoor and outdoor real-world walking environments, which were annotated using a hierarchical labelling architecture. We then trained and tested the EfficientNetB0 convolutional neural network, which was optimized for efficiency using neural architecture search, to forward predict the walking environments. Our environment recognition system achieved ~73% image classification accuracy. These results provide the inaugural benchmark performance on the ExoNet database. Future research should evaluate and compare different convolutional neural networks to develop an accurate and real- time environment-adaptive locomotion mode recognition system for robotic exoskeleton control.

Environment Classification for Robotic Leg Prostheses and Exoskeletons Using Deep Convolutional Neural Networks.

Robotic leg prostheses and exoskeletons can provide powered locomotor assistance to older adults and/or persons with physical disabilities. However, the current locomotion mode recognition systems being developed for automated high-level control and decision-making rely on mechanical, inertial, and/or neuromuscular sensors, which inherently have limited prediction horizons (i.e., analogous to walking blindfolded). Inspired by the human vision-locomotor control system, we developed an environment classification system powered by computer vision and deep learning to predict the oncoming walking environments prior to physical interaction, therein allowing for more accurate and robust high-level control decisions. In this study, we first reviewed the development of our "ExoNet" database-the largest and most diverse open-source dataset of wearable camera images of indoor and outdoor real-world walking environments, which were annotated using a hierarchical labeling architecture. We then trained and tested over a dozen state-of-the-art deep convolutional neural networks (CNNs) on the ExoNet database for image classification and automatic feature engineering, including: EfficientNetB0, InceptionV3, MobileNet, MobileNetV2, VGG16, VGG19, Xception, ResNet50, ResNet101, ResNet152, DenseNet121, DenseNet169, and DenseNet201. Finally, we quantitatively compared the benchmarked CNN architectures and their environment classification predictions using an operational metric called "NetScore," which balances the image classification accuracy with the computational and memory storage requirements (i.e., important for onboard real-time inference with mobile computing devices). Our comparative analyses showed that the EfficientNetB0 network achieves the highest test accuracy; VGG16 the fastest inference time; and MobileNetV2 the best NetScore, which can inform the optimal architecture design or selection depending on the desired performance. Overall, this study provides a large-scale benchmark and reference for next-generation environment classification systems for robotic leg prostheses and exoskeletons.

Preliminary Design of an Environment Recognition System for Controlling Robotic Lower-Limb Prostheses and Exoskeletons.

Drawing inspiration from autonomous vehicles, using future environment information could improve the control of wearable biomechatronic devices for assisting human locomotion. To the authors knowledge, this research represents the first documented investigation using machine vision and deep convolutional neural networks for environment recognition to support the predictive control of robotic lower-limb prostheses and exoskeletons. One participant was instrumented with a battery-powered, chest-mounted RGB camera system. Approximately 10 hours of video footage were experimentally collected while ambulating throughout unknown outdoor and indoor environments. The sampled images were preprocessed and individually labelled. A deep convolutional neural network was developed and trained to automatically recognize three walking environments: level-ground, incline staircases, and decline staircases. The environment recognition system achieved 94.85% overall image classification accuracy. Extending these preliminary findings, future research should incorporate other environment classes (e.g., incline ramps) and integrate the environment recognition system with electromechanical sensors and/or surface electromyography for automated locomotion mode recognition. The challenges associated with implementing deep learning on wearable biomechatronic devices are discussed.

Prenatal exposures and anti-Mullerian hormone in female adolescents: the Avon Longitudinal Study of Parents and Children.

Given that the primordial ovarian follicular pool is established in utero, it may be influenced by parental characteristics and the intrauterine environment. Anti-Müllerian hormone (AMH) levels are increasingly recognized as a biomarker of ovarian reserve in females in adulthood and adolescence. We examined and compared associations of maternal and paternal prenatal exposures with AMH levels in adolescent (mean age, 15.4 years) female offspring (n = 1,399) using data from the Avon Longitudinal Study of Parents and Children, a United Kingdom birth cohort study that originated in 1991 and is still ongoing (data are from 1991-2008). The median AMH level was 3.67 ng/mL (interquartile range: 2.46-5.57). Paternal but not maternal smoking prior to and during pregnancy were inversely associated with AMH levels. No or irregular maternal menstrual cycles before pregnancy were associated with higher AMH levels in daughter during adolescence. High maternal gestational weight gain (top fifth versus the rest of the distribution) was associated with lower AMH levels in daughters. Parental age, body mass index, and alcohol intake during pregnancy, child's birth weight, and maternal parity and time to conception were not associated with daughters' AMH levels. Our results suggest that some parental preconceptual characteristics and environmental exposures while the child is in utero may influence the long-term ovarian development and function in female offspring.

Anti-müllerian hormone is not associated with cardiometabolic risk factors in adolescent females.

OBJECTIVES: Epidemiological evidence for associations of Anti-Müllerian hormone (AMH) with cardiometabolic risk factors is lacking. Existing evidence comes from small studies in select adult populations, and findings are conflicting. We aimed to assess whether AMH is associated with cardiometabolic risk factors in a general population of adolescent females. METHODS: AMH, fasting insulin, glucose, HDLc, LDLc, triglycerides and C-reactive protein (CRP) were measured at a mean age 15.5 years in 1,308 female participants in the Avon Longitudinal Study of Parents and Children (ALSPAC). Multivariable linear regression was used to examine associations of AMH with these cardiometabolic outcomes. RESULTS: AMH values ranged from 0.16-35.84 ng/ml and median AMH was 3.57 ng/ml (IQR: 2.41, 5.49). For females classified as post-pubertal (n = 848) at the time of assessment median (IQR) AMH was 3.81 ng/ml (2.55, 5.82) compared with 3.25 ng/ml (2.23, 5.05) in those classed as early pubertal (n = 460, P≤0.001). After adjusting for birth weight, gestational age, pubertal stage, age, ethnicity, socioeconomic position, adiposity and use of hormonal contraceptives, there were no associations with any of the cardiometabolic outcomes. For example fasting insulin changed by 0% per doubling of AMH (95%CI: -3%,+2%) p  = 0.70, with identical results if HOMA-IR was used. Results were similar after additional adjustment for smoking, physical activity and age at menarche, after exclusion of 3% of females with the highest AMH values, after excluding those that had not started menarche and after excluding those using hormonal contraceptives. CONCLUSION: Our results suggest that in healthy adolescent females, AMH is not associated with cardiometabolic risk factors.