Foetus-specific neuromusculoskeletal modelling with MRI-driven vaginal delivery kinematics during the second stage of labor.

Childbirth simulations lack realism due to an oversimplification of the foetal model, particularly as most models do not allow joint motion. Foetus-specific neuromusculoskeletal (NMS) model with a detailed articulated skeleton is still not available in the literature. The present work aims at proposing the first-ever foetus-specific NMS model and then simulating the foetal descent during a vaginal delivery by using in vivo medical resonance imaging (MRI) childbirth data. Moreover, the developed model is provided open source for the community. Our foetus-specific NMS model was developed using the geometries reconstructed from a foetal computed tomography (CT) scan (Female, mass = 2.35 kg, length = 50 cm). The model contains 22 joints (64 degrees of freedom) and 65 muscles with a particular attention to the cervical spine level to enable the simulation of the cardinal movements. Then, the skull-to-cervical-spine (S/CP) and cervical-spine-to-torso (CP/T) deflection angles were extracted from in vivo MRI data for motion simulation. The S/CP and CP/T deflexion angles range from 12 degrees of flexion to 2 degrees of extension and from 7 degrees of flexion to 22 degrees of extension respectively. The developed model opens new avenues in more biofidelic childbirth simulations with a complete foetal NMS model. Obtained outcomes with the in vivo MRI data enabled to perform a first simulation of the foetal descent kinematics using real childbirth data. Future works will focus on developing a novel muscle formulation of the foetus and combining such a NMS model with a deformable model to simulate childbirth and associated complication scenarios.

Novel hybrid rigid-deformable fetal modeling for simulating the vaginal delivery within the second stage of labor.

BACKGROUND AND OBJECTIVE: The fetal representation as a 3D articulated body plays an essential role to describe a realistic vaginal delivery simulation. However, the current computational solutions have been oversimplified. The objective of the present work was to develop and evaluate a novel hybrid rigid-deformable modeling approach for the fetal body and then simulate its interaction with surrounding fetal soft tissues and with other maternal pelvis soft tissues during the second stage of labor. METHODS: CT scan data was used for 3D fetal skeleton reconstruction. Then, a novel hybrid rigid-deformable model of the fetal body was developed. This model was integrated into a maternal 3D pelvis model to simulate the vaginal delivery. Soft tissue deformation was simulated using our novel HyperMSM formulation. Magnetic resonance imaging during the second stage of labor was used to impose the trajectory of the fetus during the delivery. RESULTS: Our hybrid rigid-deformable fetal model showed a potential capacity for simulating the movements of the fetus along with the deformation of the fetal soft tissues during the vaginal delivery. The deformation energy density observed in the simulation for the fetal head fell within the strain range of 3 % to 5 %, which is in good agreement with the literature data. CONCLUSIONS: This study developed, for the first time, a hybrid rigid-deformation modeling of the fetal body and then performed a vaginal delivery simulation using MRI-driven kinematic data. This opens new avenues for describing more realistic behavior of the fetal body kinematics and deformation during the second stage of labor. As perspectives, the integration of the full skeleton body, especially the upper and lower limbs will be investigated. Then, the completed model will be integrated into our developed next-generation childbirth training simulator for vaginal delivery simulation and associated complication scenarios.

On the uncertainty quantification of the active uterine contraction during the second stage of labor simulation.

Uterine contractions in the myometrium occur at multiple scales, spanning both organ and cellular levels. This complex biological process plays an essential role in the fetus delivery during the second stage of labor. Several finite element models of active uterine contractions have already been developed to simulate the descent of the fetus through the birth canal. However, the developed models suffer severe reliability issues due to the uncertain parameters. In this context, the present study aimed to perform the uncertainty quantification (UQ) of the active uterine contraction simulation to advance our understanding of pregnancy mechanisms with more reliable indicators. A uterus model with and without fetus was developed integrating a transversely isotropic Mooney-Rivlin material with two distinct fiber orientation architectures. Different contraction patterns with complex boundary conditions were designed and applied. A global sensitivity study was performed to select the most valuable parameters for the uncertainty quantification (UQ) process using a copula-based Monte Carlo method. As results, four critical material parameters ( C 1 , C 2 , K , Ca 0 ) of the active uterine contraction model were identified and used for the UQ process. The stress distribution on the uterus during the fetus descent, considering first and second fiber orientation families, ranged from 0.144 to 1.234 MPa and 0.044 to 1.619 MPa, respectively. The simulation outcomes revealed also the segment-specific contraction pattern of the uterus tissue. The present study quantified, for the first time, the effect of uncertain parameters of the complex constitutive model of the active uterine contraction on the fetus descent process. As perspectives, a full maternal pelvis model will be coupled with reinforcement learning to automatically identify the delivery mechanism behind the cardinal movements of the fetus during the active expulsion process.

Multiphysics and multiscale modeling of uterine contractions: integrating electrical dynamics and soft tissue deformation with fiber orientation.

The development of a comprehensive uterine model that seamlessly integrates the intricate interactions between the electrical and mechanical aspects of uterine activity could potentially facilitate the prediction and management of labor complications. Such a model has the potential to enhance our understanding of the initiation and synchronization mechanisms involved in uterine contractions, providing a more profound comprehension of the factors associated with labor complications, including preterm labor. Consequently, it has the capacity to assist in more effective preparation and intervention strategies for managing such complications. In this study, we present a computational model that effectively integrates the electrical and mechanical components of uterine contractions. By combining a state-of-the-art electrical model with the Hyperelastic Mass-Spring Model (HyperMSM), we adopt a multiphysics and multiscale approach to capture the electrical and mechanical activities within the uterus. The electrical model incorporates the generation and propagation of action potentials, while the HyperMSM simulates the mechanical behavior and deformations of the uterine tissue. Notably, our model takes into account the orientation of muscle fibers, ensuring that the simulated contractions align with their inherent directional characteristics. One noteworthy aspect of our contraction model is its novel approach to scaling the rest state of the mesh elements, as opposed to the conventional method of applying mechanical loads. By doing so, we eliminate artificial strain energy resulting from the resistance of soft tissues' elastic properties during contractions. We validated our proposed model through test simulations, demonstrating its feasibility and its ability to reproduce expected contraction patterns across different mesh resolutions and configurations. Moving forward, future research efforts should prioritize the validation of our model using robust clinical data. Additionally, it is crucial to refine the model by incorporating a more realistic uterus model derived from medical imaging. Furthermore, applying the model to simulate the entire childbirth process holds immense potential for gaining deeper insights into the intricate dynamics of labor.

On the estimation of hip joint centre location with incomplete bone ossification for foetus-specific neuromusculoskeletal modeling.

Childbirth is a complex physiological process in which a foetal neuromusculoskeletal model is of great importance to develop realistic delivery simulations and associated complication analyses. However, the estimation of hip joint centre (HJC) in foetuses remains a challenging issue. Thus, this paper aims to propose and evaluate a new approach to locate the HJC in foetuses. Hip CT-scans from 25 children (F = 11, age = 5.5 ± 2.6 years, height = 117 ± 21 cm, mass = 26 kg ± 9.5 kg) were used to propose and evaluate the novel acetabulum sphere fitting process to locate the HJC. This new approach using the acetabulum surface was applied to a population of 57 post-mortem foetal CT scans to locate the HJC as well as to determine associated regression equations using multiple linear regression. As results, the average distance between the HJC located using acetabulum sphere fitting and femoral head sphere fitting in children was 1.5 ± 0.7 mm. The average prediction error using our developed foetal HJC regression equations was 3.0 ± 1.5 mm, even though the equation for the x coordinate had a poor value of R2 (R2 for the x coordinate = 0.488). The present study suggests that the use of the acetabulum sphere fitting approach is a valid and accurate method to locate the HJC in children, and then can be extrapolated to get an estimation of the HJC in foetuses with incomplete bone ossification. Therefore, the present paper can be used as a guideline for foetus specific neuromusculoskeletal modelling.

Fast soft-tissue deformations coupled with mixed reality toward the next-generation childbirth training simulator.

High-quality gynecologist and midwife training is particularly relevant to limit medical complications and reduce maternal and fetal morbimortalities. Physical and virtual training simulators have been developed. However, physical simulators offer a simplified model and limited visualization of the childbirth process, while virtual simulators still lack a realistic interactive system and are generally limited to imposed predefined gestures. Objective performance assessment based on the simulation numerical outcomes is still not at hand. In the present work, we developed a virtual childbirth simulator based on the Mixed-Reality (MR) technology coupled with HyperMSM (Hyperelastic Mass-Spring Model) formulation for real-time soft-tissue deformations, providing intuitive user interaction with the virtual physical model and a quantitative assessment to enhance the trainee's gestures. Microsoft HoloLens 2 was used and the MR simulator was developed including a complete holographic obstetric model. A maternal pelvis system model of a pregnant woman (including the pelvis bone, the pelvic floor muscles, the birth canal, the uterus, and the fetus) was generated, and HyperMSM formulation was applied to simulate the soft tissue deformations. To induce realistic reactions to free gestures, the virtual replicas of the user's detected hands were introduced into the physical simulation and were associated with a contact model between the hands and the HyperMSM models. The gesture of pulling any part of the virtual models with two hands was also implemented. Two labor scenarios were implemented within the MR childbirth simulator: physiological labor and forceps-assisted labor. A scoring system for the performance assessment was included based on real-time biofeedback. As results, our developed MR simulation application was developed in real-time with a refresh rate of 30-50 FPS on the HoloLens device. HyperMSM model was validated using FE outcomes: high correlation coefficients of [0.97-0.99] and weighted root mean square relative errors of 9.8% and 8.3% were obtained for the soft tissue displacement and energy density respectively. Experimental tests showed that the implemented free-user interaction system allows to apply the correct maneuvers (in particular the "Viennese" maneuvers) during the labor process, and is capable to induce a truthful reaction of the model. Obtained results confirm also the possibility of using our simulation's outcomes to objectively evaluate the trainee's performance with a reduction of 39% for the perineal strain energy density and 5.6 mm for the vertical vaginal diameter when the "Viennese" technique is applied. This present study provides, for the first time, an interactive childbirth simulator with an MR immersive experience with direct free-hand interaction, real-time soft-tissue deformation feedback, and an objective performance assessment based on numerical outcomes. This offers a new perspective for enhancing next-generation training-based obstetric teaching. The used models of the maternal pelvic system and the fetus will be enhanced, and more delivery scenarios (e.g. instrumental delivery, breech delivery, shoulder dystocia) will be designed and integrated. The third stage of labor will be also investigated to include the delivery of the placenta, and the clamping and cutting of the umbilical cord.

Novel Baseline Facial Muscle Database Using Statistical Shape Modeling and In Silico Trials toward Decision Support for Facial Rehabilitation.

Backgrounds and Objective: Facial palsy is a complex pathophysiological condition affecting the personal and professional lives of the involved patients. Sudden muscle weakness or paralysis needs to be rehabilitated to recover a symmetric and expressive face. Computer-aided decision support systems for facial rehabilitation have been developed. However, there is a lack of facial muscle baseline data to evaluate the patient states and guide as well as optimize the rehabilitation strategy. In this present study, we aimed to develop a novel baseline facial muscle database (static and dynamic behaviors) using the coupling between statistical shape modeling and in-silico trial approaches. Methods: 10,000 virtual subjects (5000 males and 5000 females) were generated from a statistical shape modeling (SSM) head model. Skull and muscle networks were defined so that they statistically fit with the head shapes. Two standard mimics: smiling and kissing were generated. The muscle strains of the lengths in neutral and mimic positions were computed and recorded thanks to the muscle insertion and attachment points on the animated head and skull meshes. For validation, five head and skull meshes were reconstructed from the five computed tomography (CT) image sets. Skull and muscle networks were then predicted from the reconstructed head meshes. The predicted skull meshes were compared with the reconstructed skull meshes based on the mesh-to-mesh distance metrics. The predicted muscle lengths were also compared with those manually defined on the reconstructed head and skull meshes. Moreover, the computed muscle lengths and strains were compared with those in our previous studies and the literature. Results: The skull prediction's median deviations from the CT-based models were 2.2236 mm, 2.1371 mm, and 2.1277 mm for the skull shape, skull mesh, and muscle attachment point regions, respectively. The median deviation of the muscle lengths was 4.8940 mm. The computed muscle strains were compatible with the reported values in our previous Kinect-based method and the literature. Conclusions: The development of our novel facial muscle database opens new avenues to accurately evaluate the facial muscle states of facial palsy patients. Based on the evaluated results, specific types of facial mimic rehabilitation exercises can also be selected optimally to train the target muscles. In perspective, the database of the computed muscle lengths and strains will be integrated into our available clinical decision support system for automatically detecting malfunctioning muscles and proposing patient-specific rehabilitation serious games.

Fast 3D Face Reconstruction from a Single Image Using Different Deep Learning Approaches for Facial Palsy Patients.

The 3D reconstruction of an accurate face model is essential for delivering reliable feedback for clinical decision support. Medical imaging and specific depth sensors are accurate but not suitable for an easy-to-use and portable tool. The recent development of deep learning (DL) models opens new challenges for 3D shape reconstruction from a single image. However, the 3D face shape reconstruction of facial palsy patients is still a challenge, and this has not been investigated. The contribution of the present study is to apply these state-of-the-art methods to reconstruct the 3D face shape models of facial palsy patients in natural and mimic postures from one single image. Three different methods (3D Basel Morphable model and two 3D Deep Pre-trained models) were applied to the dataset of two healthy subjects and two facial palsy patients. The reconstructed outcomes were compared to the 3D shapes reconstructed using Kinect-driven and MRI-based information. As a result, the best mean error of the reconstructed face according to the Kinect-driven reconstructed shape is 1.5±1.1 mm. The best error range is 1.9±1.4 mm when compared to the MRI-based shapes. Before using the procedure to reconstruct the 3D faces of patients with facial palsy or other facial disorders, several ideas for increasing the accuracy of the reconstruction can be discussed based on the results. This present study opens new avenues for the fast reconstruction of the 3D face shapes of facial palsy patients from a single image. As perspectives, the best DL method will be implemented into our computer-aided decision support system for facial disorders.

Reinforcement learning coupled with finite element modeling for facial motion learning.

BACKGROUND AND OBJECTIVE: Facial palsy patients or patients with facial transplantation have abnormal facial motion due to altered facial muscle functions and nerve damage. Computer-aided system and physics-based models have been developed to provide objective and quantitative information. However, the predictive capacity of these solutions is still limited to explore the facial motion patterns with emerging properties. The present study aims to couple the reinforcement learning and the finite element modeling for facial motion learning and prediction. METHODS: A novel modeling workflow for learning facial motion was developed. A physically-based model of the face within the Artisynth modeling platform was used. Information exchange protocol was proposed to link reinforcement learning and rigid multi-bodies dynamics outcomes. Two reinforcement learning algorithms (deep deterministic policy gradient (DDPG) and Twin-delayed DDPG (TD3)) were used and implemented to drive the simulations of symmetry-oriented and smile movements. Numerical outcomes were compared to experimental observations (Bosphorus database) for evaluation and validation purposes. RESULTS: As result, after more than 100 episodes of exploring the environment, the agent starts to learn from previous trials and can find the optimal policy after more than 300 episodes of training. Regarding the symmetry-oriented motion, the muscle excitations predicted by the trained agent help to increase the value of reward from R = -2.06 to R = -0.23, which counts for ∼89% improvement of the symmetry value of the face. For smile-oriented motion, two points at the edge of the mouth move up 0.35 cm, which is within the range of movements estimated from the Bosphorus database (0.4 ± 0.32 cm). CONCLUSIONS: The present study explored the muscle excitation patterns by coupling reinforcement learning with a detailed finite element model of the face. We developed, for the first time, a novel coupling scheme to integrate the finite element simulation into the reinforcement learning process for facial motion learning. As perspectives, this present workflow will be applied for facial palsy and facial transplantation patients to guide and optimize the functional rehabilitation program.

Deep reinforcement learning coupled with musculoskeletal modelling for a better understanding of elderly falls.

Reinforcement learning (RL) has been used to study human locomotion learning. One of the current challenges in healthcare is our understanding of and ability to slow the decline due to muscle ageing and its effect on human falls. The purpose of this study was to investigate reinforcement learning for human movement strategies when modifying muscle parameters to account for age-related changes. In particular, human falls with modified physiological factors were modelled and simulated to determine the effect of muscle descriptors for ageing on kinematic behaviour and muscle force control. A 3D musculoskeletal model (8 DoF and 22 muscles) of the human body was used. The deep deterministic policy gradient (DDPG) method was implemented. Different muscle descriptors for ageing were integrated, including changes in maximum isometric force, contraction velocity, the deactivation time constant and passive muscle strain. Additionally, the effects of isometric force reductions of 10, 20 and 30% were also considered independently. An environment for the simulation was developed using the opensim-rl package for Python with the training process completed on Google Compute Engine. The simulation outcomes for healthy young adult and elderly falls under modified muscle behaviours were compared to experimental observations for validation. The result of our elderly simulation for multiple ageing-related factors (M_all) produced a walking speed of 0.26 m/s for the two steps taken prior to the fall. The over activation of the hip extensors and inactivation of knee extensors led to a backward fall for this elderly simulation. The inactivated rectus femoris and right tibialis are main actors of the forward fall. Our simulation outcomes are consistent with experimental observations through the comparison of kinematic features and motion history evolution. We showed in the present study, for the first time, that RL can be used as a strategy to explore the effect of ageing muscle physiological factors on kinematics and muscle control during falls. Our findings show that the elderly fall model for the M_all condition more closely resembles experimental elderly fall data than our simulations which considered age-related reductions of force alone. As future perspectives, the behaviour preceding a fall will be studied to establish the strategies used to avoid falls or fall with minimal consequence, leading to the identification of patient-specific rehabilitation programmes for elderly people.

Recurrent neural network to predict hyperelastic constitutive behaviors of the skeletal muscle.

Hyperelastic constitutive laws have been commonly used to model the passive behavior of the human skeletal muscle. Despite many efforts, the use of accurate finite element formulations of hyperelastic constitutive laws is still time-consuming for a real-time medical simulation system. The objective of the present study was to develop a deep learning model to predict the hyperelastic constitutive behaviors of the skeletal muscle toward a fast estimation of the muscle tissue stress.A finite element (FE) model of the right psoas muscle was developed. Neo-Hookean and Mooney-Rivlin laws were used. A tensile test was performed with an applied body force. A learning database was built from this model using an automatic probabilistic generation process. A long-short term memory (LSTM) neural network was implemented to predict the stress evolution of the skeletal muscle tissue. A hyperparameter tuning process was conducted. Root mean square error (RMSE) and associated relative error was quantified to evaluate the precision of the predictive capacity of the developed deep learning model. Pearson correlation coefficients (R) was also computed.The nodal displacements and the maximal stresses range from 70 to 227 mm and from 2.79 to 5.61 MPa for Neo-Hookean and Monney-Rivlin laws, respectively. Regarding the LSTM predictions, the RMSE ranges from 224.3 ± 3.9 Pa (8%) to 227.5 [Formula: see text] 5.7 Pa (4%) for Neo-Hookean and Monney-Rivlin laws, respectively. Pearson correlation coefficients (R) of 0.78 [Formula: see text] 0.02 and 0.77 [Formula: see text] 0.02 were obtained for Neo-Hookean and Monney-Rivlin laws, respectively.The present study showed that, for the first time, the use of a deep learning model can reproduce the time-series behaviors of the complex FE formulations for skeletal muscle modeling. In particular, the use of a LSTM neural network leads to a fast and accurate surrogate model for the in silico prediction of the hyperelastic constitutive behaviors of the skeletal muscle. As perspectives, the developed deep learning model will be integrated into a real-time medical simulation of the skeletal muscle for prosthetic socket design and childbirth simulator.

HyperMSM: A new MSM variant for efficient simulation of dynamic soft-tissue deformations.

BACKGROUND AND OBJECTIVE: Fast, accurate, and stable simulation of soft tissue deformation is a challenging task. Mass-Spring Model (MSM) is one of the popular methods used for this purpose for its simple implementation and potential to provide fast dynamic simulations. However, accurately simulating a non-linear material within the mass-spring framework is still challenging. The objective of the present study is to develop and evaluate a new efficient hyperelastic Mass-Spring Model formulation to simulate the Neo-Hookean deformable material, called HyperMSM. METHODS: Our novel HyperMSM formulation is applicable for both tetrahedral and hexahedral mesh configurations and is compatible with the original projective dynamics solver. In particular, the proposed MSM variant includes springs with variable rest-lengths and a volume conservation constraint. Two applications (transtibial residual limb and the skeletal muscle) were conducted. RESULTS: Compared to finite element simulations, obtained results show RMSE ranges of [2.8%-5.2%] and [0.46%-5.4%] for stress-strain and volumetric responses respectively for strains ranging from -50% to +100%. The displacement error range in our transtibial residual limb simulation is around [0.01mm-0.7 mm]. The RMSE range of relative nodal displacements for the skeletal psoas muscle model is [0.4%-1.7%]. CONCLUSIONS: Our novel HyperMSM formulation allows hyperelastic behavior of soft tissues to be described accurately and efficiently within the mass-spring framework. As perspectives, our formulation will be enhanced with electric behavior toward a multi-physical soft tissue mass-spring modeling framework. Then, the coupling with an augmented reality environment will be performed.

Enhanced head-skull shape learning using statistical modeling and topological features.

Skull prediction from the head is a challenging issue toward a cost-effective therapeutic solution for facial disorders. This issue was initially studied in our previous work using full head-to-skull relationship learning. However, the head-skull thickness topology is locally shaped, especially in the face region. Thus, the objective of the present study was to enhance our head-to-skull prediction problem by using local topological features for training and predicting. Head and skull feature points were sampled on 329 head and skull models from computed tomography (CT) images. These feature points were classified into the back and facial topologies. Head-to-skull relations were trained using the partial least square regression (PLSR) models separately in the two topologies. A hyperparameter tuning process was also conducted for selecting optimal parameters for each training model. Thus, a new skull could be generated so that its shape was statistically fitted with the target head. Mean errors of the predicted skulls using the topology-based learning method were better than those using the non-topology-based learning method. After tenfold cross-validation, the mean error was enhanced 36.96% for the skull shapes and 14.17% for the skull models. Mean error in the facial skull region was especially improved with 4.98%. The mean errors were also improved 11.71% and 25.74% in the muscle attachment regions and the back skull regions respectively. Moreover, using the enhanced learning strategy, the errors (mean ± SD) for the best and worst prediction cases are from 1.1994 ± 1.1225 mm (median: 0.9036, coefficient of multiple determination (R2): 0.997274) to 3.6972 ± 2.4118 mm (median: 3.9089, R2: 0.999614) and from 2.0172 ± 2.0454 mm (median: 1.2999, R2: 0.995959) to 4.0227 ± 2.6098 mm (median: 3.9998, R2: 0.998577) for the predicted skull shapes and the predicted skull models respectively. This present study showed that more detailed information on the head-skull shape leads to a better accuracy level for the skull prediction from the head. In particular, local topological features on the back and face regions of interest should be considered toward a better learning strategy for the head-to-skull prediction problem. In perspective, this enhanced learning strategy was used to update our developed clinical decision support system for facial disorders. Furthermore, a new class of learning methods, called geometric deep learning will be studied.

Enhanced facial expression recognition using 3D point sets and geometric deep learning.

Facial expression recognition plays an essential role in human conversation and human-computer interaction. Previous research studies have recognized facial expressions mainly based on 2D image processing requiring sensitive feature engineering and conventional machine learning approaches. The purpose of the present study was to recognize facial expressions by applying a new class of deep learning called geometric deep learning directly on 3D point cloud data. Two databases (Bosphorus and SIAT-3DFE) were used. The Bosphorus database includes sixty-five subjects with seven basic expressions (i.e., anger, disgust, fearness, happiness, sadness, surprise, and neutral). The SIAT-3DFE database has 150 subjects and 4 basic facial expressions (neutral, happiness, sadness, and surprise). First, preprocessing procedures such as face center cropping, data augmentation, and point cloud denoising were applied on 3D face scans. Then, a geometric deep learning model called PointNet++ was applied. A hyperparameter tuning process was performed to find the optimal model parameters. Finally, the developed model was evaluated using the recognition rate and confusion matrix. The facial expression recognition accuracy on the Bosphorus database was 69.01% for 7 expressions and could reach 85.85% when recognizing five specific expressions (anger, disgust, happiness, surprise, and neutral). The recognition rate was 78.70% with the SIAT-3DFE database. The present study suggested that 3D point cloud could be directly processed for facial expression recognition by using geometric deep learning approach. In perspectives, the developed model will be applied for facial palsy patients to guide and optimize the functional rehabilitation program.

An Early Stage Researcher's Primer on Systems Medicine Terminology.

Background: Systems Medicine is a novel approach to medicine, that is, an interdisciplinary field that considers the human body as a system, composed of multiple parts and of complex relationships at multiple levels, and further integrated into an environment. Exploring Systems Medicine implies understanding and combining concepts coming from diametral different fields, including medicine, biology, statistics, modeling and simulation, and data science. Such heterogeneity leads to semantic issues, which may slow down implementation and fruitful interaction between these highly diverse fields. Methods: In this review, we collect and explain more than100 terms related to Systems Medicine. These include both modeling and data science terms and basic systems medicine terms, along with some synthetic definitions, examples of applications, and lists of relevant references. Results: This glossary aims at being a first aid kit for the Systems Medicine researcher facing an unfamiliar term, where he/she can get a first understanding of them, and, more importantly, examples and references for digging into the topic.

Human locomotion with reinforcement learning using bioinspired reward reshaping strategies.

Recent learning strategies such as reinforcement learning (RL) have favored the transition from applied artificial intelligence to general artificial intelligence. One of the current challenges of RL in healthcare relates to the development of a controller to teach a musculoskeletal model to perform dynamic movements. Several solutions have been proposed. However, there is still a lack of investigations exploring the muscle control problem from a biomechanical point of view. Moreover, no studies using biological knowledge to develop plausible motor control models for pathophysiological conditions make use of reward reshaping. Consequently, the objective of the present work was to design and evaluate specific bioinspired reward function strategies for human locomotion learning within an RL framework. The deep deterministic policy gradient (DDPG) method for a single-agent RL problem was applied. A 3D musculoskeletal model (8 DoF and 22 muscles) of a healthy adult was used. A virtual interactive environment was developed and simulated using opensim-rl library. Three reward functions were defined for walking, forward, and side falls. The training process was performed with Google Cloud Compute Engine. The obtained outcomes were compared to the NIPS 2017 challenge outcomes, experimental observations, and literature data. Regarding learning to walk, simulated musculoskeletal models were able to walk from 18 to 20.5 m for the best solutions. A compensation strategy of muscle activations was revealed. Soleus, tibia anterior, and vastii muscles are main actors of the simple forward fall. A higher intensity of muscle activations was also noted after the fall. All kinematics and muscle patterns were consistent with experimental observations and literature data. Regarding the side fall, an intensive level of muscle activation on the expected fall side to unbalance the body was noted. The obtained outcomes suggest that computational and human resources as well as biomechanical knowledge are needed together to develop and evaluate an efficient and robust RL solution. As perspectives, current solutions will be extended to a larger parameter space in 3D. Furthermore, a stochastic reinforcement learning model will be investigated in the future in scope with the uncertainties of the musculoskeletal model and associated environment to provide a general artificial intelligence solution for human locomotion learning. Graphical abstract.

Kinect-driven Patient-specific Head, Skull, and Muscle Network Modelling for Facial Palsy Patients.

BACKGROUND AND OBJECTIVE: Facial palsy negatively affects both professional and personal life qualities of involved patients. Classical facial rehabilitation strategies can recover facial mimics into their normal and symmetrical movements and appearances. However, there is a lack of objective, quantitative, and in-vivo facial texture and muscle activation bio-feedbacks for personalizing rehabilitation programs and diagnosing recovering progresses. Consequently, this study proposed a novel patient-specific modelling method for generating a full patient specific head model from a visual sensor and then computing the facial texture and muscle activation in real-time for further clinical decision making. METHODS: The modeling workflow includes (1) Kinect-to-head, (2) head-to-skull, and (3) muscle network definition & generation processes. In the Kinect-to-head process, subject-specific data acquired from a new user in neutral mimic were used for generating his/her geometrical head model with facial texture. In particular, a template head model was deformed to optimally fit with high-definition facial points acquired by the Kinect sensor. Moreover, the facial texture was also merged from his/her facial images in left, right, and center points of view. In the head-to-skull process, a generic skull model was deformed so that its shape was statistically fitted with his/her geometrical head model. In the muscle network definition & generation process, a muscle network was defined from the head and skull models for computing muscle strains during facial movements. Muscle insertion points and muscle attachment points were defined as vertex positions on the head model and the skull model respectively based on the standard facial anatomy. Three healthy subjects and two facial palsy patients were selected for validating the proposed method. In neutral positions, magnetic resonance imaging (MRI)-based head and skull models were compared with Kinect-based head and skull models. In mimic positions, infrared depth-based head models in smiling and [u]-pronouncing mimics were compared with appropriate animated Kinect-driven head models. The Hausdorff distance metric was used for these comparisons. Moreover, computed muscle lengths and strains in the tested facial mimics were validated with reported values in literature. RESULTS: With the current hardware configuration, the patient-specific head model with skull and muscle network could be fast generated within 17.16±0.37s and animated in real-time with the framerate of 40 fps. In neutral positions, the best mean error was 1.91 mm for the head models and 3.21 mm for the skull models. On facial regions, the best mean errors were 1.53 mm and 2.82 mm for head and skull models respectively. On muscle insertion/attachment point regions, the best mean errors were 1.09 mm and 2.16 mm for head and skull models respectively. In mimic positions, these errors were 2.02 mm in smiling mimics and 2.00 mm in [u]-pronouncing mimics for the head models on facial regions. All above error values were computed on a one-time validation procedure. Facial muscles exhibited muscle shortening and muscle elongating for smiling and pronunciation of sound [u] respectively. Extracted muscle features (i.e. muscle length and strain) are in agreement with experimental and literature data. CONCLUSIONS: This study proposed a novel modeling method for fast generating and animating patient-specific biomechanical head model with facial texture and muscle activation bio-feedbacks. The Kinect-driven muscle strains could be applied for further real-time muscle-oriented facial paralysis grading and other facial analysis applications.

A statistical shape modeling approach for predicting subject-specific human skull from head surface.

Human skull is an important body structure for jaw movement and facial mimic simulations. Surface head can be reconstructed using 3D scanners in a straightforward way. However, internal skull is challenging to be generated when only external information is available. Very few studies in the literature focused on the skull generation from outside head information, especially in a subject-specific manner with a complete skull. Consequently, this present study proposes a novel process for predicting a subject-specific skull with full details from a given head surface using a statistical shape modeling approach. Partial least squared regression (PLSR)-based method was used. A CT image database of 209 subjects (genders-160 males and 49 females; ages-34-88 years) was used for learning head-to-skull relationship. Heads and skulls were reconstructed from CT images to extract head/skull feature points, head/skull feature distances, head-skull thickness, and head/skull volume descriptors for the learning process. A hyperparameter turning process was performed to determine the optimal numbers of head/skull feature points, PLSR components, deformation control points, and appropriate learning strategies for our learning problem. Two learning strategies (point-to-thickness with/without volume descriptor and distance-to-thickness with/without volume descriptor) were proposed. Moreover, a 10-fold cross-validation procedure was conducted to evaluate the accuracy of the proposed learning strategies. Finally, the best and worst reconstructed skulls were analyzed based on the best learning strategy with its optimal parameters. The optimal number of head/skull feature points and deformation control points are 2300 and 1300 points, respectively. The optimal number of PLSR components ranges from 4 to 8 for all learning configurations. Cross-validation showed that grand means and standard deviations of the point-to-thickness, point-to-thickness with volumes, distance-to-thickness, and distance-to-thickness with volumes learning configurations are 2.48 ± 0.27 mm, 2.46 ± 0.19 mm, 2.46 ± 0.15 mm, and 2.48 ± 0.22 mm, respectively. Thus, the distance-to-thickness is the best learning configuration for our head-to-skull prediction problem. Moreover, the mean Hausdorff distances are 2.09 ± 0.15 mm and 2.64 ± 0.26 mm for the best and worst predicted skull, respectively. A novel head-to-skull prediction process based on the PLSR method was developed and evaluated. This process allows, for the first time, predicting 3D subject-specific human skulls from head surface information with a very good accuracy level. As perspective, the proposed head-to-skull prediction process will be integrated into our real-time computer-aided vision system for facial animation and rehabilitation. Graphical abstract.

Real-time computer vision system for tracking simultaneously subject-specific rigid head and non-rigid facial mimic movements using a contactless sensor and system of systems approach.

BACKGROUND AND OBJECTIVE: Head and facial mimic animations play important roles in various fields such as human-machine interactions, internet communications, multimedia applications, and facial mimic analysis. Numerous studies have been trying to simulate these animations. However, they hardly achieved all requirements of full rigid head and non-rigid facial mimic animations in a subject-specific manner with real-time framerates. Consequently, this present study aimed to develop a real-time computer vision system for tracking simultaneously rigid head and non-rigid facial mimic movements. METHODS: Our system was developed using the system of systems approach. A data acquisition sub-system was implemented using a contactless Kinect sensor. A subject-specific model generation sub-system was designed to create the geometrical model from the Kinect sensor without texture information. A subject-specific texture generation sub-system was designed for enhancing the reality of the generated model with texture information. A head animation sub-system with graphical user interfaces was also developed. Model accuracy and system performances were analyzed. RESULTS: The comparison with MRI-based model shows a very good accuracy level (distance deviation of ~1 mm in neutral position and an error range of [2-3 mm] for different facial mimic positions) for the generated model from our system. Moreover, the system speed can be optimized to reach a high framerate (up to 60 fps) during different head and facial mimic animations. CONCLUSIONS: This study presents a novel computer vision system for tracking simultaneously subject-specific rigid head and non-rigid facial mimic movements in real time. In perspectives, serious game technology will be integrated into this system towards a full computer-aided decision support system for facial rehabilitation.

A Systematic Review of Real-Time Medical Simulations with Soft-Tissue Deformation: Computational Approaches, Interaction Devices, System Architectures, and Clinical Validations.

Simulating deformations of soft tissues is a complex engineering task, and it is even more difficult when facing the constraint between computation speed and system accuracy. However, literature lacks of a holistic review of all necessary aspects (computational approaches, interaction devices, system architectures, and clinical validations) for developing an effective system of soft-tissue simulations. This paper summarizes and analyses recent achievements of resolving these issues to estimate general trends and weakness for future developments. A systematic review process was conducted using the PRISMA protocol with three reliable scientific search engines (ScienceDirect, PubMed, and IEEE). Fifty-five relevant papers were finally selected and included into the review process, and a quality assessment procedure was also performed on them. The computational approaches were categorized into mesh, meshfree, and hybrid approaches. The interaction devices concerned about combination between virtual surgical instruments and force-feedback devices, 3D scanners, biomechanical sensors, human interface devices, 3D viewers, and 2D/3D optical cameras. System architectures were analysed based on the concepts of system execution schemes and system frameworks. In particular, system execution schemes included distribution-based, multithread-based, and multimodel-based executions. System frameworks are grouped into the input and output interaction frameworks, the graphic interaction frameworks, the modelling frameworks, and the hybrid frameworks. Clinical validation procedures are ordered as three levels: geometrical validation, model behavior validation, and user acceptability/safety validation. The present review paper provides useful information to characterize how real-time medical simulation systems with soft-tissue deformations have been developed. By clearly analysing advantages and drawbacks in each system development aspect, this review can be used as a reference guideline for developing systems of soft-tissue simulations.