Foetus-specific neuromusculoskeletal modelling with MRI-driven vaginal delivery kinematics during the second stage of labor.

Childbirth simulations lack realism due to an oversimplification of the foetal model, particularly as most models do not allow joint motion. Foetus-specific neuromusculoskeletal (NMS) model with a detailed articulated skeleton is still not available in the literature. The present work aims at proposing the first-ever foetus-specific NMS model and then simulating the foetal descent during a vaginal delivery by using in vivo medical resonance imaging (MRI) childbirth data. Moreover, the developed model is provided open source for the community. Our foetus-specific NMS model was developed using the geometries reconstructed from a foetal computed tomography (CT) scan (Female, mass = 2.35 kg, length = 50 cm). The model contains 22 joints (64 degrees of freedom) and 65 muscles with a particular attention to the cervical spine level to enable the simulation of the cardinal movements. Then, the skull-to-cervical-spine (S/CP) and cervical-spine-to-torso (CP/T) deflection angles were extracted from in vivo MRI data for motion simulation. The S/CP and CP/T deflexion angles range from 12 degrees of flexion to 2 degrees of extension and from 7 degrees of flexion to 22 degrees of extension respectively. The developed model opens new avenues in more biofidelic childbirth simulations with a complete foetal NMS model. Obtained outcomes with the in vivo MRI data enabled to perform a first simulation of the foetal descent kinematics using real childbirth data. Future works will focus on developing a novel muscle formulation of the foetus and combining such a NMS model with a deformable model to simulate childbirth and associated complication scenarios.

Novel Baseline Facial Muscle Database Using Statistical Shape Modeling and In Silico Trials toward Decision Support for Facial Rehabilitation.

Backgrounds and Objective: Facial palsy is a complex pathophysiological condition affecting the personal and professional lives of the involved patients. Sudden muscle weakness or paralysis needs to be rehabilitated to recover a symmetric and expressive face. Computer-aided decision support systems for facial rehabilitation have been developed. However, there is a lack of facial muscle baseline data to evaluate the patient states and guide as well as optimize the rehabilitation strategy. In this present study, we aimed to develop a novel baseline facial muscle database (static and dynamic behaviors) using the coupling between statistical shape modeling and in-silico trial approaches. Methods: 10,000 virtual subjects (5000 males and 5000 females) were generated from a statistical shape modeling (SSM) head model. Skull and muscle networks were defined so that they statistically fit with the head shapes. Two standard mimics: smiling and kissing were generated. The muscle strains of the lengths in neutral and mimic positions were computed and recorded thanks to the muscle insertion and attachment points on the animated head and skull meshes. For validation, five head and skull meshes were reconstructed from the five computed tomography (CT) image sets. Skull and muscle networks were then predicted from the reconstructed head meshes. The predicted skull meshes were compared with the reconstructed skull meshes based on the mesh-to-mesh distance metrics. The predicted muscle lengths were also compared with those manually defined on the reconstructed head and skull meshes. Moreover, the computed muscle lengths and strains were compared with those in our previous studies and the literature. Results: The skull prediction's median deviations from the CT-based models were 2.2236 mm, 2.1371 mm, and 2.1277 mm for the skull shape, skull mesh, and muscle attachment point regions, respectively. The median deviation of the muscle lengths was 4.8940 mm. The computed muscle strains were compatible with the reported values in our previous Kinect-based method and the literature. Conclusions: The development of our novel facial muscle database opens new avenues to accurately evaluate the facial muscle states of facial palsy patients. Based on the evaluated results, specific types of facial mimic rehabilitation exercises can also be selected optimally to train the target muscles. In perspective, the database of the computed muscle lengths and strains will be integrated into our available clinical decision support system for automatically detecting malfunctioning muscles and proposing patient-specific rehabilitation serious games.

Fast 3D Face Reconstruction from a Single Image Using Different Deep Learning Approaches for Facial Palsy Patients.

The 3D reconstruction of an accurate face model is essential for delivering reliable feedback for clinical decision support. Medical imaging and specific depth sensors are accurate but not suitable for an easy-to-use and portable tool. The recent development of deep learning (DL) models opens new challenges for 3D shape reconstruction from a single image. However, the 3D face shape reconstruction of facial palsy patients is still a challenge, and this has not been investigated. The contribution of the present study is to apply these state-of-the-art methods to reconstruct the 3D face shape models of facial palsy patients in natural and mimic postures from one single image. Three different methods (3D Basel Morphable model and two 3D Deep Pre-trained models) were applied to the dataset of two healthy subjects and two facial palsy patients. The reconstructed outcomes were compared to the 3D shapes reconstructed using Kinect-driven and MRI-based information. As a result, the best mean error of the reconstructed face according to the Kinect-driven reconstructed shape is 1.5±1.1 mm. The best error range is 1.9±1.4 mm when compared to the MRI-based shapes. Before using the procedure to reconstruct the 3D faces of patients with facial palsy or other facial disorders, several ideas for increasing the accuracy of the reconstruction can be discussed based on the results. This present study opens new avenues for the fast reconstruction of the 3D face shapes of facial palsy patients from a single image. As perspectives, the best DL method will be implemented into our computer-aided decision support system for facial disorders.

Enhanced head-skull shape learning using statistical modeling and topological features.

Skull prediction from the head is a challenging issue toward a cost-effective therapeutic solution for facial disorders. This issue was initially studied in our previous work using full head-to-skull relationship learning. However, the head-skull thickness topology is locally shaped, especially in the face region. Thus, the objective of the present study was to enhance our head-to-skull prediction problem by using local topological features for training and predicting. Head and skull feature points were sampled on 329 head and skull models from computed tomography (CT) images. These feature points were classified into the back and facial topologies. Head-to-skull relations were trained using the partial least square regression (PLSR) models separately in the two topologies. A hyperparameter tuning process was also conducted for selecting optimal parameters for each training model. Thus, a new skull could be generated so that its shape was statistically fitted with the target head. Mean errors of the predicted skulls using the topology-based learning method were better than those using the non-topology-based learning method. After tenfold cross-validation, the mean error was enhanced 36.96% for the skull shapes and 14.17% for the skull models. Mean error in the facial skull region was especially improved with 4.98%. The mean errors were also improved 11.71% and 25.74% in the muscle attachment regions and the back skull regions respectively. Moreover, using the enhanced learning strategy, the errors (mean ± SD) for the best and worst prediction cases are from 1.1994 ± 1.1225 mm (median: 0.9036, coefficient of multiple determination (R2): 0.997274) to 3.6972 ± 2.4118 mm (median: 3.9089, R2: 0.999614) and from 2.0172 ± 2.0454 mm (median: 1.2999, R2: 0.995959) to 4.0227 ± 2.6098 mm (median: 3.9998, R2: 0.998577) for the predicted skull shapes and the predicted skull models respectively. This present study showed that more detailed information on the head-skull shape leads to a better accuracy level for the skull prediction from the head. In particular, local topological features on the back and face regions of interest should be considered toward a better learning strategy for the head-to-skull prediction problem. In perspective, this enhanced learning strategy was used to update our developed clinical decision support system for facial disorders. Furthermore, a new class of learning methods, called geometric deep learning will be studied.

Kinect-driven Patient-specific Head, Skull, and Muscle Network Modelling for Facial Palsy Patients.

BACKGROUND AND OBJECTIVE: Facial palsy negatively affects both professional and personal life qualities of involved patients. Classical facial rehabilitation strategies can recover facial mimics into their normal and symmetrical movements and appearances. However, there is a lack of objective, quantitative, and in-vivo facial texture and muscle activation bio-feedbacks for personalizing rehabilitation programs and diagnosing recovering progresses. Consequently, this study proposed a novel patient-specific modelling method for generating a full patient specific head model from a visual sensor and then computing the facial texture and muscle activation in real-time for further clinical decision making. METHODS: The modeling workflow includes (1) Kinect-to-head, (2) head-to-skull, and (3) muscle network definition & generation processes. In the Kinect-to-head process, subject-specific data acquired from a new user in neutral mimic were used for generating his/her geometrical head model with facial texture. In particular, a template head model was deformed to optimally fit with high-definition facial points acquired by the Kinect sensor. Moreover, the facial texture was also merged from his/her facial images in left, right, and center points of view. In the head-to-skull process, a generic skull model was deformed so that its shape was statistically fitted with his/her geometrical head model. In the muscle network definition & generation process, a muscle network was defined from the head and skull models for computing muscle strains during facial movements. Muscle insertion points and muscle attachment points were defined as vertex positions on the head model and the skull model respectively based on the standard facial anatomy. Three healthy subjects and two facial palsy patients were selected for validating the proposed method. In neutral positions, magnetic resonance imaging (MRI)-based head and skull models were compared with Kinect-based head and skull models. In mimic positions, infrared depth-based head models in smiling and [u]-pronouncing mimics were compared with appropriate animated Kinect-driven head models. The Hausdorff distance metric was used for these comparisons. Moreover, computed muscle lengths and strains in the tested facial mimics were validated with reported values in literature. RESULTS: With the current hardware configuration, the patient-specific head model with skull and muscle network could be fast generated within 17.16±0.37s and animated in real-time with the framerate of 40 fps. In neutral positions, the best mean error was 1.91 mm for the head models and 3.21 mm for the skull models. On facial regions, the best mean errors were 1.53 mm and 2.82 mm for head and skull models respectively. On muscle insertion/attachment point regions, the best mean errors were 1.09 mm and 2.16 mm for head and skull models respectively. In mimic positions, these errors were 2.02 mm in smiling mimics and 2.00 mm in [u]-pronouncing mimics for the head models on facial regions. All above error values were computed on a one-time validation procedure. Facial muscles exhibited muscle shortening and muscle elongating for smiling and pronunciation of sound [u] respectively. Extracted muscle features (i.e. muscle length and strain) are in agreement with experimental and literature data. CONCLUSIONS: This study proposed a novel modeling method for fast generating and animating patient-specific biomechanical head model with facial texture and muscle activation bio-feedbacks. The Kinect-driven muscle strains could be applied for further real-time muscle-oriented facial paralysis grading and other facial analysis applications.

A statistical shape modeling approach for predicting subject-specific human skull from head surface.

Human skull is an important body structure for jaw movement and facial mimic simulations. Surface head can be reconstructed using 3D scanners in a straightforward way. However, internal skull is challenging to be generated when only external information is available. Very few studies in the literature focused on the skull generation from outside head information, especially in a subject-specific manner with a complete skull. Consequently, this present study proposes a novel process for predicting a subject-specific skull with full details from a given head surface using a statistical shape modeling approach. Partial least squared regression (PLSR)-based method was used. A CT image database of 209 subjects (genders-160 males and 49 females; ages-34-88 years) was used for learning head-to-skull relationship. Heads and skulls were reconstructed from CT images to extract head/skull feature points, head/skull feature distances, head-skull thickness, and head/skull volume descriptors for the learning process. A hyperparameter turning process was performed to determine the optimal numbers of head/skull feature points, PLSR components, deformation control points, and appropriate learning strategies for our learning problem. Two learning strategies (point-to-thickness with/without volume descriptor and distance-to-thickness with/without volume descriptor) were proposed. Moreover, a 10-fold cross-validation procedure was conducted to evaluate the accuracy of the proposed learning strategies. Finally, the best and worst reconstructed skulls were analyzed based on the best learning strategy with its optimal parameters. The optimal number of head/skull feature points and deformation control points are 2300 and 1300 points, respectively. The optimal number of PLSR components ranges from 4 to 8 for all learning configurations. Cross-validation showed that grand means and standard deviations of the point-to-thickness, point-to-thickness with volumes, distance-to-thickness, and distance-to-thickness with volumes learning configurations are 2.48 ± 0.27 mm, 2.46 ± 0.19 mm, 2.46 ± 0.15 mm, and 2.48 ± 0.22 mm, respectively. Thus, the distance-to-thickness is the best learning configuration for our head-to-skull prediction problem. Moreover, the mean Hausdorff distances are 2.09 ± 0.15 mm and 2.64 ± 0.26 mm for the best and worst predicted skull, respectively. A novel head-to-skull prediction process based on the PLSR method was developed and evaluated. This process allows, for the first time, predicting 3D subject-specific human skulls from head surface information with a very good accuracy level. As perspective, the proposed head-to-skull prediction process will be integrated into our real-time computer-aided vision system for facial animation and rehabilitation. Graphical abstract.

Real-time computer vision system for tracking simultaneously subject-specific rigid head and non-rigid facial mimic movements using a contactless sensor and system of systems approach.

BACKGROUND AND OBJECTIVE: Head and facial mimic animations play important roles in various fields such as human-machine interactions, internet communications, multimedia applications, and facial mimic analysis. Numerous studies have been trying to simulate these animations. However, they hardly achieved all requirements of full rigid head and non-rigid facial mimic animations in a subject-specific manner with real-time framerates. Consequently, this present study aimed to develop a real-time computer vision system for tracking simultaneously rigid head and non-rigid facial mimic movements. METHODS: Our system was developed using the system of systems approach. A data acquisition sub-system was implemented using a contactless Kinect sensor. A subject-specific model generation sub-system was designed to create the geometrical model from the Kinect sensor without texture information. A subject-specific texture generation sub-system was designed for enhancing the reality of the generated model with texture information. A head animation sub-system with graphical user interfaces was also developed. Model accuracy and system performances were analyzed. RESULTS: The comparison with MRI-based model shows a very good accuracy level (distance deviation of ~1 mm in neutral position and an error range of [2-3 mm] for different facial mimic positions) for the generated model from our system. Moreover, the system speed can be optimized to reach a high framerate (up to 60 fps) during different head and facial mimic animations. CONCLUSIONS: This study presents a novel computer vision system for tracking simultaneously subject-specific rigid head and non-rigid facial mimic movements in real time. In perspectives, serious game technology will be integrated into this system towards a full computer-aided decision support system for facial rehabilitation.

A Systematic Review of Real-Time Medical Simulations with Soft-Tissue Deformation: Computational Approaches, Interaction Devices, System Architectures, and Clinical Validations.

Simulating deformations of soft tissues is a complex engineering task, and it is even more difficult when facing the constraint between computation speed and system accuracy. However, literature lacks of a holistic review of all necessary aspects (computational approaches, interaction devices, system architectures, and clinical validations) for developing an effective system of soft-tissue simulations. This paper summarizes and analyses recent achievements of resolving these issues to estimate general trends and weakness for future developments. A systematic review process was conducted using the PRISMA protocol with three reliable scientific search engines (ScienceDirect, PubMed, and IEEE). Fifty-five relevant papers were finally selected and included into the review process, and a quality assessment procedure was also performed on them. The computational approaches were categorized into mesh, meshfree, and hybrid approaches. The interaction devices concerned about combination between virtual surgical instruments and force-feedback devices, 3D scanners, biomechanical sensors, human interface devices, 3D viewers, and 2D/3D optical cameras. System architectures were analysed based on the concepts of system execution schemes and system frameworks. In particular, system execution schemes included distribution-based, multithread-based, and multimodel-based executions. System frameworks are grouped into the input and output interaction frameworks, the graphic interaction frameworks, the modelling frameworks, and the hybrid frameworks. Clinical validation procedures are ordered as three levels: geometrical validation, model behavior validation, and user acceptability/safety validation. The present review paper provides useful information to characterize how real-time medical simulation systems with soft-tissue deformations have been developed. By clearly analysing advantages and drawbacks in each system development aspect, this review can be used as a reference guideline for developing systems of soft-tissue simulations.