3D Body shape for regional and appendicular body composition estimation.

Body composition is correlated to bone mineral density, muscle strength, and physical performance. This is important for diagnosing conditions like sarcopenia, which is defined as the age-associated decrease in muscle mass leading to decreased mobile function, increased frailty, and imbalance. Existing methods for body composition measurement either suffer from inaccurate results or require expensive equipment such as Dual-energy x-ray absorptiometry (DXA). Although DXA measures lean mass and not muscle mass, previous studies have considered extremity lean mass as appendicular skeletal muscle mass (ASMM) approximation. In this study, we develop a new shape descriptor to predict regional body composition (in particular, regional lean mass) from 3D body shapes. In addition, we propose a neural network for ASMM assessment which is calculated by lean mass. We evaluate the effectiveness by comparing adjusted R-Squared values and Root Mean Square Error (RMSE). In our experiment, the regression models utilizing level circumference as the training feature outperforms all regional anthropometric measurements and lowers the average RMSE by about 21%. For ASMM, the proposed neural network, which combines shape features and demographic features, surpasses all other traditional regression models and reaches the lowest RMSE at 1.85 kg. Compared to the vanilla linear regression model, our approach improves the RMSE by 17%. The experimental results suggest that the 3D body shape has the potential to be used to predict body composition, and in particular lean mass, for the whole body as well as specific regions of the body.

An Automatic Grading System for Neonatal Endotracheal Intubation with Multi-Task Convolutional Neural Network.

Neonatal endotracheal intubation (ETI) is an intricate medical procedure that poses considerable challenges, demanding comprehensive training to effectively address potential complications in clinical practice. However, due to limited access to clinical opportunities, ETI training relies heavily on physical manikins to develop a certain level of competence before clinical exposure. Nonetheless, traditional training methods prove ineffective due to scarcity of expert instructors and the absence of internal situational awareness within the manikins, preventing thorough performance assessment for both trainees and instructors. To address this gap, there is a need to develop an automatic grading system that can assist trainees in performance assessment. In this paper, we proposed a multi-task Convolutional Neural Network (MTCNN) based model for assessing ETI proficiency, specifically targeting key performance features recommended by expert instructors. The model comprises three modules: an ETI simulation module that captures the ETI procedures performed on a standard neonatal task trainer manikin, an automatic grading module that extracts and grades the identified key performance features, and a data visualization module that presents the assessment results in a user-friendly manner. The experimental results demonstrated that the proposed automatic grading system achieved an average classification accuracy of 93.6%. This study established the successful integration of intuitive observed features with latent features derived from multivariate time series (MTS) data, coupled with multi-task deep learning techniques, for the automatic assessment of ETI performance. Clinical relevance­: The proposed automatic grading system facilitates an enhanced neonatal endotracheal intubation training experience for neonatologists.

Liver Fat Assessment with Body Shape.

Hepatic steatosis has become a serious health concern among the general population, but especially for those who are obese. Liver fat can increase the risk of cirrhosis and even liver cancer. Current standard methods to assess hepatic steatosis, such as liver biopsy and CT/MR imaging techniques, are expensive and/or may have associated risks to health. In this paper, we use body shapes to assess hepatic steatosis using both traditional linear regression models and a deep neural network. We apply our models to a medical dataset and evaluate the approaches for both regression and classification. We compare the performance of several models via popular evaluation metrics. The experimental results indicate that our proposed neural network outperforms the vanilla linear regression model by 22.37% in RMSE and the accuracy by 18%. The R-squared value of the neural model is more than 0.72 and the accuracy reaches 78%. Hence, the body shape features can provide an additional accurate and affordable choice to monitor the degree of the patient's liver fat. Clinical relevance - This paper presents a low cost and convenient approach to predict liver fat percentage using body shapes. This approach will not replace the gold standard for assessing hepatic steatosis. However, with the wide availability for depth cameras (including on smartphones), the approach promises to provide another modality that can be deployed widely in clinical setting as well for home use for telehealth.

Automated Assessment System for Neonatal Endotracheal Intubation Using Dilated Convolutional Neural Network.

Neonatal endotracheal intubation (ETI) is an important, complex resuscitation skill, which requires a significant amount of practice to master. Current ETI practice is conducted on the physical manikin and relies on the expert instructors' assessment. Since the training opportunities are limited by the availability of expert instructors, an automatic assessment model is highly desirable. However, automating ETI assessment is challenging due to the complexity of identifying crucial features, providing accurate evaluations and offering valuable feedback to trainees. In this paper, we propose a dilated Convolutional Neural Network (CNN) based ETI assessment model, which can automatically provide an overall score and performance feedback to pediatric trainees. The proposed assessment model takes the captured kinematic multivariate time-series (MTS) data from the manikin-based augmented ETI system that we developed, automatically extracts the crucial features of captured data, and eventually provides an overall score as output. Furthermore, the visualization based on the class activation mapping (CAM) can automatically identify the motions that have significant impact on the overall score, thus providing useful feedback to trainees. Our model can achieve 92.2% average classification accuracy using the Leave-One-Out-Cross-Validation (LOOCV).

Robust Template-Based Non-Rigid Motion Tracking Using Local Coordinate Regularization.

In this paper, we propose our template-based non-rigid registration algorithm to address the misalignments in the frame-to-frame motion tracking with single or multiple commodity depth cameras. We analyze the deformation in the local coordinates of neighboring nodes and use this differential representation to formulate the regularization term for the deformation field in our non-rigid registration. The local coordinate regularizations vary for each pair of neighboring nodes based on the tracking status of the surface regions. We propose our tracking strategies for different surface regions to minimize misalignments and reduce error accumulation. This method can thus preserve local geometric features and prevent undesirable distortions. Moreover, we introduce a geodesic-based correspondence estimation algorithm to align surfaces with large displacements. Finally, we demonstrate the effectiveness of our proposed method with detailed experiments.

3D Shape-Based Body Composition Inference Model Using a Bayesian Network.

Body composition can be assessed in many different ways. High-end medical equipment, such as Dual-energy X-ray Absorptiometry (DXA), Computed Tomography (CT), and Magnetic Resonance Imaging (MRI) offers high-fidelity pixel/voxel-level assessment, but is prohibitive in cost. In the case of DXA and CT, the approach exposes users to ionizing radiation. Whole-body air displacement plethysmography (BOD POD) can accurately estimate body density, but the assessment is limited to the whole-body fat percentage. Optical three-dimensional (3D) scan and reconstruction techniques, such as using depth cameras, have brought new opportunities for improving body composition assessment by intelligently analyzing body shape features. In this paper, we present a novel supervised inference model to predict pixel-level body composition and percentage of body fat using 3D geometry features and body density. First, we use body density to model a fat distribution base prediction. Then, we use a Bayesian network to infer the probability of the base prediction bias with 3D geometry features. Finally, we correct the bias using non-parametric regression. We use DXA assessment as the ground truth in model training and validation. We compare our method, in terms of pixel-level body composition assessment, with the current state-of-the-art prediction models. Our method outperforms those prediction models by 52.69% on average. We also compare our method, in terms of whole-body fat percentage assessment, with the medical-level equipment-BOD POD. Our method outperforms the BOD POD by 23.28%.

Shape-based Three-dimensional Body Composition Extrapolation Using Multimodality Registration.

The ubiquity of commodity-level optical scan devices and reconstruction technologies has enabled the general public to monitor their body shape related health status anywhere, anytime, without assistance from professionals. Commercial optical body scan systems extract anthropometries from the virtual body shapes, from which body compositions are estimated. However, in most cases, these estimations are limited to the quantity of fat in the whole body instead of a fine-granularity voxel-level fat distribution estimation. To bridge the gap between the 3D body shape and fine-granularity voxel-level fat distribution, we present an innovative shape-based voxel-level body composition extrapolation method using multimodality registration. First, we optimize shape compliance between a generic body composition template and the 3D body shape. Then, we optimize data compliance between the shape-optimized body composition template and a body composition reference from the DXA pixel-level body composition assessment. We evaluate the performance of our method with different subjects. On average, the Root Mean Square Error (RMSE) of our body composition extrapolation is 1.19%, and the R-squared value between our estimation and the ground truth is 0.985. The experimental result shows that our algorithm can robustly estimate voxel-level body composition for 3D body shapes with a high degree of accuracy.

ReTouchImg: Fusioning from-local-to-global context detection and graph data structures for fully-automatic specular reflection removal for endoscopic images.

Minimally invasive surgical and diagnostic systems are commonly used in clinical practices. However, the accuracy and robustness of these systems depend heavily on computer based processes such as tracking, detecting or segmenting clinically meaningful regions of interest, which are significantly affected by the inherent specular reflections that appear on the organs' surfaces. Restoration of the acquired data for clinical purposes still presents challenges because of the high texture and color variations across the image. In this work, we propose a novel fully-automated solution for endoscopic image restoration, which we call ReTouchImg. Our approach is designed as a two-step scheme. The first is a detection step that is based on the synergy of a set of color variations and gradient information conditions. For the second step, we introduce an inpainting process which is based on graph data structures for recovering the missing information. We exhaustively evaluate our approach on real endoscopic datasets and compare it against some works from the body of literature. We also demonstrate that our solution deals with complex cases such as strong illumination variation and large affected areas through a careful quantitative evaluation of a range of numerical results.

A Novel Hybrid Model for Visceral Adipose Tissue Prediction using Shape Descriptors.

Obesity is gaining increasing attention in modern society since it is associated with various health issues. The visceral adipose tissue (VAT) deposits around the abdominal organs and is considered an extremely important indicator of health risk. VAT can be assessed through magnetic resonance imaging (MRI) or computed tomography (CT) accurately, but the cost is prohibitive. Shape-based body composition prediction has become a promising topic thanks to the prevalence of commodity optical body scan systems, from which numerous anthropometries can be extracted automatically. In this paper, we propose an innovative shape-based hybrid VAT prediction model. The most appealing benefit of our method is to robustly handle the lack of knowledge about gender and demographics. First, we train a baseline VAT prediction model for each gender separately. Second, we train a classifier to predict the gender likelihood and a classifier to predict the shape likelihood of being overestimated in VAT baseline prediction. Third, we integrate the gender likelihood and shape likelihood into the baseline models to derive one hybrid VAT prediction model. We compare our prediction model with other state-of-the-art VAT prediction methods. The result shows that our method outperforms the comparison methods by 21.8% on average.

3D Shape-based Body Composition Prediction Model Using Machine Learning.

A booming development of 3D body scan and modeling technologies has facilitated large-scale anthropometric data collections for biomedical research and applications. However, usages of the digitalized human body shape data are relatively limited due to a lack of corresponding medical data to establish correlations between body shapes and underlying health information, such as the Body Fat Percentage (BFP). We present a novel prediction model to estimate the BFP by analyzing 3D body shapes. We introduce the concept of "visual cue" by analyzing the second-order shape descriptors. We first establish our baseline regression model for feature selection of the zeroth-order shape descriptors. Then, we use the visual cue as a shape-prior to improve the baseline prediction. In our study, we take the Dual-energy X-ray Absorptiometry (DXA) BFP measure as the ground truth for model training and evaluation. DXA is considered the "gold standard" in body composition assessment. We compare our results with the clinical BFP estimation instrument-the BOD POD. The result shows that our prediction model, on the average, outperforms the BOD POD by 20.28% in prediction accuracy.

Evaluation of performance, acceptance, and compliance of an auto-injector in healthy and rheumatoid arthritic subjects measured by a motion capture system.

PURPOSE: The study aimed to develop a motion capture system that can track, visualize, and analyze the entire performance of self-injection with the auto-injector. METHODS: Each of nine healthy subjects and 29 rheumatoid arthritic (RA) patients with different degrees of hand disability performed two simulated injections into an injection pad while six degrees of freedom (DOF) motions of the auto-injector and the injection pad were captured. We quantitatively measured the performance of the injection by calculating needle displacement from the motion trajectories. The max, mean, and SD of needle displacement were analyzed. Assessments of device acceptance and usability were evaluated by a survey questionnaire and independent observations of compliance with the device instruction for use (IFU). RESULTS: A total of 80 simulated injections were performed. Our results showed a similar level of performance among all the subjects with slightly larger, but not statistically significant, needle displacement in the RA group. In particular, no significant effects regarding previous experience in self-injection, grip method, pain in hand, and Cochin score in the RA group were found to have an impact on the mean needle displacement. Moreover, the analysis of needle displacement for different durations of injections indicated that most of the subjects reached their personal maximum displacement in 15 seconds and remained steady or exhibited a small amount of increase from 15 to 60 seconds. Device acceptance was high for most of the questions (ie, >4; >80%) based on a 0-5-point scale or percentage of acceptance. The overall compliance with the device IFU was high for the first injection (96.05%) and reached 98.02% for the second injection. CONCLUSION: We demonstrated the feasibility of tracking the motions of injection to measure the performance of simulated self-injection. The comparisons of needle displacement showed that even RA patients with severe hand disability could properly perform self-injection with this auto-injector at a similar level with the healthy subjects. Finally, the observed high device acceptance and compliance with device IFU suggest that the system is convenient and easy to use.

Accurate nonrigid 3D human body surface reconstruction using commodity depth sensors.

In the last decade, 3D modeling techniques enjoyed a booming development in both hardware and software. High-end hardware generates high fidelity results, but the cost is prohibitive, whereas consumer-level devices generate plausible results for entertainment purposes but are not appropriate for medical uses. We present a cost-effective and easy-to-use 3D body reconstruction system using consumer-grade depth sensors, which provides reconstructed body shapes with a high degree of accuracy and reliability appropriate for medical applications. Our surface registration framework integrates the articulated motion assumption, global loop closure constraint, and a general as-rigid-as-possible deformation model. To enhance the reconstruction quality, we propose a novel approach to accurately infer skeletal joints from anatomical data using multimodality registration. We further propose a supervised predictive model to infer the skeletal joints for arbitrary subjects independent from anatomical data reference. A rigorous validation test has been conducted on real subjects to evaluate the reconstruction accuracy and repeatability. Our system has the potential to make accurate body surface scanning systems readily available for medical professionals and the general public. The system can be used to obtain additional health data derived from 3D body shapes, such as the percentage of body fat.

Towards Retrieving Force Feedback in Robotic-Assisted Surgery: A Supervised Neuro-Recurrent-Vision Approach.

Robotic-assisted minimally invasive surgeries have gained a lot of popularity over conventional procedures as they offer many benefits to both surgeons and patients. Nonetheless, they still suffer from some limitations that affect their outcome. One of them is the lack of force feedback which restricts the surgeon's sense of touch and might reduce precision during a procedure. To overcome this limitation, we propose a novel force estimation approach that combines a vision based solution with supervised learning to estimate the applied force and provide the surgeon with a suitable representation of it. The proposed solution starts with extracting the geometry of motion of the heart's surface by minimizing an energy functional to recover its 3D deformable structure. A deep network, based on a LSTM-RNN architecture, is then used to learn the relationship between the extracted visual-geometric information and the applied force, and to find accurate mapping between the two. Our proposed force estimation solution avoids the drawbacks usually associated with force sensing devices, such as biocompatibility and integration issues. We evaluate our approach on phantom and realistic tissues in which we report an average root-mean square error of 0.02 N.

Adaptive segmentation and mask-specific Sobolev inpainting of specular highlights for endoscopic images.

Minimally invasive surgical and diagnostic systems rely on endoscopic images of internal organs to assist medical tasks. Specular highlights are common on those images due to the strong reflectivity of the mucus layer on the organs and the relatively high intensity of the light source. This is a significant source of error that can affect the systems' performance. In this paper, we propose a segmentation method of the specular regions based on an automatic color-adaptive threshold and a gradient-based edge detector. The segmented regions are then recovered using a robust mask-specific Sobolev inpainting approach. Experimental results demonstrate the precision and efficiency of the proposed method. In contrast to the existing approaches, the proposed solution does not require manual threshold selection or complex computations to achieve accurate results. Moreover, our method has a real-time performance and can be generalized to various applications.

Automatic and robust single-camera specular highlight removal in cardiac images.

In computer-assisted beating heart surgeries, accurate tracking of the heart's motion is of huge importance and there is a continuous need to eliminate any source of error that might disturb the tracking process. One source of error is the specular reflection that appears on the glossy surface of the heart. In this paper, we propose a robust solution for the detection and removal of specular highlights. A hybrid color attributes and wavelet based edge projection approach is applied to accurately identify the affected regions. These regions are then recovered using a dynamic search-based inpainting with adaptive windowing. Experimental results demonstrate the precision and efficiency of the proposed method. Moreover, it has a real-time performance and can be generalized to various other applications.

Computer-based Planning System for Mandibular Reconstruction.

Mandibular reconstruction is typically performed for traumatic or postsurgical conditions, and may involve the use of autologous osteocutaneous fibula free flaps for large defects. Recreating the native contour of the mandible during reconstructive surgery is challenging. Existed pre-operative planning software has limitations. In this paper we present a novel pre-operative planning system that helps to optimize the number and location of bony osteotomies, and the orientation of the harvested bone segments by specifically prioritizing the position and number of the cutanteous perforators to the osteocutaneous fibula free flap, in order to improve the reliability of the osteocutaneous fibula free flap.

Image guided medialization laryngoplasty.

Techniques that originate in computer graphics and computer vision have found prominent applications in the medical domain. In this paper, we have seamlessly developed techniques from computer graphics and computer vision together with domain knowledge from medicine to develop an image guided surgical system for medialization laryngoplasty. The technical focus of this paper is to register the preoperative radiological data to the intraoperative anatomical structure of the patient. With careful analysis of the real-world surgical environment, we have developed an ICP-based partial shape matching algorithm to register the partially visible anatomical structure to the preoperative CT data. We extracted distinguishable features from the human thyroid cartilage surface and applied image space template matching to find the initial guess for the shape matching. The experimental result shows that our feature-based partial shape matching method has better performance and robustness compared with original ICP-based shape matching method. Although this paper concentrates on the medialization laryngoplasty procedure, its generality makes our methods ideal for future applications in other image guided surgical areas.

An immersed-boundary method for flow-structure interaction in biological systems with application to phonation.

A new numerical approach for modeling a class of flow-structure interaction problems typically encountered in biological systems is presented. In this approach, a previously developed, sharp-interface, immersed-boundary method for incompressible flows is used to model the fluid flow and a new, sharp-interface Cartesian grid, immersed boundary method is devised to solve the equations of linear viscoelasticity that governs the solid. The two solvers are coupled to model flow-structure interaction. This coupled solver has the advantage of simple grid generation and efficient computation on simple, single-block structured grids. The accuracy of the solid-mechanics solver is examined by applying it to a canonical problem. The solution methodology is then applied to the problem of laryngeal aerodynamics and vocal fold vibration during human phonation. This includes a three-dimensional eigen analysis for a multi-layered vocal fold prototype as well as two-dimensional, flow-induced vocal fold vibration in a modeled larynx. Several salient features of the aerodynamics as well as vocal-fold dynamics are presented.

Cryotherapy simulator for localized prostate cancer.

Cryotherapy is a treatment modality that uses a technique to selectively freeze tissue and thereby cause controlled tissue destruction. The procedure involves placement of multiple small diameter probes through the perineum into the prostate tissue at selected spatial intervals. Transrectal ultrasound is used to properly position the cylindrical probes before activation of the liquid Argon cooling element, which lowers the tissue temperature below -40 degrees Centigrade. Tissue effect is monitored by transrectal ultrasound changes as well as thermocouples placed in the tissue. The computer-based cryotherapy simulation system mimics the major surgical steps involved in the procedure. The simulated real-time ultrasound display is generated from 3-D ultrasound datasets where the interaction of the ultrasound with the instruments as well as the frozen tissue is simulated by image processing. The thermal and mechanical simulations of the tissue are done using a modified finite-difference/finite-element method optimized for real-time performance. The simulator developed is a part of a comprehensive training program, including a computer-based learning system and hands-on training program with a proctor, designed to familiarize the physician with the technique and equipment involved.