3D Convolutional Deep Learning for Nonlinear Estimation of Body Composition from Whole-Body Morphology.

Total and regional body composition are strongly correlated with metabolic syndrome and have been estimated non-invasively from 3D optical scans using linear parameterizations of body shape and linear regression models. Prior works produced accurate and precise predictions on many, but not all, body composition targets relative to the reference dual X-Ray absorptiometry (DXA) measurement. Here, we report the effects of replacing linear models with nonlinear parameterization and regression models on the precision and accuracy of body composition estimation in a novel application of deep 3D convolutional graph networks to human body composition modeling. We assembled an ensemble dataset of 4286 topologically standardized 3D optical scans from four different human body shape databases, DFAUST, CAESAR, Shape Up! Adults, and Shape Up! Kids and trained a parameterized shape model using a graph convolutional 3D autoencoder (3DAE) in lieu of linear PCA. We trained a nonlinear Gaussian process regression (GPR) on the 3DAE parameter space to predict body composition via correlations to paired DXA reference measurements from the Shape Up! scan subset. We tested our model on a set of 424 randomly withheld test meshes and compared the effects of nonlinear computation against prior linear models. Nonlinear GPR produced up to 20% reduction in prediction error and up to 30% increase in precision over linear regression for both sexes in 10 tested body composition variables. Deep shape features produced 6-8% reduction in prediction error over linear PCA features for males only and a 4-14% reduction in precision error for both sexes. Our best performing nonlinear model predicting body composition from deep features outperformed prior work using linear methods on all tested body composition prediction metrics in both precision and accuracy. All coefficients of determination (R2) for all predicted variables were above 0.86. We show that GPR is a more precise and accurate method for modeling body composition mappings from body shape features than linear regression. Deep 3D features learned by a graph convolutional autoencoder only improved male body composition accuracy but improved precision in both sexes. Our work achieved lower estimation RMSEs than all previous work on 10 metrics of body composition.

Automated body composition estimation from device-agnostic 3D optical scans in pediatric populations.

BACKGROUND: Excess adiposity in children is strongly correlated with obesity-related metabolic disease in adulthood, including diabetes, cardiovascular disease, and 13 types of cancer. Despite the many long-term health risks of childhood obesity, body mass index (BMI) Z-score is typically the only adiposity marker used in pediatric studies and clinical applications. The effects of regional adiposity are not captured in a single scalar measurement, and their effects on short- and long-term metabolic health are largely unknown. However, clinicians and researchers rarely deploy gold-standard methods for measuring compartmental fat such as magnetic resonance imaging (MRI) and dual X-ray absorptiometry (DXA) on children and adolescents due to cost or radiation concerns. Three-dimensional optical (3DO) scans are relatively inexpensive to obtain and use non-invasive and radiation-free imaging techniques to capture the external surface geometry of a patient's body. This 3D shape contains cues about the body composition that can be learned from a structured correlation between 3D body shape parameters and reference DXA scans obtained on a sample population. STUDY AIM: This study seeks to introduce a radiation-free, automated 3D optical imaging solution for monitoring body shape and composition in children aged 5-17. METHODS: We introduce an automated, linear learning method to predict total and regional body composition of children aged 5-17 from 3DO scans. We collected 145 male and 206 female 3DO scans on children between the ages of 5 and 17 with three scanners from independent manufacturers. We used an automated shape templating method first introduced on an adult population to fit a topologically consistent 60,000 vertex (60 k) mesh to 3DO scans of arbitrary scanning source and mesh topology. We constructed a parameterized body shape space using principal component analysis (PCA) and estimated a regression matrix between the shape parameters and their associated DXA measurements. We automatically fit scans of 30 male and 38 female participants from a held-out test set and predicted 12 body composition measurements. RESULTS: The coefficient of determination (R2) between 3DO predicted body composition and DXA measurements was at least 0.85 for all measurements with the exception of visceral fat on 3D scan predictions. Precision error was 1-4 times larger than that of DXA. No predicted variable was significantly different from DXA measurement except for male trunk lean mass. CONCLUSION: Optical imaging can quickly, safely, and inexpensively estimate regional body composition in children aged 5-17. Frequent repeat measurements can be taken to chart changes in body adiposity over time without risk of radiation overexposure.

A device-agnostic shape model for automated body composition estimates from 3D optical scans.

BACKGROUND: Many predictors of morbidity caused by metabolic disease are associated with body shape. 3D optical (3DO) scanning captures body shape and has been shown to accurately and precisely predict body composition variables associated with mortality risk. 3DO is safer, less expensive, and more accessible than criterion body composition assessment methods such as dual-energy X-ray absorptiometry (DXA). However, 3DO scanning has not been standardized across manufacturers for pose, mesh resolution, and post processing methods. PURPOSE: We introduce a scanner-agnostic algorithm that automatically fits a topologically consistent human mesh to 3DO scanned point clouds and predicts clinically important body metrics using a standardized body shape model. Our models transform raw scans captured by any 3DO scanner into fixed topology meshes with anatomical consistency, standardizing the outputs of 3DO scans across manufacturers and allowing for the use of common prediction models across scanning devices. METHODS: A fixed-topology body mesh template was automatically registered to 848 training scans from three different 3DO systems. Participants were between 18 and 89 years old with body mass index ranging from 14 to 52 kg/m2 . Scans were registered by first performing a coarse nearest neighbor alignment between the template and the input scan with an anatomically constrained principal component analysis (PCA) domain deformation using a device and gender specific bootstrap basis trained on 70 seed scans each. The template mesh was then optimized to fit the target with a smooth per-vertex surface-to-surface deformation. A combined unified PCA model was created from the superset of all automatically fit training scans including all three devices. Body composition predictions to DXA measurements were learned from the training mesh PCA coefficients using linear regression. Using this final unified model, we tested the accuracy of our body composition models on a withheld sample of 562 scans by fitting a PCA parameterized template mesh to each raw scan and predicting the expected body composition metrics from the principal components using the learned regression model. RESULTS: We achieved coefficients of determination (R2 ) above 0.8 on all nine fat and lean predictions except female visceral fat (0.77). R2 was as high as 0.94 (total fat and lean, trunk fat), and all root-mean-squared errors were below 3.0 kg. All predicted body composition variables were not significantly different from reference DXA measurements except for visceral fat and female trunk fat. Repeatability precision as measured by the coefficient of variation (%CV) was around 2-3x worse than DXA precision, with visceral fat %CV below 2x DXA %CV and female total fat mass at 5x. CONCLUSIONS: Our method provides an accurate, automated, and scanner agnostic framework for standardizing 3DO scans and a low cost, radiation-free alternative to criterion radiology imaging for body composition analysis. We published a web-app version of this work at https://shapeup.shepherdresearchlab.org/3do-bodycomp-analyzer/ that accepts mesh file uploads and returns templated meshes with body composition predictions for demo purposes.

A pose-independent method for accurate and precise body composition from 3D optical scans.

OBJECTIVE: The aim of this study was to investigate whether digitally re-posing three-dimensional optical (3DO) whole-body scans to a standardized pose would improve body composition accuracy and precision regardless of the initial pose. METHODS: Healthy adults (n = 540), stratified by sex, BMI, and age, completed whole-body 3DO and dual-energy X-ray absorptiometry (DXA) scans in the Shape Up! Adults study. The 3DO mesh vertices were represented with standardized templates and a low-dimensional space by principal component analysis (stratified by sex). The total sample was split into a training (80%) and test (20%) set for both males and females. Stepwise linear regression was used to build prediction models for body composition and anthropometry outputs using 3DO principal components (PCs). RESULTS: The analysis included 472 participants after exclusions. After re-posing, three PCs described 95% of the shape variance in the male and female training sets. 3DO body composition accuracy compared with DXA was as follows: fat mass R2 = 0.91 male, 0.94 female; fat-free mass R2 = 0.95 male, 0.92 female; visceral fat mass R2 = 0.77 male, 0.79 female. CONCLUSIONS: Re-posed 3DO body shape PCs produced more accurate and precise body composition models that may be used in clinical or nonclinical settings when DXA is unavailable or when frequent ionizing radiation exposure is unwanted.

Predicting 3D body shape and body composition from conventional 2D photography.

PURPOSE: Total and regional body composition are important indicators of health and mortality risk, but their measurement is usually restricted to controlled environments in clinical settings with expensive and specialized equipment. A method that approaches the accuracy of the current gold standard method, dual-energy x-ray absorptiometry (DXA), while only requiring input from widely available consumer grade equipment, would enable the measurement of these important biometrics in the wild, enabling data collection at a scale that would have previously been prohibitive in time and expense. We describe an algorithm for predicting three-dimensional (3D) body shape and composition from a single frontal 2-dimensional image acquired with a digital consumer camera. METHODS: Duplicate 3D optical scans, two-dimensional (2D) optical images, and DXA whole-body scans were available for 183 men and 233 women from the Shape Up! Adults Study. A principal component analysis vector basis was fit to 3D point clouds of a training subset of 152 men and 194 women. The relationship between this vector space and DXA-derived body composition was modeled with linear regression. The principal component 3D shape was then fitted to match a silhouette extracted from a 2D photograph of a novel body. Body composition was predicted from the resulting 3D shape match using the linear mapping between the principal component parameters and the DXA metrics. Accuracy of body composition estimates from the silhouette method was evaluated against a simple model using height and weight as a baseline, and against DXA measurements as ground truth. Test-retest precision of the silhouette method was evaluated using the duplicate 2D optical images and compared against precision of the duplicate DXA scans. Paired t-tests were performed to detect significant differences between the sets. RESULTS: Results were reported on a held-out set. Body composition prediction achieved R2 s of 0.81 and 0.74 for percent fat prediction of males and females, respectively, on a held-out test set consisting of 31 males and 39 females. Precision estimates for fat mass were 2.31% and 2.06% for males and females, respectively, compared to 1.26% and 0.68% for DXA scans. The t-tests revealed no statistically significant differences between the silhouette method measurements and DXA measurements, or between retests. CONCLUSION: Total and regional body composition measures can be estimated from a single frontal photograph of a human body. Body composition prediction using consumer level photography can enable early screening and monitoring of possible physiological indicators of metabolic disease in regions where medical imagery or clinical assessment is inaccessible.

Detailed 3-dimensional body shape features predict body composition, blood metabolites, and functional strength: the Shape Up! studies.

BACKGROUND: Three-dimensional optical (3DO) body scanning has been proposed for automatic anthropometry. However, conventional measurements fail to capture detailed body shape. More sophisticated shape features could better indicate health status. OBJECTIVES: The objectives were to predict DXA total and regional body composition, serum lipid and diabetes markers, and functional strength from 3DO body scans using statistical shape modeling. METHODS: Healthy adults underwent whole-body 3DO and DXA scans, blood tests, and strength assessments in the Shape Up! Adults cross-sectional observational study. Principal component analysis was performed on registered 3DO scans. Stepwise linear regressions were performed to estimate body composition, serum biomarkers, and strength using 3DO principal components (PCs). 3DO model accuracy was compared with simple anthropometric models and precision was compared with DXA. RESULTS: This analysis included 407 subjects. Eleven PCs for each sex captured 95% of body shape variance. 3DO body composition accuracy to DXA was: fat mass R2 = 0.88 male, 0.93 female; visceral fat mass R2 = 0.67 male, 0.75 female. 3DO body fat test-retest precision was: root mean squared error = 0.81 kg male, 0.66 kg female. 3DO visceral fat was as precise (%CV = 7.4 for males, 6.8 for females) as DXA (%CV = 6.8 for males, 7.4 for females). Multiple 3DO PCs were significantly correlated with serum HDL cholesterol, triglycerides, glucose, insulin, and HOMA-IR, independent of simple anthropometrics. 3DO PCs improved prediction of isometric knee strength (combined model R2 = 0.67 male, 0.59 female; anthropometrics-only model R2 = 0.34 male, 0.24 female). CONCLUSIONS: 3DO body shape PCs predict body composition with good accuracy and precision comparable to existing methods. 3DO PCs improve prediction of serum lipid and diabetes markers, and functional strength measurements. The safety and accessibility of 3DO scanning make it appropriate for monitoring individual body composition, and metabolic health and functional strength in epidemiological settings.This trial was registered at clinicaltrials.gov as NCT03637855.

Image-Based Remodeling.

Imagining what a proposed home remodel might look like without actually performing it is challenging. We present an image-based remodeling methodology that allows real-time photorealistic visualization during both the modeling and remodeling process of a home interior. Large-scale edits, like removing a wall or enlarging a window, are performed easily and in real time, with realistic results. Our interface supports the creation of concise, parameterized, and constrained geometry, as well as remodeling directly from within the photographs. Real-time texturing of modified geometry is made possible by precomputing view-dependent textures for all faces that are potentially visible to each original camera viewpoint, blending multiple viewpoints and hole-filling when necessary. The resulting textures are stored and accessed efficiently enabling intuitive real-time realistic visualization, modeling, and editing of the building interior.

Vacuum-assisted socket suspension compared with pin suspension for lower extremity amputees: effect on fit, activity, and limb volume.

OBJECTIVE: To investigate the effect of a vacuum-assisted socket suspension system as compared with pin suspension on lower extremity amputees. DESIGN: Randomized crossover with 3-week acclimation. SETTING: Household, community, and laboratory environments. PARTICIPANTS: Unilateral, transtibial amputees (N=20 enrolled, N=5 completed). INTERVENTIONS: (1) Total surface-bearing socket with a vacuum-assisted suspension system (VASS), and (2) modified patellar tendon-bearing socket with a pin lock suspension system. MAIN OUTCOME MEASURES: Activity level, residual limb volume before and after a 30-minute treadmill walk, residual limb pistoning, and Prosthesis Evaluation Questionnaire. RESULTS: Activity levels were significantly lower while wearing the vacuum-assisted socket suspension system than the pin suspension (P=.0056; 38,000 ± 9,000 steps per 2 wk vs 73,000 ± 18,000 steps per 2 wk, respectively). Residual limb pistoning was significantly less while wearing the vacuum-assisted socket suspension system than the pin suspension (P=.0021; 1 ± 3mm vs 6 ± 4mm, respectively). Treadmill walking had no effect on residual limb volume. In general, participants ranked their residual limb health higher, were less frustrated, and claimed it was easier to ambulate while wearing a pin suspension compared with the VASS. CONCLUSIONS: The VASS resulted in a better fitting socket as measured by limb movement relative to the prosthetic socket (pistoning), although the clinical relevance of the small but statistically significant difference is difficult to discern. Treadmill walking had no effect, suggesting that a skilled prosthetist can control for daily limb volume fluctuations by using conventional, nonvacuum systems. Participants took approximately half as many steps while wearing the VASS which, when coupled with their subjective responses, suggests a preference for the pin suspension system.

Staphylococcus aureus recovery from environmental and human locations in 2 collegiate athletic teams.

CONTEXT: Staphylococcus aureus is spread via direct contact with persons and indirect contact via environmental surfaces such as weight benches. Athletes participating in direct-contact sports have an increased risk of acquiring S aureus infections. OBJECTIVE: To determine (1) potential environmental reservoirs of S aureus in football and wrestling locker rooms and weight rooms, (2) environmental bacterial status after employing more stringent cleaning methods, (3) differences in colonization rates between athletes and nonathletes, (4) exposed body locations where Staphylococcus was recovered more frequently, and (5) personal hygiene practices of athletes and nonathletes. DESIGN: Cross-sectional study. SETTING: Locker room and strengthening and conditioning facilities at a National Collegiate Athletic Association Division II university. PATIENTS OR OTHER PARTICIPANTS: Collegiate football players and wrestlers, with nonathlete campus residents serving as the control group. INTERVENTION(S): Infection control methods, education of the custodial staff, and education of the athletes regarding the Centers for Disease Control and Prevention guidelines for infection prevention. MAIN OUTCOME MEASURE(S): Cultures were taken from the participants' noses, fingertips, knuckles, forearms, and shoes and from the environment. RESULTS: Before the intervention, from the 108 environmental samples taken from the football locker room and weight room, 26 (24%) contained methicillin-susceptible S aureus (MSSA) and 33 (31%) contained methicillin-resistant S aureus (MRSA). From the 39 environmental samples taken from the wrestling locker room and pit areas, 1 (3%) contained MSSA and 4 (10%) contained MRSA. The MRSA rates were different between the 2 locations according to a chi(2) test (P = .01). Seven MRSA isolates were recovered from football players and 1 from a wrestler; no MRSA isolates were recovered from the control group. The fingertip location of S aureus recovery from football players was significant when compared with both other locations in football players and fingertips in wrestlers and the control group (P < .05). Football players and wrestlers shared more personal items than the control group (P < .05). After the intervention, the football locker room and weight room samples were negative for S aureus. CONCLUSIONS: Intact strengthening and conditioning equipment, proper hygiene, and proper disinfection methods lowered both environmental and human S aureus recovery at 1 university.

Shape and spatially-varying BRDFs from photometric stereo.

This paper describes a photometric stereo method designed for surfaces with spatially-varying BRDFs, including surfaces with both varying diffuse and specular properties. Our optimization-based method builds on the observation that most objects are composed of a small number of fundamental materials by constraining each pixel to be representable by a combination of at most two such materials. This approach recovers not only the shape but also material BRDFs and weight maps, yielding accurate rerenderings under novel lighting conditions for a wide variety of objects. We demonstrate examples of interactive editing operations made possible by our approach.

A theory of frequency domain invariants: spherical harmonic identities for BRDF/lighting transfer and image consistency.

This paper develops a theory of frequency domain invariants in computer vision. We derive novel identities using spherical harmonics, which are the angular frequency domain analog to common spatial domain invariants such as reflectance ratios. These invariants are derived from the spherical harmonic convolution framework for reflection from a curved surface. Our identities apply in a number of canonical cases, including single and multiple images of objects under the same and different lighting conditions. One important case we consider is two different glossy objects in two different lighting environments. For this case, we derive a novel identity, independent of the specific lighting configurations or BRDFs, that allows us to directly estimate the fourth image if the other three are available. The identity can also be used as an invariant to detecttampering in the images. While this paper is primarily theoretical, it has the potential to lay the mathematical foundations for two important practical applications. First, we can develop more general algorithms for inverse rendering problems, which can directly relight and change material properties by transferring the BRDF or lighting from another object or illumination. Second, we can check the consistency of an image, to detect tampering or image splicing.