Using Electric Stimulation of the Spinal Muscles and Electromyography during Motor Tasks for Evaluation of the Role in Development and Progression of Adolescent Idiopathic Scoliosis.

Introduction: The role of the spinal muscles in scoliogenesis is not fully substantiated. Do they act scoliogenic (inducing scoliosis) or counteract scoliosis in adolescent idiopathic scoliosis (AIS)? In this study, we will examine this by using selectively placed Transcutaneous Electric Stimulation (TES) combined with a cinematic radiographic technique and by performing electromyographic (EMG) evaluations during various motor tasks. Method: This is a cross-sectional study of subjects with small-curve AIS. Using cinematic radiography, they were evaluated dynamically either under electrical stimulation or when performing motor tasks of left and right lateral bending and rotation whilst measuring the muscle activity by EMG. Results: Forty-five patients with AIS were included as subjects. Five subjects volunteered for TES and six subjects performed the motor tasks with EMG. At the initial visual evaluation, and when stimulated with TES, the frontal plane spatial positions of the vertebral bodies showed discrete changes without an apparent pattern. However, analyzing the spatial positions when calibrated, we found that the spinal muscles exert a compressive 'response' with a minor change in the Cobb angle (CA) in small-curve AIS (CA = 10-20°). In larger curves (CA > 20°), TES induced a 'larger deformity' with a relative four-fold change in the CA compared to small-curve AIS with a ratio of 0.6. When evaluating local amplitude (peak) or cumulative (mean) EMG signals, we were unable to find consistent asymmetries. However, one subject had rapid progression and one regressed to a straight spine. When adding the absolute EMG ratios for all four motor tasks, the subject with progression had almost 10-fold less summed EMG ratios, and the subject with regression had more than 3-fold higher summed EMG ratios. Discussion: Based on these findings, we suggest that the spinal muscles in small-curve AIS have a stabilizing function maintaining a straight spine and keeping it in the midline. When deformities are larger (CA > 20°), the spine muscle curve exerts a scoliogenic 'response'. This suggests that the role of the muscles converts from counteracting AIS and stabilizing the spine to being scoliogenic for a CA of more than 20°. Moreover, we interpret higher EMG ratios as heightened asymmetric spinal muscle activity when the spinal muscles try to balance the spine to maintain or correct the deformity. When progression occurs, this is preceded or accompanied by decreased EMG ratios. These findings must be substantiated by larger studies.

Design of personalized scoliosis braces based on differentiable biomechanics-Synthetic study.

This work describes a computational methodology for the design of braces for adolescent idiopathic scoliosis. The proposed methodology relies on a personalized simulation model of the patient's trunk, and automatically searches for the brace geometry that optimizes the trade-off between clinical improvement and patient comfort. To do this, we introduce a formulation of differentiable biomechanics of the patient's trunk, the brace, and their interaction. We design a simulation model that is differentiable with respect to both the deformation state and the brace design parameters, and we show how this differentiable model is used for the efficient update of brace design parameters within a numerical optimization algorithm. To evaluate the proposed methodology, we have obtained trunk models with personalized geometry for five patients of adolescent idiopathic scoliosis, and we have designed Boston-type braces. In a simulation setting, the designed braces improve clinical metrics by 45% on average, under acceptable comfort conditions. In the future, the methodology can be extended beyond synthetic validation, and tested with physical braces on the actual patients.

Soft-tissue simulation of the breast for intraoperative navigation and fusion of preoperative planning.

Computational preoperative planning offers the opportunity to reduce surgery time and patient risk. However, on soft tissues such as the breast, deviations between the preoperative and intraoperative settings largely limit the applicability of preoperative planning. In this work, we propose a high-performance accurate simulation model of the breast, to fuse preoperative information with the intraoperative deformation setting. Our simulation method encompasses three major elements: high-quality finite-element modeling (FEM), efficient handling of anatomical couplings for high-performance computation, and personalized parameter estimation from surface scans. We show the applicability of our method on two problems: 1) transforming high-quality preoperative scans to the intraoperative setting for fusion of preoperative planning data, and 2) real-time tracking of breast tumors for navigation during intraoperative radiotherapy. We have validated our methodology on a test cohort of nine patients who underwent tumor resection surgery and intraoperative radiotherapy, and we have quantitatively compared simulation results to intraoperative scans. The accuracy of our simulation results suggest clinical viability of the proposed methodology.

Biomechanical Morphing for Personalized Fitting of Scoliotic Torso Skeleton Models.

The use of patient-specific biomechanical models offers many opportunities in the treatment of adolescent idiopathic scoliosis, such as the design of personalized braces. The first step in the development of these patient-specific models is to fit the geometry of the torso skeleton to the patient's anatomy. However, existing methods rely on high-quality imaging data. The exposure to radiation of these methods limits their applicability for regular monitoring of patients. We present a method to fit personalized models of the torso skeleton that takes as input biplanar low-dose radiographs. The method morphs a template to fit annotated points on visible portions of the spine, and it relies on a default biomechanical model of the torso for regularization and robust fitting of hardly visible parts of the torso skeleton, such as the rib cage. The proposed method provides an accurate and robust solution to obtain personalized models of the torso skeleton, which can be adopted as part of regular management of scoliosis patients. We have evaluated the method on ten young patients who participated in our study. We have analyzed and compared clinical metrics on the spine and the full torso skeleton, and we have found that the accuracy of the method is at least comparable to other methods that require more demanding imaging methods, while it offers superior robustness to artifacts such as interpenetration of ribs. Normal-dose X-rays were available for one of the patients, and for the other nine we acquired low-dose X-rays, allowing us to validate that the accuracy of the method persisted under less invasive imaging modalities.

Soft-Tissue Simulation for Computational Planning of Orthognathic Surgery.

Simulation technologies offer interesting opportunities for computer planning of orthognathic surgery. However, the methods used to date require tedious set up of simulation meshes based on patient imaging data, and they rely on complex simulation models that require long computations. In this work, we propose a modeling and simulation methodology that addresses model set up and runtime simulation in a holistic manner. We pay special attention to modeling the coupling of rigid-bone and soft-tissue components of the facial model, such that the resulting model is computationally simple yet accurate. The proposed simulation methodology has been evaluated on a cohort of 10 patients of orthognathic surgery, comparing quantitatively simulation results to post-operative scans. The results suggest that the proposed simulation methods admit the use of coarse simulation meshes, with planning computation times of less than 10 seconds in most cases, and with clinically viable accuracy.

A study of the sensitivity of biomechanical models of the spine for scoliosis brace design.

BACKGROUND AND OBJECTIVE: The development of biomechanical models of the torso and the spine opens the door to computational solutions for the design of braces for adolescent idiopathic scoliosis. However, the design of such biomechanical models faces several unknowns, such as the correct identification of relevant mechanical elements, or the required accuracy of model parameters. The objective of this study was to design a methodology for the identification of the aforementioned elements, with the purpose of creating personalized models suited for patient-specific brace design and the definition of parameter estimation criteria. METHODS: We have developed a comprehensive model of the torso, including spine, ribcage and soft tissue, and we have developed computational tools for the analysis of the model parameters. With these tools, we perform an analysis of the model under typical loading conditions of scoliosis braces. RESULTS: We present a complete sensitivity analysis of the models mechanical parameters and a comparison between a reference healthy subject and a subject suffering from scoliosis. Furthermore, we make a direct connection between error bounds on the deformation and tolerances for parameter estimation, which can guide the personalization of the model. CONCLUSIONS: Not surprisingly, the stiffness parameters that govern the lateral deformation of the spine in the frontal plane are some of the most relevant parameters, and require careful modeling. More surprisingly, their relevance is on par with the correct parameterization of the soft tissue of the torso. For scoliosis patients, but not for healthy subjects, we observe that the axial rotation of the spine also requires careful modeling.

Author Correction: Perceived match between own and observed models' bodies: influence of face, viewpoints, and body size.

An amendment to this paper has been published and can be accessed via a link at the top of the paper.

Perceived match between own and observed models' bodies: influence of face, viewpoints, and body size.

People are generally unable to accurately determine their own body measurements and to translate this knowledge to identifying a model/avatar that best represents their own body. This inability has not only been related to health problems (e.g. anorexia nervosa), but has important practical implications as well (e.g. online retail). Here we aimed to investigate the influence of three basic visual features-face presence, amount of viewpoints, and observed model size-on the perceived match between own and observed models' bodies and on attitudes towards these models. Models were real-life models (Experiment 1) or avatar models based on participants' own bodies (Experiment 2). Results in both experiments showed a strong effect of model size, irrespective of participants' own body measurements. When models were randomly presented one by one, participants gave significantly higher ratings to smaller- compared to bigger-sized models. The reverse was true, however, when participants observed and compared models freely, suggesting that the mode of presentation affected participants' judgments. Limited evidence was found for an effect of facial presence or amount of viewpoints. These results add evidence to research on visual features affecting the ability to match observed bodies with own body image, which has biological, clinical, and practical implications.

Rendering of Constraints With Underactuated Haptic Devices.

Several previous works have studied the application of proxy-based rendering algorithms to underactuated haptic devices. However, all these works make oversimplifying assumptions about the configuration of the haptic device, and they ignore the user's intent. In this article, we lift those assumptions, and we carry out a theoretical study that unveils the existence of unnatural ghost forces under typical proxy-based rendering. We characterize and quantify those ghost forces. In addition, we design a novel rendering strategy, with anisotropic coupling between the device and the proxy. With this strategy, the forces rendered by an underactuated device are a best match of the forces rendered by a fully actuated device. We have demonstrated our findings on synthetic experiments and a simple real-world experiment.

Path Routing Optimization for STM Ultrasound Rendering.

Ultrasound transducer arrays are capable of producing tactile sensations on the hand, promising hands-free haptic interaction for virtual environments. However, controlling such an array with respect to reproducing a desired perceived interaction remains a challenging problem. In this article, we approach this problem as a dynamic mapping of virtual interactions to existing control metaphors of ultrasound devices, namely, the modulation of focal point positions and intensities over time, a method known as Spatiotemporal Modulation (STM). In particular, we propose an optimization approach that takes into account known perceptual parameters and limitations of the STM method. This results in a set of focal point paths optimized to best reconstruct an arbitrary target pressure field.

MakeSense: Automated Sensor Design for Proprioceptive Soft Robots.

Soft robots have applications in safe human-robot interactions, manipulation of fragile objects, and locomotion in challenging and unstructured environments. In this article, we present a computational method for augmenting soft robots with proprioceptive sensing capabilities. Our method automatically computes a minimal stretch-receptive sensor network to user-provided soft robotic designs, which is optimized to perform well under a set of user-specified deformation-force pairs. The sensorized robots are able to reconstruct their full deformation state, under interaction forces. We cast our sensor design as a subselection problem, selecting a minimal set of sensors from a large set of fabricable ones, which minimizes the error when sensing specified deformation-force pairs. Unique to our approach is the use of an analytical gradient of our reconstruction performance measure with respect to selection variables. We demonstrate our technique on a bending bar and gripper example, illustrating more complex designs with a simulated tentacle.

Modeling Behavioral Experiment Interaction and Environmental Stimuli for a Synthetic C. elegans.

This paper focusses on the simulation of the neural network of the Caenorhabditis elegans living organism, and more specifically in the modeling of the stimuli applied within behavioral experiments and the stimuli that is generated in the interaction of the C. elegans with the environment. To the best of our knowledge, all efforts regarding stimuli modeling for the C. elegansare focused on a single type of stimulus, which is usually tested with a limited subnetwork of the C. elegansneural system. In this paper, we follow a different approach where we model a wide-range of different stimuli, with more flexible neural network configurations and simulations in mind. Moreover, we focus on the stimuli sensation by different types of sensory organs or various sensory principles of the neurons. As part of this work, most common stimuli involved in behavioral assays have been modeled. It includes models for mechanical, thermal, chemical, electrical and light stimuli, and for proprioception-related self-sensed information exchange with the neural network. The developed models have been implemented and tested with the hardware-based Si elegans simulation platform.

Optimization-Based Wearable Tactile Rendering.

Novel wearable tactile interfaces offer the possibility to simulate tactile interactions with virtual environments directly on our skin. But, unlike kinesthetic interfaces, for which haptic rendering is a well explored problem, they pose new questions about the formulation of the rendering problem. In this work, we propose a formulation of tactile rendering as an optimization problem, which is general for a large family of tactile interfaces. Based on an accurate simulation of contact between a finger model and the virtual environment, we pose tactile rendering as the optimization of the device configuration, such that the contact surface between the device and the actual finger matches as close as possible the contact surface in the virtual environment. We describe the optimization formulation in general terms, and we also demonstrate its implementation on a thimble-like wearable device. We validate the tactile rendering formulation by analyzing its force error, and we show that it outperforms other approaches.

Yarn-Level Cloth Simulation with Sliding Persistent Contacts.

Cloth is made of yarns that are stitched together forming semi-regular patterns. Due to the complexity of stitches and patterns, the macroscopic behavior of cloth is dictated by the contact interactions between yarns, not by the mechanical properties of yarns alone. The computation of cloth mechanics at the yarn level appears as a computationally complex and costly process at first sight, due to the need to resolve many fine-scale contact interactions. We propose instead an efficient representation of cloth at the yarn level that treats yarn-yarn contacts as persistent, but with the possibility to slide, thereby avoiding expensive contact handling altogether. We introduce a compact representation of yarn geometry and kinematics, capturing the essential deformation modes of yarn crossings, loops, stitches, and stacks, with a minimum cost. Based on this representation, we design force models that reproduce the characteristic macroscopic behavior of yarn-based fabrics. Our approach is suited for both woven and knitted fabrics. We demonstrate the efficiency of our method on simulations with millions of degrees of freedom (hundreds of thousands of yarn loops), almost one order of magnitude faster than previous techniques. We also compare the different macroscopic behavior under woven and knitted patterns with the same yarn density.

Aggregate Constraints for Virtual Manipulation with Soft Fingers.

Interactive dexterous manipulation of virtual objects remains a complex challenge that requires both appropriate hand models and accurate physically-based simulation of interactions. In this paper, we propose an approach based on novel aggregate constraints for simulating dexterous grasping using soft fingers. Our approach aims at improving the computation of contact mechanics when many contact points are involved, by aggregating the multiple contact constraints into a minimal set of constraints. We also introduce a method for non-uniform pressure distribution over the contact surface, to adapt the response when touching sharp edges. We use the Coulomb-Contensou friction model to efficiently simulate tangential and torsional friction. We show through different use cases that our aggregate constraint formulation is well-suited for simulating interactively dexterous manipulation of virtual objects through soft fingers, and efficiently reduces the computation time of constraint solving.

Volume Haptics with Topology-Consistent Isosurfaces.

Haptic interfaces offer an intuitive way to interact with and manipulate 3D datasets, and may simplify the interpretation of visual information. This work proposes an algorithm to provide haptic feedback directly from volumetric datasets, as an aid to regular visualization. The haptic rendering algorithm lets users perceive isosurfaces in volumetric datasets, and it relies on several design features that ensure a robust and efficient rendering. A marching tetrahedra approach enables the dynamic extraction of a piecewise linear continuous isosurface. Robustness is achieved using a continuous collision detection step coupled with state-of-the-art proxy-based rendering methods over the extracted isosurface. The introduced marching tetrahedra approach guarantees that the extracted isosurface will match the topology of an equivalent isosurface computed using trilinear interpolation. The proposed haptic rendering algorithm improves the consistency between haptic and visual cues computing a second proxy on the isosurface displayed on screen. Our experiments demonstrate the improvements on the isosurface extraction stage as well as the robustness and the efficiency of the complete algorithm.

Fast collision detection for fracturing rigid bodies.

In complex scenes with many objects, collision detection plays a key role in the simulation performance. This is particularly true in fracture simulation for two main reasons. One is that fracture fragments tend to exhibit very intensive contact, and the other is that collision detection data structures for new fragments need to be computed on the fly. In this paper, we present novel collision detection algorithms and data structures for real-time simulation of fracturing rigid bodies. We build on a combination of well-known efficient data structures, namely, distance fields and sphere trees, making our algorithm easy to integrate on existing simulation engines. We propose novel methods to construct these data structures, such that they can be efficiently updated upon fracture events and integrated in a simple yet effective self-adapting contact selection algorithm. Altogether, we drastically reduce the cost of both collision detection and collision response. We have evaluated our global solution for collision detection on challenging scenarios, achieving high frame rates suited for hard real-time applications such as video games or haptics. Our solution opens promising perspectives for complex fracture simulations involving many dynamically created rigid objects.

Animating Wrinkles by Example on Non-Skinned Cloth.

The simulation of cloth with rich folds and wrinkles is a computationally expensive process. In this paper, we introduce an example-based algorithm for fast animation of plausible cloth wrinkles. Our algorithm does not depend on a character's pose, therefore it is valid for loose dresses, curtains, etc., not just cloth defined by skinning techniques. Central to our approach is a correspondence between low and high-resolution cloth deformations, both at the training and synthesis stages. Based on this correspondence, we define an algorithm for synthesizing cloth wrinkles as a function of the deformation of a low-resolution cloth and a set of example poses. We demonstrate the animation of plausible high-resolution wrinkles at high frame rates, suitable for interactive applications such as video games.

Visuo-Haptic Mixed Reality with Unobstructed Tool-Hand Integration.

Visuo-haptic mixed reality consists of adding to a real scene the ability to see and touch virtual objects. It requires the use of see-through display technology for visually mixing real and virtual objects, and haptic devices for adding haptic interaction with the virtual objects. Unfortunately, the use of commodity haptic devices poses obstruction and misalignment issues that complicate the correct integration of a virtual tool and the user's real hand in the mixed reality scene. In this work, we propose a novel mixed reality paradigm where it is possible to touch and see virtual objects in combination with a real scene, using commodity haptic devices, and with a visually consistent integration of the user's hand and the virtual tool. We discuss the visual obstruction and misalignment issues introduced by commodity haptic devices, and then propose a solution that relies on four simple technical steps: color-based segmentation of the hand, tracking-based segmentation of the haptic device, background repainting using image-based models, and misalignment-free compositing of the user's hand. We have developed a successful proof-of-concept implementation, where a user can touch virtual objects and interact with them in the context of a real scene, and we have evaluated the impact on user performance of obstruction and misalignment correction.

Virtual reality software and technology.

This special issue contains an introductory survey on 3D user interfaces by leading VR authorities and significantly expanded versions of the four best short papers from the proceedings of the 14th ACM Symposium on Virtual Reality Software and Technology (VRST 07).