Identification of excessive contact pressures under hand orthosis based on finite element analysis.

BACKGROUND: Implicit magnitudes and distribution of excessive contact pressures under hand orthoses hinder clinicians from precisely adjusting them to relieve the pressures. To address this, contact pressure under a hand orthosis were analysed using finite element method. METHODS: This paper proposed a method to numerically predict the relatively high magnitudes and critical distribution of contact pressures under hand orthosis through finite element analysis, to identify excessive contact pressure locations. The finite element model was established consisting of the hand, orthosis and bones. The hand and bones were assumed to be homogeneous and elastic bodies, and the orthosis was considered as an isotropic and elastic shell. Two predictions were conducted by assigning either low (fat) or high (skin) material stiffness to the hand model to attain the range of pressure magnitudes. An experiment was conducted to measure contact pressures at the predicted pressure locations. RESULTS: Identical pressure distributions were obtained from both predictions with relatively high pressure values disseminated at 12 anatomical locations. The highest magnitude was found at the thumb metacarpophalangeal joint with the maximum pressure range from 13 to 78 KPa. The measured values were within the predicted range of pressure magnitudes. Moreover, 6 excessive contact pressure locations were identified. CONCLUSIONS: The proposed method was verified by the measurement results. It renders understanding of interface conditions underneath the orthosis to inform clinicians regarding orthosis design and adjustment. It could also guide the development of 3D printed or sensorised orthosis by indicating optimal locations for perforations or pressure sensors.

A hierarchical sensorimotor control framework for human-in-the-loop robotic hands.

Human manual dexterity relies critically on touch. Robotic and prosthetic hands are much less dexterous and make little use of the many tactile sensors available. We propose a framework modeled on the hierarchical sensorimotor controllers of the nervous system to link sensing to action in human-in-the-loop, haptically enabled, artificial hands.

Tapered whisker reservoir computing for real-time terrain identification-based navigation.

This paper proposes a new method for real-time terrain recognition-based navigation for mobile robots. Mobile robots performing tasks in unstructured environments need to adapt their trajectories in real-time to achieve safe and efficient navigation in complex terrains. However, current methods largely depend on visual and IMU (inertial measurement units) that demand high computational resources for real-time applications. In this paper, a real-time terrain identification-based navigation method is proposed using an on-board tapered whisker-based reservoir computing system. The nonlinear dynamic response of the tapered whisker was investigated in various analytical and Finite Element Analysis frameworks to demonstrate its reservoir computing capabilities. Numerical simulations and experiments were cross-checked with each other to verify that whisker sensors can separate different frequency signals directly in the time domain and demonstrate the computational superiority of the proposed system, and that different whisker axis locations and motion velocities provide variable dynamical response information. Terrain surface-following experiments demonstrated that our system could accurately identify changes in the terrain in real-time and adjust its trajectory to stay on specific terrain.

Robotic Simulators for Tissue Examination Training With Multimodal Sensory Feedback.

Tissue examination by hand remains an essential technique in clinical practice. The effective application depends on skills in sensorimotor coordination, mainly involving haptic, visual, and auditory feedback. The skills clinicians have to learn can be as subtle as regulating finger pressure with breathing, choosing palpation action, monitoring involuntary facial and vocal expressions in response to palpation, and using pain expressions both as a source of information and as a constraint on physical examination. Patient simulators can provide a safe learning platform to novice physicians before trying real patients. This paper reviews state-of-the-art medical simulators for the training for the first time with a consideration of providing multimodal feedback to learn as many manual examination techniques as possible. The study summarizes current advances in tissue examination training devices simulating different medical conditions and providing different types of feedback modalities. Opportunities with the development of pain expression, tissue modeling, actuation, and sensing are also analyzed to support the future design of effective tissue examination simulators.

A Surrogate Model Based on a Finite Element Model of Abdomen for Real-Time Visualisation of Tissue Stress during Physical Examination Training.

Robotic patients show great potential for helping to improve medical palpation training, as they can provide feedback that cannot be obtained in a real patient. They provide information about internal organ deformation that can significantly enhance palpation training by giving medical trainees visual insight based on the pressure they apply for palpation. This can be achieved by using computational models of abdomen mechanics. However, such models are computationally expensive, and thus unable to provide real-time predictions. In this work, we proposed an innovative surrogate model of abdomen mechanics by using machine learning (ML) and finite element (FE) modelling to virtually render internal tissue deformation in real time. We first developed a new high-fidelity FE model of the abdomen mechanics from computerized tomography (CT) images. We performed palpation simulations to produce a large database of stress distribution on the liver edge, an area of interest in most examinations. We then used artificial neural networks (ANNs) to develop the surrogate model and demonstrated its application in an experimental palpation platform. Our FE simulations took 1.5 h to predict stress distribution for each palpation while this only took a fraction of a second for the surrogate model. Our results show that our artificial neural network (ANN) surrogate has an accuracy of 92.6%. We also showed that the surrogate model is able to use the experimental input of palpation location and force to provide real-time projections onto the robotics platform. This enhanced robotics platform has the potential to be used as a training simulator for trainees to hone their palpation skills.

Face mediated human-robot interaction for remote medical examination.

Realtime visual feedback from consequences of actions is useful for future safety-critical human-robot interaction applications such as remote physical examination of patients. Given multiple formats to present visual feedback, using face as feedback for mediating human-robot interaction in remote examination remains understudied. Here we describe a face mediated human-robot interaction approach for remote palpation. It builds upon a robodoctor-robopatient platform where user can palpate on the robopatient to remotely control the robodoctor to diagnose a patient. A tactile sensor array mounted on the end effector of the robodoctor measures the haptic response of the patient under diagnosis and transfers it to the robopatient to render pain facial expressions in response to palpation forces. We compare this approach against a direct presentation of tactile sensor data in a visual tactile map. As feedback, the former has the advantage of recruiting advanced human capabilities to decode expressions on a human face whereas the later has the advantage of being able to present details such as intensity and spatial information of palpation. In a user study, we compare these two approaches in a teleoperated palpation task to find the hard nodule embedded in the remote abdominal phantom. We show that the face mediated human-robot interaction approach leads to statistically significant improvements in localizing the hard nodule without compromising the nodule position estimation time. We highlight the inherent power of facial expressions as communicative signals to enhance the utility and effectiveness of human-robot interaction in remote medical examinations.

3D-Printed Soft Sensors for Adaptive Sensing with Online and Offline Tunable Stiffness.

The stiffness of a soft robot with structural cavities can be regulated by controlling the pressure of a fluid to render predictable changes in mechanical properties. When the soft robot interacts with the environment, the mediating fluid can also be considered an inherent information pathway for sensing. This approach to using structural tuning to improve the efficacy of a sensing task with specific states has not yet been well studied. A tunable stiffness soft sensor also renders task-relevant contact dynamics in soft robotic manipulation tasks. This article proposes a type of adaptive soft sensor that can be directly three-dimensional printed and controlled using pneumatic pressure. The tunability of such a sensor helps to adjust the sensing characteristics to better capturing specific tactile features, demonstrated by detecting texture with different frequencies. We present the design, modeling, Finite Element Simulation, and experimental characterization of a single unit of such a tunable stiffness sensor. How the sensing characteristics are affected by adjusting its stiffness is studied in depth. In addition to the tunability, the results show that such types of adaptive sensors exhibit good sensitivity (up to 2.6 KPa/N), high sensor repeatability (average std <0.008 KPa/N), low hysteresis (<6%), and good manufacturing repeatability (average std = 0.0662 KPa/N).

Simulating dynamic facial expressions of pain from visuo-haptic interactions with a robotic patient.

Medical training simulators can provide a safe and controlled environment for medical students to practice their physical examination skills. An important source of information for physicians is the visual feedback of involuntary pain facial expressions in response to physical palpation on an affected area of a patient. However, most existing robotic medical training simulators that can capture physical examination behaviours in real-time cannot display facial expressions and comprise a limited range of patient identities in terms of ethnicity and gender. Together, these limitations restrict the utility of medical training simulators because they do not provide medical students with a representative sample of pain facial expressions and face identities, which could result in biased practices. Further, these limitations restrict the utility of such medical simulators to detect and correct early signs of bias in medical training. Here, for the first time, we present a robotic system that can simulate facial expressions of pain in response to palpations, displayed on a range of patient face identities. We use the unique approach of modelling dynamic pain facial expressions using a data-driven perception-based psychophysical method combined with the visuo-haptic inputs of users performing palpations on a robot medical simulator. Specifically, participants performed palpation actions on the abdomen phantom of a simulated patient, which triggered the real-time display of six pain-related facial Action Units (AUs) on a robotic face (MorphFace), each controlled by two pseudo randomly generated transient parameters: rate of change [Formula: see text] and activation delay [Formula: see text]. Participants then rated the appropriateness of the facial expression displayed in response to their palpations on a 4-point scale from "strongly disagree" to "strongly agree". Each participant ([Formula: see text], 4 Asian females, 4 Asian males, 4 White females and 4 White males) performed 200 palpation trials on 4 patient identities (Black female, Black male, White female and White male) simulated using MorphFace. Results showed facial expressions rated as most appropriate by all participants comprise a higher rate of change and shorter delay from upper face AUs (around the eyes) to those in the lower face (around the mouth). In contrast, we found that transient parameter values of most appropriate-rated pain facial expressions, palpation forces, and delays between palpation actions varied across participant-simulated patient pairs according to gender and ethnicity. These findings suggest that gender and ethnicity biases affect palpation strategies and the perception of pain facial expressions displayed on MorphFace. We anticipate that our approach will be used to generate physical examination models with diverse patient demographics to reduce erroneous judgments in medical students, and provide focused training to address these errors.

Action Augmentation of Tactile Perception for Soft-Body Palpation.

Medical palpation is a diagnostic technique in which physicians use the sense of touch to manipulate the soft human tissue. This can be done to enable the diagnosis of possibly life-threatening conditions, such as cancer. Palpation is still poorly understood because of the complex interaction dynamics between the practitioners' hands and the soft human body. To understand this complex of soft body interactions, we explore robotic palpation for the purpose of diagnosing the presence of abnormal inclusions, or tumors. Using a Bayesian framework for training and classification, we show that the exploration of soft bodies requires complex, multi-axis, palpation trajectories. We also find that this probabilistic approach is capable of rapidly searching the large action space of the robot. This work progresses "robotic" palpation, and it provides frameworks for understanding and exploiting soft body interactions.

Comparative Analysis of Model-Based Predictive Shared Control for Delayed Operation in Object Reaching and Recognition Tasks With Tactile Sensing.

Communication delay represents a fundamental challenge in telerobotics: on one hand, it compromises the stability of teleoperated robots, on the other hand, it decreases the user's awareness of the designated task. In scientific literature, such a problem has been addressed both with statistical models and neural networks (NN) to perform sensor prediction, while keeping the user in full control of the robot's motion. We propose shared control as a tool to compensate and mitigate the effects of communication delay. Shared control has been proven to enhance precision and speed in reaching and manipulation tasks, especially in the medical and surgical fields. We analyse the effects of added delay and propose a unilateral teleoperated leader-follower architecture that both implements a predictive system and shared control, in a 1-dimensional reaching and recognition task with haptic sensing. We propose four different control modalities of increasing autonomy: non-predictive human control (HC), predictive human control (PHC), (shared) predictive human-robot control (PHRC), and predictive robot control (PRC). When analyzing how the added delay affects the subjects' performance, the results show that the HC is very sensitive to the delay: users are not able to stop at the desired position and trajectories exhibit wide oscillations. The degree of autonomy introduced is shown to be effective in decreasing the total time requested to accomplish the task. Furthermore, we provide a deep analysis of environmental interaction forces and performed trajectories. Overall, the shared control modality, PHRC, represents a good trade-off, having peak performance in accuracy and task time, a good reaching speed, and a moderate contact with the object of interest.

Adapting the visuo-haptic perception through muscle coactivation.

While the nervous system can coordinate muscles' activation to shape the mechanical interaction with the environment, it is unclear if and how the arm's coactivation influences visuo-haptic perception and motion planning. Here we show that the nervous system can voluntarily coactivate muscles to improve the quality of the haptic percept. Subjects tracked a randomly moving visual target they were physically coupled to through a virtual elastic band, where the stiffness of the coupling increased with wrist coactivation. Subjects initially relied on vision alone to track the target, but with practice they learned to combine the visual and haptic percepts in a Bayesian manner to improve their tracking performance. This improvement cannot be explained by the stronger mechanical guidance from the elastic band. These results suggest that with practice the nervous system can learn to integrate a novel haptic percept with vision in an optimal fashion.

A Soft Pressure Sensor Skin to Predict Contact Pressure Limit Under Hand Orthosis.

Customized static orthoses in rehabilitation clinics often cause side effects, such as discomfort and skin damage due to excessive local contact pressure. Currently, clinicians adjust orthoses to reduce high contact pressure based on subjective feedback from patients. However, the adjustment is inefficient and prone to variability due to the unknown contact pressure distribution as well as differences in discomfort due to pressure across patients. This paper proposed a new method to predict a threshold of contact pressure (pressure limit) associated with moderate discomfort at each critical spot under hand orthoses. A new pressure sensor skin with 13 sensing units was configured from FEA results of pressure distribution simulated with hand geometry data of six healthy participants. It was used to measure contact pressure under two types of customized orthoses for 40 patients with bone fractures. Their subjective perception of discomfort was also measured using a 6 scores discomfort scale. Based on these data, five critical spots were identified that correspond to high discomfort scores (>1) or high pressure magnitudes (>0.024 MPa). An artificial neural network was trained to predict contact pressure at each critical spot with orthosis type, gender, height, weight, discomfort scores and pressure measurements as input variables. The neural networks show satisfactory prediction accuracy with R2 values over 0.81 of regression between network outputs and measurements. This new method predicts a set of pressure limits at critical locations under the orthosis that the clinicians can use to make orthosis adjustment decisions.

Conditioned haptic perception for 3D localization of nodules in soft tissue palpation with a variable stiffness probe.

This paper provides a solution for fast haptic information gain during soft tissue palpation using a Variable Lever Mechanism (VLM) probe. More specifically, we investigate the impact of stiffness variation of the probe to condition likelihood functions of the kinesthetic force and tactile sensors measurements during a palpation task for two sweeping directions. Using knowledge obtained from past probing trials or Finite Element (FE) simulations, we implemented this likelihood conditioning in an autonomous palpation control strategy. Based on a recursive Bayesian inferencing framework, this new control strategy adapts the sweeping direction and the stiffness of the probe to detect abnormal stiff inclusions in soft tissues. This original control strategy for compliant palpation probes shows a sub-millimeter accuracy for the 3D localization of the nodules in a soft tissue phantom as well as a 100% reliability detecting the existence of nodules in a soft phantom.

Elasticity Versus Hyperelasticity Considerations in Quasistatic Modeling of a Soft Finger-Like Robotic Appendage for Real-Time Position and Force Estimation.

Various methods based on hyperelastic assumptions have been developed to address the mathematical complexities of modeling motion and deformation of continuum manipulators. In this study, we propose a quasistatic approach for 3D modeling and real-time simulation of a pneumatically actuated soft continuum robotic appendage to estimate the contact force and overall pose. Our model can incorporate external load at any arbitrary point on the body and deliver positional and force propagation information along the entire backbone. In line with the proposed model, the effectiveness of elasticity versus hyperelasticity assumptions (neo-Hookean and Gent) is investigated and compared. Experiments are carried out with and without external load, and simulations are validated across a range of Young's moduli. Results show best conformity with Hooke's model for limited strains with about 6% average normalized error of position; and a mean absolute error of less than 0.08 N for force applied at the tip and on the body, demonstrating high accuracy in estimating the position and the contact force.

Modelling the structure of object-independent human affordances of approaching to grasp for robotic hands.

Grasp affordances in robotics represent different ways to grasp an object involving a variety of factors from vision to hand control. A model of grasp affordances that is able to scale across different objects, features and domains is needed to provide robots with advanced manipulation skills. The existing frameworks, however, can be difficult to extend towards a more general and domain independent approach. This work is the first step towards a modular implementation of grasp affordances that can be separated into two stages: approach to grasp and grasp execution. In this study, human experiments of approaching to grasp are analysed, and object-independent patterns of motion are defined and modelled analytically from the data. Human subjects performed a specific action (hammering) using objects of different geometry, size and weight. Motion capture data relating the hand-object approach distance was used for the analysis. The results showed that approach to grasp can be structured in four distinct phases that are best represented by non-linear models, independent from the objects being handled. This suggests that approaching to grasp patterns are following an intentionally planned control strategy, rather than implementing a reactive execution.

Predicting the mean first passage time (MFPT) to reach any state for a passive dynamic walker with steady state variability.

Idealized passive dynamic walkers (PDW) exhibit limit cycle stability at steady state. Yet in reality, uncertainty in ground interaction forces result in variability in limit cycles even for a simple walker known as the Rimless Wheel (RW) on seemingly even slopes. This class of walkers is called metastable walkers in that they usually walk in a stable limit cycle, though guaranteed to eventually fail. Thus, control action is only needed if a failure state (i.e. RW stopping down the ramp) is imminent. Therefore, efficiency of estimating the time to reach a failure state is key to develop a minimal intervention controller to inject just enough energy to overcome a failure state when required. Current methods use what is known as a Mean First Passage Time (MFPT) from current state (rotary speed of RW at the most recent leg collision) to an arbitrary state deemed to be a failure in the future. The frequently used Markov chain based MFPT prediction requires an absorbing state, which in this case is a collision where the RW comes to a stop without an escape. Here, we propose a novel method to estimate an MFPT from current state to an arbitrary state which is not necessarily an absorbing state. This provides freedom to a controller to adaptively take action when deemed necessary. We demonstrate the proposed MFPT predictions in a minimal intervention controller for a RW. Our results show that the proposed method is useful in controllers for walkers showing up to 44.1% increase of time-to-fail compared to a PID based closed-loop controller.

Granular Jamming Based Controllable Organ Design for Abdominal Palpation.

Medical manikins play an essential role in the training process of physicians. Currently, most available simulators for abdominal palpation training do not contain controllable organs for dynamic simulations. In this paper, we present a soft robotics controllable liver that can simulate various liver diseases and symptoms for effective and realistic palpation training. The tumors in the liver model are designed based on granular jamming with positive pressure, which converts the fluid-like impalpable particles to a solid-like tumor state by applying low positive pressure on the membrane. Through inflation, the tumor size, liver stiffness, and liver size can be controlled from normal liver state to various abnormalities including enlarged liver, cirrhotic liver, and multiple cancerous and malignant tumors. Mechanical tests have been conducted in the study to evaluate the liver design and the role of positive pressure granular jamming in tumor simulations.

Correction: Palpation force modulation strategies to identify hard regions in soft tissue organs.

[This corrects the article DOI: 10.1371/journal.pone.0171706.].