The future lies in a pair of tactile hands.

Advancing robot hand dexterity with optical tactile sensing raises questions about humanoid robotics.

A Lightweight and Affordable Wearable Haptic Controller for Robot-Assisted Microsurgery.

In robot-assisted microsurgery (RAMS), surgeons often face the challenge of operating with minimal feedback, particularly lacking in haptic feedback. However, most traditional desktop haptic devices have restricted operational areas and limited dexterity. This report describes a novel, lightweight, and low-budget wearable haptic controller for teleoperated microsurgical robotic systems. We designed a wearable haptic interface entirely made using off-the-shelf material-PolyJet Photopolymer, fabricated using liquid and solid hybrid 3D co-printing technology. This interface was designed to resemble human soft tissues and can be wrapped around the fingertips, offering direct contact feedback to the operator. We also demonstrated that the device can be easily integrated with our motion tracking system for remote microsurgery. Two motion tracking methods, marker-based and marker-less, were compared in trajectory-tracking experiments at different depths to find the most effective motion tracking method for our RAMS system. The results indicate that within the 4 to 8 cm tracking range, the marker-based method achieved exceptional detection rates. Furthermore, the performance of three fusion algorithms was compared to establish the unscented Kalman filter as the most accurate and reliable. The effectiveness of the wearable haptic controller was evaluated through user studies focusing on the usefulness of haptic feedback. The results revealed that haptic feedback significantly enhances depth perception for operators during teleoperated RAMS.

Towards Living Machines: current and future trends of tactile sensing, grasping, and social robotics.

The development of future technologies can be highly influenced by our deeper understanding of the principles that underlie living organisms. The Living Machines conference aims at presenting (among others) the interdisciplinary work of behaving systems based on such principles. Celebrating the 10 years of the conference, we present the progress and future challenges of some of the key themes presented in the robotics workshop of the Living Machines conference. More specifically, in this perspective paper, we focus on the advances in the field of biomimetics and robotics for the creation of artificial systems that can robustly interact with their environment, ranging from tactile sensing, grasping, and manipulation to the creation of psychologically plausible agents.

Neuromorphic Tactile Edge Orientation Classification in an Unsupervised Spiking Neural Network.

Dexterous manipulation in robotic hands relies on an accurate sense of artificial touch. Here we investigate neuromorphic tactile sensation with an event-based optical tactile sensor combined with spiking neural networks for edge orientation detection. The sensor incorporates an event-based vision system (mini-eDVS) into a low-form factor artificial fingertip (the NeuroTac). The processing of tactile information is performed through a Spiking Neural Network with unsupervised Spike-Timing-Dependent Plasticity (STDP) learning, and the resultant output is classified with a 3-nearest neighbours classifier. Edge orientations were classified in 10-degree increments while tapping vertically downward and sliding horizontally across the edge. In both cases, we demonstrate that the sensor is able to reliably detect edge orientation, and could lead to accurate, bio-inspired, tactile processing in robotics and prosthetics applications.

Degenerate boundaries for multiple-alternative decisions.

Integration-to-threshold models of two-choice perceptual decision making have guided our understanding of human and animal behavior and neural processing. Although such models seem to extend naturally to multiple-choice decision making, consensus on a normative framework has yet to emerge, and hence the implications of threshold characteristics for multiple choices have only been partially explored. Here we consider sequential Bayesian inference and a conceptualisation of decision making as a particle diffusing in n-dimensions. We show by simulation that, within a parameterised subset of time-independent boundaries, the optimal decision boundaries comprise a degenerate family of nonlinear structures that jointly depend on the state of multiple accumulators and speed-accuracy trade-offs. This degeneracy is contrary to current 2-choice results where there is a single optimal threshold. Such boundaries support both stationary and collapsing thresholds as optimal strategies for decision-making, both of which result from stationary representations of nonlinear boundaries. Our findings point towards a normative theory of multiple-choice decision making, provide a characterisation of optimal decision thresholds under this framework, and inform the debate between stationary and dynamic decision boundaries for optimal decision making.

Artificial SA-I, RA-I and RA-II/vibrotactile afferents for tactile sensing of texture.

Robot touch can benefit from how humans perceive tactile textural information, from the stimulation mode to which tactile channels respond, then the tactile cues and encoding. Using a soft biomimetic tactile sensor (the TacTip) based on the physiology of the dermal-epidermal boundary, we construct two biomimetic tactile channels based on slowly adapting SA-I and rapidly adapting RA-I afferents, and introduce an additional sub-modality for vibrotactile information with an embedded microphone interpreted as an artificial RA-II channel. These artificial tactile channels are stimulated dynamically with a set of 13 artificial rigid textures comprising raised-bump patterns on a rotating drum that vary systematically in roughness. Methods employing spatial, spatio-temporal and temporal codes are assessed for texture classification insensitive to stimulation speed. We find: (i) spatially encoded frictional cues provide a salient representation of texture; (ii) a simple transformation of spatial tactile features to model natural afferent responses improves the temporal coding; and (iii) the harmonic structure of induced vibrations provides a pertinent code for speed-invariant texture classification. Just as human touch relies on an interplay between slowly adapting (SA-I), rapidly adapting (RA-I) and vibrotactile (RA-II) channels, this tripartite structure may be needed for future robot applications with human-like dexterity, from prosthetics to materials testing, handling and manipulation.

Artificial SA-I and RA-I afferents for tactile sensing of ridges and gratings.

For robot touch to reach the capabilities of human touch, artificial tactile sensors may require transduction principles like those of natural tactile afferents. Here we propose that a biomimetic tactile sensor (the TacTip) could provide suitable artificial analogues of the tactile skin dynamics, afferent responses and population encoding. Our three-dimensionally printed sensor skin is based on the physiology of the dermal-epidermal interface with an underlying mesh of biomimetic intermediate ridges and dermal papillae, comprising inner pins tipped with markers. Slowly adapting SA-I activity is modelled by marker displacements and rapidly adapting RA-I activity by marker speeds. We test the biological plausibility of these artificial population codes with three classic experiments used for natural touch: (1a) responses to normal pressure to test adaptation of single afferents and spatial modulation across the population; (1b) responses to bars, edges and gratings to compare with measurements from monkey primary afferents; and (2) discrimination of grating orientation to compare with human perceptual performance. Our results show a match between artificial and natural touch at single afferent, population and perceptual levels. As expected, natural skin is more sensitive, which raises a challenge to fabricate a biomimetic fingertip that demonstrates human sensitivity using the transduction principles of human touch.

Learning offline: memory replay in biological and artificial reinforcement learning.

Learning to act in an environment to maximise rewards is among the brain's key functions. This process has often been conceptualised within the framework of reinforcement learning, which has also gained prominence in machine learning and artificial intelligence (AI) as a way to optimise decision making. A common aspect of both biological and machine reinforcement learning is the reactivation of previously experienced episodes, referred to as replay. Replay is important for memory consolidation in biological neural networks and is key to stabilising learning in deep neural networks. Here, we review recent developments concerning the functional roles of replay in the fields of neuroscience and AI. Complementary progress suggests how replay might support learning processes, including generalisation and continual learning, affording opportunities to transfer knowledge across the two fields to advance the understanding of biological and artificial learning and memory.

Tactile Model O: Fabrication and Testing of a 3D-Printed, Three-Fingered Tactile Robot Hand.

Bringing tactile sensation to robotic hands will allow for more effective grasping, along with a wide range of benefits of human-like touch. Here, we present a three-dimensional-printed, three-fingered tactile robot hand comprising an OpenHand ModelO customized to house a TacTip soft biomimetic tactile sensor in the distal phalanx of each finger. We expect that combining the grasping capabilities of this underactuated hand with sophisticated tactile sensing will result in an effective platform for robot hand research-the Tactile Model O (T-MO). The design uses three JeVois machine vision systems, with each comprising a miniature camera in the tactile fingertip with a processing module in the base of the hand. To evaluate the capabilities of the T-MO, we benchmark its grasping performance by using the Gripper Assessment Benchmark on the Yale-CMU-Berkeley object set. Tactile sensing capabilities are evaluated by performing tactile object classification on 26 objects and predicting whether a grasp will successfully lift each object. Results are consistent with the state of the art, taking advantage of advances in deep learning applied to tactile image outputs. Overall, this work demonstrates that the T-MO is an effective platform for robot hand research and we expect it to open up a range of applications in autonomous object handling.

The TacTip Family: Soft Optical Tactile Sensors with 3D-Printed Biomimetic Morphologies.

Tactile sensing is an essential component in human-robot interaction and object manipulation. Soft sensors allow for safe interaction and improved gripping performance. Here we present the TacTip family of sensors: a range of soft optical tactile sensors with various morphologies fabricated through dual-material 3D printing. All of these sensors are inspired by the same biomimetic design principle: transducing deformation of the sensing surface via movement of pins analogous to the function of intermediate ridges within the human fingertip. The performance of the TacTip, TacTip-GR2, TacTip-M2, and TacCylinder sensors is here evaluated and shown to attain submillimeter accuracy on a rolling cylinder task, representing greater than 10-fold super-resolved acuity. A version of the TacTip sensor has also been open-sourced, enabling other laboratories to adopt it as a platform for tactile sensing and manipulation research. These sensors are suitable for real-world applications in tactile perception, exploration, and manipulation, and will enable further research and innovation in the field of soft tactile sensing.

Biomimetic Active Touch with Fingertips and Whiskers.

This study provides a synthetic viewpoint that compares, contrasts, and draws commonalities for biomimetic perception over a range of tactile sensors and tactile stimuli. Biomimetic active perception is formulated from three principles: (i) evidence accumulation based on leading models of perceptual decision making; (ii) action selection with an evidence-based policy, here based on overt focal attention; and (iii) sensory encoding of evidence based on neural coding. Two experiments with each of three biomimetic tactile sensors are considered: the iCub (capacitive) fingertip, the TacTip (optical) tactile sensor, and BIOTACT whiskers. For each sensor, one experiment considers a similar task (perception of shape and location) and the other a different tactile perception task. In all experiments, active perception with a biomimetic action selection policy based on focal attention outperforms passive perception with static or random action selection. The active perception also consistently reaches superresolved accuracy (hyperacuity) finer than the spacing between tactile elements. Biomimetic active touch thus offers a common approach for biomimetic tactile sensors to accurately and robustly characterize and explore non-trivial, uncertain environments analogous to how animals perceive the natural world.

Analysis of hand kinematics reveals inter-individual differences in intertemporal decision dynamics.

During intertemporal decisions, the preference for smaller, sooner reward over larger-delayed rewards (temporal discounting, TD) exhibits substantial inter-subject variability; however, it is currently unclear what are the mechanisms underlying this apparently idiosyncratic behavior. To answer this question, here we recorded and analyzed mouse movement kinematics during intertemporal choices in a large sample of participants (N = 86). Results revealed a specific pattern of decision dynamics associated with the selection of "immediate" versus "delayed" response alternatives, which well discriminated between a "discounter" versus a "farsighted" behavior-thus representing a reliable behavioral marker of TD preferences. By fitting the Drift Diffusion Model to the data, we showed that differences between discounter and farsighted subjects could be explained in terms of different model parameterizations, corresponding to the use of different choice mechanisms in the two groups. While farsighted subjects were biased toward the "delayed" option, discounter subjects were not correspondingly biased toward the "immediate" option. Rather, as shown by the dynamics of evidence accumulation over time, their behavior was characterized by high choice uncertainty.

Embodied choice: how action influences perceptual decision making.

Embodied Choice considers action performance as a proper part of the decision making process rather than merely as a means to report the decision. The central statement of embodied choice is the existence of bidirectional influences between action and decisions. This implies that for a decision expressed by an action, the action dynamics and its constraints (e.g. current trajectory and kinematics) influence the decision making process. Here we use a perceptual decision making task to compare three types of model: a serial decision-then-action model, a parallel decision-and-action model, and an embodied choice model where the action feeds back into the decision making. The embodied model incorporates two key mechanisms that together are lacking in the other models: action preparation and commitment. First, action preparation strategies alleviate delays in enacting a choice but also modify decision termination. Second, action dynamics change the prospects and create a commitment effect to the initially preferred choice. Our results show that these two mechanisms make embodied choice models better suited to combine decision and action appropriately to achieve suitably fast and accurate responses, as usually required in ecologically valid situations. Moreover, embodied choice models with these mechanisms give a better account of trajectory tracking experiments during decision making. In conclusion, the embodied choice framework offers a combined theory of decision and action that gives a clear case that embodied phenomena such as the dynamics of actions can have a causal influence on central cognition.

Probabilistic Decision Making with Spikes: From ISI Distributions to Behaviour via Information Gain.

Computational theories of decision making in the brain usually assume that sensory 'evidence' is accumulated supporting a number of hypotheses, and that the first accumulator to reach threshold triggers a decision in favour of its associated hypothesis. However, the evidence is often assumed to occur as a continuous process whose origins are somewhat abstract, with no direct link to the neural signals - action potentials or 'spikes' - that must ultimately form the substrate for decision making in the brain. Here we introduce a new variant of the well-known multi-hypothesis sequential probability ratio test (MSPRT) for decision making whose evidence observations consist of the basic unit of neural signalling - the inter-spike interval (ISI) - and which is based on a new form of the likelihood function. We dub this mechanism s-MSPRT and show its precise form for a range of realistic ISI distributions with positive support. In this way we show that, at the level of spikes, the refractory period may actually facilitate shorter decision times, and that the mechanism is robust against poor choice of the hypothesized data distribution. We show that s-MSPRT performance is related to the Kullback-Leibler divergence (KLD) or information gain between ISI distributions, through which we are able to link neural signalling to psychophysical observation at the behavioural level. Thus, we find the mean information needed for a decision is constant, thereby offering an account of Hick's law (relating decision time to the number of choices). Further, the mean decision time of s-MSPRT shows a power law dependence on the KLD offering an account of Piéron's law (relating reaction time to stimulus intensity). These results show the foundations for a research programme in which spike train analysis can be made the basis for predictions about behavior in multi-alternative choice tasks.

The state of the art in biomimetics.

Biomimetics is a research field that is achieving particular prominence through an explosion of new discoveries in biology and engineering. The field concerns novel technologies developed through the transfer of function from biological systems. To analyze the impact of this field within engineering and related sciences, we compiled an extensive database of publications for study with network-based information analysis techniques. Criteria included publications by year and journal or conference, and subject areas judged by popular and common terms in titles. Our results reveal that this research area has expanded rapidly from less than 100 papers per year in the 1990s to several thousand papers per year in the first decade of this century. Moreover, this research is having impact across a variety of research themes, spanning robotics, computer science and bioengineering. In consequence, biomimetics is becoming a leading paradigm for the development of new technologies that will potentially lead to significant scientific, societal and economic impact in the near future.

The basal ganglia optimize decision making over general perceptual hypotheses.

The basal ganglia are a subcortical group of interconnected nuclei involved in mediating action selection within cortex. A recent proposal is that this selection leads to optimal decision making over multiple alternatives because the basal ganglia anatomy maps onto a network implementation of an optimal statistical method for hypothesis testing, assuming that cortical activity encodes evidence for constrained gaussian-distributed alternatives. This letter demonstrates that this model of the basal ganglia extends naturally to encompass general Bayesian sequential analysis over arbitrary probability distributions, which raises the proposal to a practically realizable theory over generic perceptual hypotheses. We also show that the evidence in this model can represent either log likelihoods, log-likelihood ratios, or log odds, all leading proposals for the cortical processing of sensory data. For these reasons, we claim that the basal ganglia optimize decision making over general perceptual hypotheses represented in cortex. The relation of this theory to cortical encoding, cortico-basal ganglia anatomy, and reinforcement learning is discussed.

Optimal decision-making in mammals: insights from a robot study of rodent texture discrimination.

Texture perception is studied here in a physical model of the rat whisker system consisting of a robot equipped with a biomimetic vibrissal sensor. Investigations of whisker motion in rodents have led to several explanations for texture discrimination, such as resonance or stick-slips. Meanwhile, electrophysiological studies of decision-making in monkeys have suggested a neural mechanism of evidence accumulation to threshold for competing percepts, described by a probabilistic model of Bayesian sequential analysis. For our robot whisker data, we find that variable reaction-time decision-making with sequential analysis performs better than the fixed response-time maximum-likelihood estimation. These probabilistic classifiers also use whatever available features of the whisker signals aid the discrimination, giving improved performance over a single-feature strategy, such as matching the peak power spectra of whisker vibrations. These results cast new light on how the various proposals for texture discrimination in rodents depend on the whisker contact mechanics and suggest the possibility of a common account of decision-making across mammalian species.

Efficient fitting of conductance-based model neurons from somatic current clamp.

Estimating biologically realistic model neurons from electrophysiological data is a key issue in neuroscience that is central to understanding neuronal function and network behavior. However, directly fitting detailed Hodgkin-Huxley type model neurons to somatic membrane potential data is a notoriously difficult optimization problem that can require hours/days of supercomputing time. Here we extend an efficient technique that indirectly matches neuronal currents derived from somatic membrane potential data to two-compartment model neurons with passive dendrites. In consequence, this approach can fit semi-realistic detailed model neurons in a few minutes. For validation, fits are obtained to model-derived data for various thalamo-cortical neuron types, including fast/regular spiking and bursting neurons. A key aspect of the validation is sensitivity testing to perturbations arising in experimental data, including sampling rates, inadequately estimated membrane dynamics/channel kinetics and intrinsic noise. We find that maximal conductance estimates and the resulting membrane potential fits diverge smoothly and monotonically from near-perfect matches when unperturbed. Curiously, some perturbations have little effect on the error because they are compensated by the fitted maximal conductances. Therefore, the extended current-based technique applies well under moderately inaccurate model assumptions, as required for application to experimental data. Furthermore, the accompanying perturbation analysis gives insights into neuronal homeostasis, whereby tuning intrinsic neuronal properties can compensate changes from development or neurodegeneration.