Robotic world models-conceptualization, review, and engineering best practices.

The term "world model" (WM) has surfaced several times in robotics, for instance, in the context of mobile manipulation, navigation and mapping, and deep reinforcement learning. Despite its frequent use, the term does not appear to have a concise definition that is consistently used across domains and research fields. In this review article, we bootstrap a terminology for WMs, describe important design dimensions found in robotic WMs, and use them to analyze the literature on WMs in robotics, which spans four decades. Throughout, we motivate the need for WMs by using principles from software engineering, including "Design for use," "Do not repeat yourself," and "Low coupling, high cohesion." Concrete design guidelines are proposed for the future development and implementation of WMs. Finally, we highlight similarities and differences between the use of the term "world model" in robotic mobile manipulation and deep reinforcement learning.

Latent Ergonomics Maps: Real-Time Visualization of Estimated Ergonomics of Human Movements.

Improving the ergonomy of working environments is essential to reducing work-related musculo-skeletal disorders. We consider real-time ergonomic feedback a key technology for achieving such improvements. To this end, we present supportive tools for online evaluation and visualization of strenuous efforts and postures of a worker, also when physically interacting with a robot. A digital human model is used to estimate human kinematics and dynamics and visualize non-ergonomic joint angles, based on the on-line data acquired from a wearable motion tracking device.

A Digital Twin Approach for Contextual Assistance for Surgeons During Surgical Robotics Training.

Minimally invasive robotic surgery copes with some disadvantages for the surgeon of minimally invasive surgery while preserving the advantages for the patient. Most commercially available robotic systems are telemanipulated with haptic input devices. The exploitation of the haptics channel, e.g., by means of Virtual Fixtures, would allow for an individualized enhancement of surgical performance with contextual assistance. However, it remains an open field of research as it is non-trivial to estimate the task context itself during a surgery. In contrast, surgical training allows to abstract away from a real operation and thus makes it possible to model the task accurately. The presented approach exploits this fact to parameterize Virtual Fixtures during surgical training, proposing a Shared Control Parametrization Engine that retrieves procedural context information from a Digital Twin. This approach accelerates a proficient use of the robotic system for novice surgeons by augmenting the surgeon's performance through haptic assistance. With this our aim is to reduce the required skill level and cognitive load of a surgeon performing minimally invasive robotic surgery. A pilot study is performed on the DLR MiroSurge system to evaluate the presented approach. The participants are tasked with two benchmark scenarios of surgical training. The execution of the benchmark scenarios requires basic skills as pick, place and path following. The evaluation of the pilot study shows the promising trend that novel users profit from the haptic augmentation during training of certain tasks.

Fault-Tolerant Six-DoF Pose Estimation for Tendon-Driven Continuum Mechanisms.

We propose a fault-tolerant estimation technique for the six-DoF pose of a tendon-driven continuum mechanisms using machine learning. In contrast to previous estimation techniques, no deformation model is required, and the pose prediction is rather performed with polynomial regression. As only a few datapoints are required for the regression, several estimators are trained with structured occlusions of the available sensor information, and clustered into ensembles based on the available sensors. By computing the variance of one ensemble, the uncertainty in the prediction is monitored and, if the variance is above a threshold, sensor loss is detected and handled. Experiments on the humanoid neck of the DLR robot DAVID, demonstrate that the accuracy of the predicted pose is significantly improved, and a reliable prediction can still be performed using only 3 out of 8 sensors.

Sensorimotor impairment and haptic support in microgravity.

Future space missions envisage human operators teleoperating robotic systems from orbital spacecraft. A potential risk for such missions is the observation that sensorimotor performance deteriorates during spaceflight. This article describes an experiment on sensorimotor performance in two-dimensional manual tracking during different stages of a space mission. We investigated whether there are optimal haptic settings of the human-machine interface for microgravity conditions. Two empirical studies using the same task paradigm with a force feedback joystick with different haptic settings (no haptics, four spring stiffnesses, two motion dampings, three masses) are presented in this paper. (1) A terrestrial control study ([Formula: see text] subjects) with five experimental sessions to explore potential learning effects and interactions with haptic settings. (2) A space experiment ([Formula: see text] cosmonauts) with a pre-mission, three mission sessions on board the ISS (2, 4, and 6 weeks in space), and a post-mission session. Results provide evidence that distorted proprioception significantly affects motion smoothness in the early phase of adaptation to microgravity, while the magnitude of this effect was moderated by cosmonauts' sensorimotor capabilities. Moreover, this sensorimotor impairment can be compensated by providing subtle haptic cues. Specifically, low damping improved tracking smoothness for both motion directions (sagittal and transverse motion plane) and low stiffness improved performance in the transverse motion plane.

Sensorimotor performance and haptic support in simulated weightlessness.

The success of many space missions critically depends on human capabilities and performance. Yet, it is known that sensorimotor performance is degraded under conditions of weightlessness. Therefore, astronauts prepare for their missions in simulated weightlessness under water. In the present study, we investigated sensorimotor performance in simulated weightlessness (induced by shallow water immersion) and whether performance can be improved by choosing appropriate haptic settings of the human-machine interface (e.g., motion damping). Twenty-two participants performed basic aiming and tracking tasks with a force feedback joystick under water and on land and with different haptic settings of the joystick (no haptics, three spring stiffnesses, and two motion dampings). While higher resistive forces should be avoided for rapid aiming tasks in simulated weightlessness, tracking performance is best with higher motions damping in both land and water setups, although the performance losses due to water immersion cannot be compensated. The overall result pattern also provides insights into the causal mechanism behind the slowing effect during aiming motions and decreased accuracy of tracking motions in simulated weightlessness. Findings provide evidence that distorted proprioception due to altered muscle spindle activity seemingly is the main trigger of impaired sensorimotor performance in simulated weightlessness.

Policy search in continuous action domains: An overview.

Continuous action policy search is currently the focus of intensive research, driven both by the recent success of deep reinforcement learning algorithms and the emergence of competitors based on evolutionary algorithms. In this paper, we present a broad survey of policy search methods, providing a unified perspective on very different approaches, including also Bayesian Optimization and directed exploration methods. The main message of this overview is in the relationship between the families of methods, but we also outline some factors underlying sample efficiency properties of the various approaches.

Proximodistal exploration in motor learning as an emergent property of optimization.

To harness the complexity of their high-dimensional bodies during sensorimotor development, infants are guided by patterns of freezing and freeing of degrees of freedom. For instance, when learning to reach, infants free the degrees of freedom in their arm proximodistally, that is, from joints that are closer to the body to those that are more distant. Here, we formulate and study computationally the hypothesis that such patterns can emerge spontaneously as the result of a family of stochastic optimization processes, without an innate encoding of a maturational schedule. In particular, we present simulated experiments with an arm where a computational learner progressively acquires reaching skills through adaptive exploration, and we show that a proximodistal organization appears spontaneously, which we denote PDFF (Proximo Distal Freezing and Freeing of degrees of freedom). We also compare this emergent organization between different arm morphologies-from human-like to quite unnatural ones-to study the effect of different kinematic structures on the emergence of PDFF.

Many regression algorithms, one unified model: A review.

Regression is the process of learning relationships between inputs and continuous outputs from example data, which enables predictions for novel inputs. The history of regression is closely related to the history of artificial neural networks since the seminal work of Rosenblatt (1958). The aims of this paper are to provide an overview of many regression algorithms, and to demonstrate how the function representation whose parameters they regress fall into two classes: a weighted sum of basis functions, or a mixture of linear models. Furthermore, we show that the former is a special case of the latter. Our ambition is thus to provide a deep understanding of the relationship between these algorithms, that, despite being derived from very different principles, use a function representation that can be captured within one unified model. Finally, step-by-step derivations of the algorithms from first principles and visualizations of their inner workings allow this article to be used as a tutorial for those new to regression.

Learning local objective functions for robust face model fitting.

Model-based techniques have proven to be successful in interpreting the large amount of information contained in images. Associated fitting algorithms search for the global optimum of an objective function, which should correspond to the best model fit in a given image. Although fitting algorithms have been the subject of intensive research and evaluation, the objective function is usually designed ad hoc, based on implicit and domain-dependent knowledge. In this article, we address the root of the problem by learning more robust objective functions. First, we formulate a set of desirable properties for objective functions and give a concrete example function that has these properties. Then, we propose a novel approach that learns an objective function from training data generated by manual image annotations and this ideal objective function. In this approach, critical decisions such as feature selection are automated, and the remaining manual steps hardly require domain-dependent knowledge. Furthermore, an extensive empirical evaluation demonstrates that the obtained objective functions yield more robustness. Learned objective functions enable fitting algorithms to determine the best model fit more accurately than with designed objective functions.