Learning a Low-Dimensional Representation of a Safe Region for Safe Reinforcement Learning on Dynamical Systems.

For the safe application of reinforcement learning algorithms to high-dimensional nonlinear dynamical systems, a simplified system model is used to formulate a safe reinforcement learning (SRL) framework. Based on the simplified system model, a low-dimensional representation of the safe region is identified and used to provide safety estimates for learning algorithms. However, finding a satisfying simplified system model for complex dynamical systems usually requires a considerable amount of effort. To overcome this limitation, we propose a general data-driven approach that is able to efficiently learn a low-dimensional representation of the safe region. By employing an online adaptation method, the low-dimensional representation is updated using the feedback data to obtain more accurate safety estimates. The performance of the proposed approach for identifying the low-dimensional representation of the safe region is illustrated using the example of a quadcopter. The results demonstrate a more reliable and representative low-dimensional representation of the safe region compared with previous works, which extends the applicability of the SRL framework.

A Concise and Geometrically Exact Planar Beam Model for Arbitrarily Large Elastic Deformation Dynamics.

The potential of large elastic deformations in control applications, e.g., robotic manipulation, is not yet fully exploited, especially in dynamic contexts. Mainly because essential geometrically exact continuum models are necessary to express these arbitrarily large deformation dynamics, they typically result in a set of nonlinear, coupled, partial differential equations that are unsuited for control applications. Due to this lack of appropriate models, current approaches that try to exploit elastic properties are limited to either small deflection assumptions or quasistatic considerations only. To promote further exploration of this exciting research field of large elastic deflection control, we propose a geometrically exact, but yet concise a beam model for a planar, shear-, and torsion-free case without elongation. The model is derived by reducing the general geometrically exact the 3D Simo-Reissner beam model to this special case, where the assumption of inextensibility allows expressing the couple of planar Cartesian parameters in terms of the curve tangent angle of the beam center line alone. We further elaborate on how the necessary coupling between position-related boundary conditions (i.e., clamped and hinged ends) and the tangent angle parametrization of the beam model can be incorporated in a finite element method formulation and verify all derived expressions by comparison to analytic initial value solutions and an energy analysis of a dynamic simulation result. The presented beam model opens the possibility of designing online feedback control structures for accessing the full potential that elasticity in planar beam dynamics has to offer.

Cooperative Dynamic Manipulation of Unknown Flexible Objects: Joint Energy Injection Based on Simple Pendulum Fundamental Dynamics.

Cooperative dynamic manipulation enlarges the manipulation repertoire of human-robot teams. By means of synchronized swinging motion, a human and a robot can continuously inject energy into a bulky and flexible object in order to place it onto an elevated location and outside the partners' workspace. Here, we design leader and follower controllers based on the fundamental dynamics of simple pendulums and show that these controllers can regulate the swing energy contained in unknown objects. We consider a complex pendulum-like object controlled via acceleration, and an "arm-flexible object-arm" system controlled via shoulder torque. The derived fundamental dynamics of the desired closed-loop simple pendulum behavior are similar for both systems. We limit the information available to the robotic agent about the state of the object and the partner's intention to the forces measured at its interaction point. In contrast to a leader, a follower does not know the desired energy level and imitates the leader's energy flow to actively contribute to the task. Experiments with a robotic manipulator and real objects show the efficacy of our approach for human-robot dynamic cooperative object manipulation.

Temporal-order judgment of visual and auditory stimuli: modulations in situations with and without stimulus discrimination.

Temporal-order judgment (TOJ) tasks are an important paradigm to investigate processing times of information in different modalities. There are a lot of studies on how temporal order decisions can be influenced by stimuli characteristics. However, so far it has not been investigated whether the addition of a choice reaction time (RT) task has an influence on TOJ. Moreover, it is not known when during processing the decision about the temporal order of two stimuli is made. We investigated the first of these two questions by comparing a regular TOJ task with a dual task (DT). In both tasks, we manipulated different processing stages to investigate whether the manipulations have an influence on TOJ and to determine thereby the time of processing at which the decision about temporal order is made. The results show that the addition of a choice RT task does have an influence on the TOJ, but the influence seems to be linked to the kind of manipulation of the processing stages that is used. The results of the manipulations indicate that the temporal order decision in the DT paradigm is made after perceptual processing of the stimuli.

Bio-inspired visual ego-rotation sensor for MAVs.

Flies are capable of extraordinary flight maneuvers at very high speeds largely due to their highly elaborate visual system. In this work we present a fly-inspired FPGA based sensor system able to visually sense rotations around different body axes, for use on board micro aerial vehicles (MAVs). Rotation sensing is performed analogously to the fly's VS cell network using zero-crossing detection. An additional key feature of our system is the ease of adding new functionalities akin to the different tasks attributed to the fly's lobula plate tangential cell network, such as object avoidance or collision detection. Our implementation consists of a modified eneo SC-MVC01 SmartCam module and a custom built circuit board, weighing less than 200 g and consuming less than 4 W while featuring 57,600 individual two-dimensional elementary motion detectors, a 185° field of view and a frame rate of 350 frames per second. This makes our sensor system compact in terms of size, weight and power requirements for easy incorporation into MAV platforms, while autonomously performing all sensing and processing on-board and in real time.

Complementary limb motion estimation for the control of active knee prostheses.

To restore walking after transfemoral amputation, various actuated exoprostheses have been developed, which control the knee torque actively or via variable damping. In both cases, an important issue is to find the appropriate control that enables user-dominated gait. Recently, we suggested a generic method to deduce intended motion of impaired or amputated limbs from residual human body motion. Based on interjoint coordination in physiological gait, statistical regression is used to estimate missing motion. In a pilot study, this complementary limb motion estimation (CLME) strategy is applied to control an active knee exoprosthesis. A motor-driven prosthetic knee with one degree of freedom has been realized, and one above-knee amputee has used it with CLME. Performed tasks are walking on a treadmill and alternating stair ascent and descent. The subject was able to walk on the treadmill at varying speeds, but needed assistance with the stairs, especially to descend. The promising results with CLME are compared with the subject's performance with her own prosthesis, the C-Leg from Otto Bock.

Automation of a suturing device for minimally invasive surgery.

BACKGROUND: In minimally invasive surgery, hand suturing is categorized as a challenge in technique as well as in its duration. This calls for an easily manageable tool, permitting an all-purpose, cost-efficient, and secure viscerosynthesis. Such a tool for this field already exists: the Autosuture EndoStitch(®). In a series of studies the potential for the EndoStitch to accelerate suturing has been proven. However, its ergonomics still limits its applicability. The goal of this study was twofold: propose an optimized and partially automated EndoStitch and compare the conventional EndoStitch to the optimized and partially automated EndoStitch with respect to the speed and precision of suturing. METHODS: Based on the EndoStitch, a partially automated suturing tool has been developed. With the aid of a DC motor, triggered by a button, one can suture by one-fingered handling. Using the partially automated suturing manipulator, 20 surgeons with different levels of laparoscopic experience successfully completed a continuous suture with 10 stitches using the conventional and the partially automated suture manipulator. Before that, each participant was given 1 min of instruction and 1 min for training. Absolute suturing time and stitch accuracy were measured. The quality of the automated EndoStitch with respect to manipulation was tested with the aid of a standardized questionnaire. RESULTS: To compare the two instruments, t tests were used for suturing accuracy and time. Of the 20 surgeons with laparoscopic experience (fewer than 5 laparoscopic interventions, n=9; fewer than 20 laparoscopic interventions, n=7; more than 20 laparoscopic interventions, n=4), there was no significant difference between the two tested systems with respect to stitching accuracy. However, the suturing time was significantly shorter with the Autostitch (P=0.01). The difference in accuracy and speed was not statistically significant considering the laparoscopic experience of the surgeons. The weight and size of the Autostitch have been criticized as well as its cable. However, the comfortable handhold, automatic needle change, and ergonomic manipulation have been rated positive. CONCLUSION: Partially automated suturing in minimally invasive surgery offers advantages with respect to the speed of operation and ergonomics. Ongoing work in this field has to concentrate on minimization, implementation in robotic systems, and development of new operation methods (NOTES).

Comparison of people's responses to real and virtual handshakes within a virtual environment.

In this paper we present a method for evaluating a haptic device which simulates human handshakes interfaced via a metal rod. We provide an overview of the haptic demonstrator and the control algorithm used for delivering realistic handshakes. For the evaluation of this handshake demonstrator we introduce a 'ground truth' approach, where we compare the robot handshakes with handshakes operated by a human via the same metal rod. For this, an experiment was carried out where the participants entered a virtual environment, i.e. a virtual cocktail party, and were asked to perform a number of handshakes, either with the robot operating with one of two control algorithms operating the metal rod - a basic one for comparison or the proposed new advanced one, or with a human operating the metal rod. The virtual environment was represented only through audio and haptics, without any visual representation, i.e. the subjects participated blindfolded. The evaluation of each handshake was achieved through the subjective scoring of each of the handshakes. The results of the study show that the demonstrator operating with the proposed new control scheme was evaluated significantly more human-like than with the demonstrator operating with the basic algorithm, and also that the real human handshake was evaluated more like a real human handshake than both types of robot handshakes. Although the difference between the advanced robot and human handshake was significant, the effect sizes are not very different, indicating substantial confusion of participants between the advanced robot and human operated handshakes.

Recognition of affect based on gait patterns.

To provide a means for recognition of affect from a distance, this paper analyzes the capability of gait to reveal a person's affective state. We address interindividual versus person-dependent recognition, recognition based on discrete affective states versus recognition based on affective dimensions, and efficient feature extraction with respect to affect. Principal component analysis (PCA), kernel PCA, linear discriminant analysis, and general discriminant analysis are compared to either reduce temporal information in gait or extract relevant features for classification. Although expression of affect in gait is covered by the primary task of locomotion, person-dependent recognition of motion capture data reaches 95% accuracy based on the observation of a single stride. In particular, different levels of arousal and dominance are suitable for being recognized in gait. It is concluded that gait can be used as an additional modality for the recognition of affect. Application scenarios include monitoring in high-security areas, human-robot interaction, and cognitive home environments.

Combination and Integration in the Perception of Visual-Haptic Compliance Information.

The compliance of a material can be conveyed through mechanical interactions in a virtual environment and perceived through both visual and haptic cues. We investigated this basic aspect of perception. In two experiments, subjects performed compliance discriminations, and the mean perceptual estimate (PSE) and the perceptual standard deviation (proportional to JND) were derived from psychophysical functions. Experiment 1 supported a model in which each modality acted independently to produce a compliance estimate, and the two estimates were then integrated to produce an overall value. Experiment 2 tested three mathematical models of the integration process. The data ruled out exclusive reliance on the more reliable modality and stochastic selection of one modality. Instead, the results supported an integration process that constitutes a weighted summation of two random variables, which are defined by the single modality estimates. The model subsumes optimal fusion but provided valid predictions also if the weights were not optimal. Weights were optimal (i.e., minimized variance) when vision and haptic inputs were congruent, but not when they were incongruent.

Plugfest 2009: Global Interoperability in Telerobotics and Telemedicine.

Despite the great diversity of teleoperator designs and applications, their underlying control systems have many similarities. These similarities can be exploited to enable inter-operability between heterogeneous systems. We have developed a network data specification, the Interoperable Telerobotics Protocol, that can be used for Internet based control of a wide range of teleoperators. In this work we test interoperable telerobotics on the global Internet, focusing on the telesurgery application domain. Fourteen globally dispersed telerobotic master and slave systems were connected in thirty trials in one twenty four hour period. Users performed common manipulation tasks to demonstrate effective master-slave operation. With twenty eight (93%) successful, unique connections the results show a high potential for standardizing telerobotic operation. Furthermore, new paradigms for telesurgical operation and training are presented, including a networked surgery trainer and upper-limb exoskeleton control of micro-manipulators.

Beamforming in noninvasive brain-computer interfaces.

Spatial filtering (SF) constitutes an integral part of building EEG-based brain-computer interfaces (BCIs). Algorithms frequently used for SF, such as common spatial patterns (CSPs) and independent component analysis, require labeled training data for identifying filters that provide information on a subject's intention, which renders these algorithms susceptible to overfitting on artifactual EEG components. In this study, beamforming is employed to construct spatial filters that extract EEG sources originating within predefined regions of interest within the brain. In this way, neurophysiological knowledge on which brain regions are relevant for a certain experimental paradigm can be utilized to construct unsupervised spatial filters that are robust against artifactual EEG components. Beamforming is experimentally compared with CSP and Laplacian spatial filtering (LP) in a two-class motor-imagery paradigm. It is demonstrated that beamforming outperforms CSP and LP on noisy datasets, while CSP and beamforming perform almost equally well on datasets with few artifactual trials. It is concluded that beamforming constitutes an alternative method for SF that might be particularly useful for BCIs used in clinical settings, i.e., in an environment where artifact-free datasets are difficult to obtain.

Reference trajectory generation for rehabilitation robots: complementary limb motion estimation.

For gait rehabilitation robots, an important question is how to ensure stable gait, while avoiding any interaction forces between robot and human in case the patient walks correctly. To achieve this, the definition of "correct" gait needs to adapted both to the individual patient and to the situation. Recently, we proposed a method for online trajectory generation that can be applied for hemiparetic subjects. Desired states for one (disabled) leg are generated online based on the movements of the other (sound) leg. An instantaneous mapping between legs is performed by exploiting physiological interjoint couplings. This way, the patient generates the reference motion for the affected leg autonomously. The approach, called Complementary Limb Motion Estimation (CLME), is implemented on the LOPES gait rehabilitation robot and evaluated with healthy subjects in two different experiments. In a previously described study, subjects walk only with one leg, while the robot's other leg acts as a fake prosthesis, to simulate complete loss of function in one leg. This study showed that CLME ensures stable gait. In a second study, to be presented in this paper, healthy subjects walk with both their own legs to assess the interference with self-determined walking. Evaluation criteria are: Power delivered to the joints by the robot, electromyography (EMG) distortions, and kinematic distortions, all compared to zero torque control, which is the baseline of minimum achievable interference. Results indicate that interference of the robot is lower with CLME than with a fixed reference trajectory, mainly in terms of lowered exchanged power and less alteration of EMG. This implies that subjects can walk more naturally with CLME, and they are assisted less by the robot when it is not needed. Future studies with patients are yet to show whether these properties of CLME transfer to the clinical domain.

Multiclass common spatial patterns and information theoretic feature extraction.

We address two shortcomings of the common spatial patterns (CSP) algorithm for spatial filtering in the context of brain--computer interfaces (BCIs) based on electroencephalography/magnetoencephalography (EEG/MEG): First, the question of optimality of CSP in terms of the minimal achievable classification error remains unsolved. Second, CSP has been initially proposed for two-class paradigms. Extensions to multiclass paradigms have been suggested, but are based on heuristics. We address these shortcomings in the framework of information theoretic feature extraction (ITFE). We show that for two-class paradigms, CSP maximizes an approximation of mutual information of extracted EEG/MEG components and class labels. This establishes a link between CSP and the minimal classification error. For multiclass paradigms, we point out that CSP by joint approximate diagonalization (JAD) is equivalent to independent component analysis (ICA), and provide a method to choose those independent components (ICs) that approximately maximize mutual information of ICs and class labels. This eliminates the need for heuristics in multiclass CSP, and allows incorporating prior class probabilities. The proposed method is applied to the dataset IIIa of the third BCI competition, and is shown to increase the mean classification accuracy by 23.4% in comparison to multiclass CSP.

Neural observer based spasticity quantification during therapeutic muscle stimulation.

Repetitive peripheral magnetic stimulation (RPMS) is an innovative approach in treatment of central paresis, e.g. after stroke. The main goals of our current research are the improvement of the therapeutic effect by inducing closed loop controlled movements on the one hand, and the objective assessment of the RPMS therapy on the other hand. One important parameter that allows the evaluation of the therapy progress and that gives insight in neurological processes is the level of spasticity. Current methods to evaluate spasticity are not completely objective and error-prone. This paper presents a novel method of spasticity quantification. The used algorithms are based on parameter estimation methods and can be executed during the therapeutic stimulation. Hence, objective spasticity parameters can be obtained without applying any extra equipment. The presented method has been tested with one patient.