Anomaly Detection Methods in Autonomous Robotic Missions.

Since 2015, there has been an increase in articles on anomaly detection in robotic systems, reflecting its growing importance in improving the robustness and reliability of the increasingly utilized autonomous robots. This review paper investigates the literature on the detection of anomalies in Autonomous Robotic Missions (ARMs). It reveals different perspectives on anomaly and juxtaposition to fault detection. To reach a consensus, we infer a unified understanding of anomalies that encapsulate their various characteristics observed in ARMs and propose a classification of anomalies in terms of spatial, temporal, and spatiotemporal elements based on their fundamental features. Further, the paper discusses the implications of the proposed unified understanding and classification in ARMs and provides future directions. We envisage a study surrounding the specific use of the term anomaly, and methods for their detection could contribute to and accelerate the research and development of a universal anomaly detection system for ARMs.

Exploring the role of disgust in hands and feet laterality judgement tasks.

The hand laterality task (HLT) and the foot laterality task (FLT) are used to explore motor imagery, the ability to imagine an action without executing it. With our limbs, we interact with our body, with others, and with the environment. These contacts might cause negative feelings, such as disgust. Disgust is elicited by different drivers. For instance, body products and body envelope violations provoke disgust to avoid contaminations and to avoid damaging our bodies. However, not much is known about how disgust changes our motor imagery processes. In this study, we examined whether there is any difference in the ability to imagine hands and feet when these are emotionally charged with reminders of disgust. Thirty-six participants completed an online version of a classic (neutral) HLT and FLT and two emotionally charged (disgust) versions. Our findings show that when body parts are modified so that they elicit emotional processing, disgust is salient overall, rather than being salient specifically for actions. This is true for both our hands and our feet.

More implicit and more explicit motor imagery tasks for exploring the mental representation of hands and feet in action.

The mental representation of the body in action can be explored using motor imagery (MI) tasks. MI tasks can be allocated along a continuum going from more implicit to more explicit tasks, where the discriminant is the degree of action monitoring required to solve the tasks (which is the awareness of using the mental representation of our own body to monitor our motor imagery). Tasks based on laterality judgments, such as the Hand Laterality Task (HLT) and the Foot Laterality Task (FLT), provide an example of more implicit tasks (i.e., less action monitoring is required). While, an example of a more explicit task is the Mental Motor Chronometry task (MMC) for hands and feet, where individuals are asked to perform or imagine performing movements with their limbs (i.e., more action monitoring is required). In our study, we directly compared hands and feet at all these tasks for the first time, as these body districts have different physical features as well as functions. Fifty-five participants were asked to complete an online version of the HLT and FLT (more implicit measure), and an online version of the MMC task for hands and feet (more explicit measure). The mental representation of hands and feet in action differed only when the degree of action monitoring decreased (HLT ≠ FLT); we observed the presence of biomechanical constraints only for hands. Differently, when the degree of action monitoring increased hands and feet did not show any difference (MMC hands = MMC feet). Our results show the presence of a difference in the mental representation of hands and feet in action that specifically depends on the degree of action monitoring.

Influence of the Somatic Rubber Hand Illusion on Maximum Grip Aperture.

The classic rubber hand illusion (RHI), based on visual, proprioceptive, and tactile feedback, can affect actions. However, it is not known whether these effects still occur if the paradigm is administered without visual feedback. In this study, we used the somatic RHI to test in thirty-two healthy individuals whether the incorporation of the rubber hand based on proprioceptive and tactile information only is sufficient to generate changes in actions. We measured maximum grip aperture (GA) changes towards a target and associated brain activations within the dorsal stream before and after the somatic RHI. Behavioural and neuroimaging data do not support an effect on maximum GA when the RHI is based on proprioceptive and tactile information only.

A Review of Exoskeletons Considering Nurses.

Daily tasks of nurses include manual handling to assist patients. Repetitive manual handling leads to high risk of injuries due to the loads on nurses' bodies. Nurses, in hospitals and care homes, can benefit from the advances in exoskeleton technology assisting their manual handling tasks. There are already exoskeletons both in the market and in the research area made to assist physical workers to handle heavy loads. However, those exoskeletons are mostly designed for men, as most physical workers are men, whereas most nurses are women. In the case of nurses, they handle patients, a more delicate task than handling objects, and any such device used by nurses should easily be disinfected. In this study, the needs of nurses are examined, and a review of the state-of-the-art exoskeletons is conducted from the perspective of to what extent the existing technologies address the needs of nurses. Possible solutions and technologies and particularly the needs that have not been addressed by the existing technologies are discussed.

A Robotic Experimental Setup with a Stewart Platform to Emulate Underwater Vehicle-Manipulator Systems.

This study presents an experimental robotic setup with a Stewart platform and a robot manipulator to emulate an underwater vehicle-manipulator system (UVMS). This hardware-based emulator setup consists of a KUKA IIWA14 robotic manipulator mounted on a parallel manipulator, known as Stewart Platform, and a force/torque sensor attached to the end-effector of the robotic arm interacting with a pipe. In this setup, we use realistic underwater vehicle movements either communicated to a system in real-time through 4G routers or recorded in advance in a water tank environment. In addition, we simulate both the water current impact on vehicle movement and dynamic coupling effects between the vehicle and manipulator in a Gazebo-based software simulator and transfer these to the physical robotic experimental setup. Such a complete setup is useful to study the control techniques to be applied on the underwater robotic systems in a dry lab environment and allows us to carry out fast and numerous experiments, circumventing the difficulties with performing similar experiments and data collection with actual underwater vehicles in water tanks. Exemplary controller development studies are carried out for contact management of the UVMS using the experimental setup.

Robust induced ℓ2-ℓ∞ optimal control of discrete-time systems having magnitude and rate-bounded actuators.

The design of robust state- and output-feedback control for uncertain discrete-time systems with physical magnitude and rate constraints on their actuator dynamics was addressed. Unlike the traditional methods such as anti-windup (AW) methods, nested ellipsoids, model predictive controllers (MPCs) and integral quadratic constraints(IQCs) formulated by sector bounded inequalities, this paper uses a transformation of the system dynamics to a form which considers control signal and its rate as controlled outputs and using discrete-time ℓ∞ induced (peak-to-peak) norm from disturbance inputs to these outputs. To cope with the magnitude and rate bound non-linearities together, the induced ℓ∞ norm from disturbance input to the outputs involving control signal and its rate is utilised. On the other hand, discrete-time(DT) induced ℓ2 norm from disturbance input to the main controlled output is used to mitigate the effects of disturbances. We can tackle this ambitious non-linear control problem in the domain of linear convex multi-objective optimal control problem, which can be solved by effective semi-definite optimisation methods by using the proposed transformation and handling the control constraints in terms of worst case peak-to-peak gain of the system. Extended Linear Matrix Inequalities (LMIs) and full block S-procedure based design conditions developed over Linear Fractional Representation(LFR) framework allow the user to obtain robust state- and output-feedback control solutions with reduced conservatism. For the first time, this paper introduces an extended LMI based robust output-feedback control design for magnitude and rate bounded (MRB) systems, using full block S-procedure. We demonstrate the performance of the proposed controller through several simulations over benchmark examples covering systems having multi-variable structures and uncertainties. Our study also involves comparison results with a recently introduced technique based on multi-stage AW technique. The simulation results show that the proposed method of this paper is much effective and less conservative compared to the recent AW method provided in the literature.

Real-Time 3D Tracking of Laparoscopy Training Instruments for Assessment and Feedback.

Assessment of minimally invasive surgical skills is a non-trivial task, usually requiring the presence and time of expert observers, including subjectivity and requiring special and expensive equipment and software. Although there are virtual simulators that provide self-assessment features, they are limited as the trainee loses the immediate feedback from realistic physical interaction. The physical training boxes, on the other hand, preserve the immediate physical feedback, but lack the automated self-assessment facilities. This study develops an algorithm for real-time tracking of laparoscopy instruments in the video cues of a standard physical laparoscopy training box with a single fisheye camera. The developed visual tracking algorithm recovers the 3D positions of the laparoscopic instrument tips, to which simple colored tapes (markers) are attached. With such system, the extracted instrument trajectories can be digitally processed, and automated self-assessment feedback can be provided. In this way, both the physical interaction feedback would be preserved and the need for the observance of an expert would be overcome. Real-time instrument tracking with a suitable assessment criterion would constitute a significant step towards provision of real-time (immediate) feedback to correct trainee actions and show them how the action should be performed. This study is a step towards achieving this with a low cost, automated, and widely applicable laparoscopy training and assessment system using a standard physical training box equipped with a fisheye camera.

Application of Adaptive and Switching Control for Contact Maintenance of a Robotic Vehicle-Manipulator System for Underwater Asset Inspection.

The aim of this study is to design an adaptive controller for the hard contact interaction problem of underwater vehicle-manipulator systems (UVMS) to realize asset inspection through physical interaction. The proposed approach consists of a force and position controller in the operational space of the end effector of the robot manipulator mounted on an underwater vehicle. The force tracking algorithm keeps the end effector perpendicular to the unknown surface of the asset and the position tracking algorithm makes it follow a desired trajectory on the surface. The challenging problem in such a system is to maintain the end effector of the manipulator in continuous and stable contact with the unknown surface in the presence of disturbances and reaction forces that constantly move the floating robot base in an unexpected manner. The main contribution of the proposed controller is the development of the adaptive force tracking control algorithm based on switching actions between contact and noncontact states. When the end effector loses contact with the surface, a velocity feed-forward augmented impedance controller is activated to rapidly regain contact interaction by generating a desired position profile whose speed is adjusted depending on the time and the point where the contact was lost. Once the contact interaction is reestablished, a dynamic adaptive damping-based admittance controller is operated for fast adaptation and continuous stable force tracking. To validate the proposed controller, we conducted experiments with a land robotic setup composed of a 6 degrees of freedom (DOF) Stewart Platform imitating an underwater vehicle and a 7 DOF KUKA IIWA robotic arm imitating the underwater robot manipulator attached to the vehicle. The proposed scheme significantly increases the contact time under realistic disturbances, in comparison to our former controllers without an adaptive control scheme. We have demonstrated the superior performance of the current controller with experiments and quantified measures.

Development of a Robotic Surgery Training System.

Robotic Surgery is getting widely spread and applied to more and more clinical cases due to its advantages compared to open surgery, for both the patients and surgeons. However, Robotic Surgery requires a different set of skills and learning compared to open and also laparoscopic surgery. Tele-operation for a robotic system with hand controllers, the delay in the hand commands to be translated into robotic movements, slowness of the robotic movements, remote 2D or 3D vision of the actual operation, and lack of haptic feedback are some of the challenges that Robotic Surgery poses. Surgeons need to go through an intensive training for Robotic Surgery, and the learning and skill development continues throughout their early professional years. Despite the importance of training for Robotic Surgery, there are not yet dedicated, low-cost, and widespread training platforms; rather, surgeons mostly train with the same Robotic Surgery system they use in surgery; hence institutions need to invest on a separate surgical setup for training purposes. This is expensive for the institutions, it provides very limited access to the surgeons for training, and very limited, if any, access to researchers for experimentation. To address these, we have developed in our laboratory a low-cost, and experimental Robotic Surgery Trainer. This setup replicates the challenges that a Robotic Surgery system poses and further provides widespread access through internet connected control of the actual physical system. The overall system is composed of equipment that a standard engineering laboratory can afford. In this paper, we introduce the Robotic Surgery Training System and explain its development, parts, and functionality.

Causes of Performance Degradation in Non-invasive Electromyographic Pattern Recognition in Upper Limb Prostheses.

Surface Electromyography (EMG)-based pattern recognition methods have been investigated over the past years as a means of controlling upper limb prostheses. Despite the very good reported performance of myoelectric controlled prosthetic hands in lab conditions, real-time performance in everyday life conditions is not as robust and reliable, explaining the limited clinical use of pattern recognition control. The main reason behind the instability of myoelectric pattern recognition control is that EMG signals are non-stationary in real-life environments and present a lot of variability over time and across subjects, hence affecting the system's performance. This can be the result of one or many combined changes, such as muscle fatigue, electrode displacement, difference in arm posture, user adaptation on the device over time and inter-subject singularity. In this paper an extensive literature review is performed to present the causes of the drift of EMG signals, ways of detecting them and possible techniques to counteract for their effects in the application of upper limb prostheses. The suggested techniques are organized in a table that can be used to recognize possible problems in the clinical application of EMG-based pattern recognition methods for upper limb prosthesis applications and state-of-the-art methods to deal with such problems.

Three-dimensional echocardiography for the assessment of left ventricular geometry and papillary muscle morphology in hypertrophic cardiomyopathy.

BACKGROUND: Hypertrophic cardiomyopathy (HC) is characterized by left ventricular (LV) hypertrophy and associated with papillary muscle (PM) abnormalities. The aim of this study was to evaluate the utility of three-dimensional echocardiography (3DE) for the geometric assessment of LV hypertrophy and PM morphology. METHODS: The study included 24 patients with an established diagnosis of HC and 31 healthy controls. 3DE was performed using an iE33 or EPIQ 7C ultrasound system with an X5-1 transducer. QLAB software was used for the 3D analysis of LV wall thickness (LVWT) and PM morphology and hypertrophy; the number and cross-sectional area (CSA) of anterolateral and posteromedial PMs; and the presence of bifid or accessory PMs. RESULTS: Patients with HC had a larger LVWT compared to controls in all segments (p < 0.001), and LVWT was largest in the midventricular septal segment (2.12 ± 0.68 cm). The maximum LVWT followed a spiral pattern from the LV base to the apex. The CSA of both anterolateral and posteromedial PMs was larger in patients with HC than in controls (1.92 vs. 1.15 cm2; p = 0.001 and 1.46 vs. 1.08 cm2; p = 0.033, respectively). The CSA of the posteromedial PM was larger in patients with LVOT obstruction than in those without (2.64 vs 1.16 cm2, p = 0.021). CONCLUSIONS: 3DE allows the assessment of LV geometry and PM abnormalities in patients with HC. 3DE demonstrated that the maximum hypertrophy was variable and generally located in a spiral from the LV base to the apex.

Robotic Assistance by Impedance Compensation for Hand Movements While Manual Welding.

In this paper, we present a robotic assistance scheme which allows for impedance compensation with stiffness, damping, and mass parameters for hand manipulation tasks and we apply it to manual welding. The impedance compensation does not assume a preprogrammed hand trajectory. Rather, the intention of the human for the hand movement is estimated in real time using a smooth Kalman filter. The movement is restricted by compensatory virtual impedance in the directions perpendicular to the estimated direction of movement. With airbrush painting experiments, we test three sets of values for the impedance parameters as inspired from impedance measurements with manual welding. We apply the best of the tested sets for assistance in manual welding and perform welding experiments with professional and novice welders. We contrast three conditions: 1) welding with the robot's assistance; 2) with the robot when the robot is passive; and 3) welding without the robot. We demonstrate the effectiveness of the assistance through quantitative measures of both task performance and perceived user's satisfaction. The performance of both the novice and professional welders improves significantly with robotic assistance compared to welding with a passive robot. The assessment of user satisfaction shows that all novice and most professional welders appreciate the robotic assistance as it suppresses the tremors in the directions perpendicular to the movement for welding.

End-point impedance measurements across dominant and nondominant hands and robotic assistance with directional damping.

The goal of this paper is to perform end-point impedance measurements across dominant and nondominant hands while doing airbrush painting and to use the results for developing a robotic assistance scheme. We study airbrush painting because it resembles in many ways manual welding, a standard industrial task. The experiments are performed with the 7 degrees of freedom KUKA lightweight robot arm. The robot is controlled in admittance using a force sensor attached at the end-point, so as to act as a free-mass and be passively guided by the human. For impedance measurements, a set of nine subjects perform 12 repetitions of airbrush painting, drawing a straight-line on a cartoon horizontally placed on a table, while passively moving the airbrush mounted on the robot's end-point. We measure hand impedance during the painting task by generating sudden and brief external forces with the robot. The results show that on average the dominant hand displays larger impedance than the nondominant in the directions perpendicular to the painting line. We find the most significant difference in the damping values in these directions. Based on this observation, we develop a "directional damping" scheme for robotic assistance and conduct a pilot study with 12 subjects to contrast airbrush painting with and without robotic assistance. Results show significant improvement in precision with both dominant and nondominant hands when using robotic assistance.

Understanding soft-tissue behavior for application to microlaparoscopic surface scan.

This paper presents an approach for understanding the soft-tissue behavior in surface contact with a probe scanning the tissue. The application domain is confocal microlaparoscopy, mostly used for imaging the outer surface of the organs in the abdominal cavity. The probe is swept over the tissue to collect sequential images to obtain a large field of view with mosaicking. The problem we address is that the tissue also moves with the probe due to its softness; therefore, the resulting mosaic is not in the same shape and dimension as traversed by the probe. Our approach is inspired by the finger slip studies and adapts the idea of load-slip phenomenon that explains the movement of the soft part of the finger when dragged on a hard surface. We propose the concept of loading-distance and perform measurements on beef liver and chicken breast tissues. We propose a protocol to determine the loading-distance prior to an automated scan and introduce an approach to compensate the tissue movement in raster scans. Our implementation and experiments show that we can have an image mosaic of the tissue surface in a desired rectangular shape with this approach.

Building large mosaics of confocal edomicroscopic images using visual servoing.

Probe-based confocal laser endomicroscopy provides real-time microscopic images of tissues contacted by a small probe that can be inserted in vivo through a minimally invasive access. Mosaicking consists in sweeping the probe in contact with a tissue to be imaged while collecting the video stream, and process the images to assemble them in a large mosaic. While most of the literature in this field has focused on image processing, little attention has been paid so far to the way the probe motion can be controlled. This is a crucial issue since the precision of the probe trajectory control drastically influences the quality of the final mosaic. Robotically controlled motion has the potential of providing enough precision to perform mosaicking. In this paper, we emphasize the difficulties of implementing such an approach. First, probe-tissue contacts generate deformations that prevent from properly controlling the image trajectory. Second, in the context of minimally invasive procedures targeted by our research, robotic devices are likely to exhibit limited quality of the distal probe motion control at the microscopic scale. To cope with these problems visual servoing from real-time endomicroscopic images is proposed in this paper. It is implemented on two different devices (a high-accuracy industrial robot and a prototype minimally invasive device). Experiments on different kinds of environments (printed paper and ex vivo tissues) show that the quality of the visually servoed probe motion is sufficient to build mosaics with minimal distortion in spite of disturbances.