Dataset with Tactile and Kinesthetic Information from a Human Forearm and Its Application to Deep Learning.

There are physical Human-Robot Interaction (pHRI) applications where the robot has to grab the human body, such as rescue or assistive robotics. Being able to precisely estimate the grasping location when grabbing a human limb is crucial to perform a safe manipulation of the human. Computer vision methods provide pre-grasp information with strong constraints imposed by the field environments. Force-based compliant control, after grasping, limits the amount of applied strength. On the other hand, valuable tactile and proprioceptive information can be obtained from the pHRI gripper, which can be used to better know the features of the human and the contact state between the human and the robot. This paper presents a novel dataset of tactile and kinesthetic data obtained from a robot gripper that grabs a human forearm. The dataset is collected with a three-fingered gripper with two underactuated fingers and a fixed finger with a high-resolution tactile sensor. A palpation procedure is performed to record the shape of the forearm and to recognize the bones and muscles in different sections. Moreover, an application for the use of the database is included. In particular, a fusion approach is used to estimate the actual grasped forearm section using both kinesthetic and tactile information on a regression deep-learning neural network. First, tactile and kinesthetic data are trained separately with Long Short-Term Memory (LSTM) neural networks, considering the data are sequential. Then, the outputs are fed to a Fusion neural network to enhance the estimation. The experiments conducted show good results in training both sources separately, with superior performance when the fusion approach is considered.

Automatically Annotated Dataset of a Ground Mobile Robot in Natural Environments via Gazebo Simulations.

This paper presents a new synthetic dataset obtained from Gazebo simulations of an Unmanned Ground Vehicle (UGV) moving on different natural environments. To this end, a Husky mobile robot equipped with a tridimensional (3D) Light Detection and Ranging (LiDAR) sensor, a stereo camera, a Global Navigation Satellite System (GNSS) receiver, an Inertial Measurement Unit (IMU) and wheel tachometers has followed several paths using the Robot Operating System (ROS). Both points from LiDAR scans and pixels from camera images, have been automatically labeled into their corresponding object class. For this purpose, unique reflectivity values and flat colors have been assigned to each object present in the modeled environments. As a result, a public dataset, which also includes 3D pose ground-truth, is provided as ROS bag files and as human-readable data. Potential applications include supervised learning and benchmarking for UGV navigation on natural environments. Moreover, to allow researchers to easily modify the dataset or to directly use the simulations, the required code has also been released.

Metaheuristic Parameter Identification of Motors Using Dynamic Response Relations.

This article presents the use of the equations of the dynamic response to a step input in metaheuristic algorithm for the parametric estimation of a motor model. The model equations are analyzed, and the relations in steady-state and transient-state are used as delimiters in the search. These relations reduce the number of random parameters in algorithm search and reduce the iterations to find an acceptable result. The tests were implemented in two motors of known parameters to estimate the performance of the modifications in the algorithms. Tests were carried out with three algorithms (Gray Wolf Optimizer, Jaya Algorithm, and Cuckoo Search Algorithm) to prove that the benefits can be extended to various metaheuristics. The search parameters were also varied, and tests were developed with different iterations and populations. The results show an improvement for all the algorithms used, achieving the same error as the original method but with 10 to 50% fewer iterations.

Enabling Remote Responder Bio-Signal Monitoring in a Cooperative Human-Robot Architecture for Search and Rescue.

The roles of emergency responders are challenging and often physically demanding, so it is essential that their duties are performed safely and effectively. In this article, we address real-time bio-signal sensor monitoring for responders in disaster scenarios. In particular, we propose the integration of a set of health monitoring sensors suitable for detecting stress, anxiety and physical fatigue in an Internet of Cooperative Agents architecture for search and rescue (SAR) missions (SAR-IoCA), which allows remote control and communication between human and robotic agents and the mission control center. With this purpose, we performed proof-of-concept experiments with a bio-signal sensor suite worn by firefighters in two high-fidelity SAR exercises. Moreover, we conducted a survey, distributed to end-users through the Fire Brigade consortium of the Provincial Council of Málaga, in order to analyze the firefighters' opinion about biological signals monitoring while on duty. As a result of this methodology, we propose a wearable sensor suite design with the aim of providing some easy-to-wear integrated-sensor garments, which are suitable for emergency worker activity. The article offers discussion of user acceptance, performance results and learned lessons.

The Internet of Cooperative Agents Architecture (X-IoCA) for Robots, Hybrid Sensor Networks, and MEC Centers in Complex Environments: A Search and Rescue Case Study.

Cloud robotics and advanced communications can foster a step-change in cooperative robots and hybrid wireless sensor networks (H-WSN) for demanding environments (e.g., disaster response, mining, demolition, and nuclear sites) by enabling the timely sharing of data and computational resources between robot and human teams. However, the operational complexity of such multi-agent systems requires defining effective architectures, coping with implementation details, and testing in realistic deployments. This article proposes X-IoCA, an Internet of robotic things (IoRT) and communication architecture consisting of a hybrid and heterogeneous network of wireless transceivers (H2WTN), based on LoRa and BLE technologies, and a robot operating system (ROS) network. The IoRT is connected to a feedback information system (FIS) distributed among multi-access edge computing (MEC) centers. Furthermore, we present SAR-IoCA, an implementation of the architecture for search and rescue (SAR) integrated into a 5G network. The FIS for this application consists of an SAR-FIS (including a path planner for UGVs considering risks detected by a LoRa H-WSN) and an ROS-FIS (for real-time monitoring and processing of information published throughout the ROS network). Moreover, we discuss lessons learned from using SAR-IoCA in a realistic exercise where three UGVs, a UAV, and responders collaborated to rescue victims from a tunnel accessible through rough terrain.

Path Planning for Autonomous Mobile Robots: A Review.

Providing mobile robots with autonomous capabilities is advantageous. It allows one to dispense with the intervention of human operators, which may prove beneficial in economic and safety terms. Autonomy requires, in most cases, the use of path planners that enable the robot to deliberate about how to move from its location at one moment to another. Looking for the most appropriate path planning algorithm according to the requirements imposed by users can be challenging, given the overwhelming number of approaches that exist in the literature. Moreover, the past review works analyzed here cover only some of these approaches, missing important ones. For this reason, our paper aims to serve as a starting point for a clear and comprehensive overview of the research to date. It introduces a global classification of path planning algorithms, with a focus on those approaches used along with autonomous ground vehicles, but is also extendable to other robots moving on surfaces, such as autonomous boats. Moreover, the models used to represent the environment, together with the robot mobility and dynamics, are also addressed from the perspective of path planning. Each of the path planning categories presented in the classification is disclosed and analyzed, and a discussion about their applicability is added at the end.

A Redundant Configuration of Four Low-Cost GNSS-RTK Receivers for Reliable Estimation of Vehicular Position and Posture.

This paper proposes a low-cost sensor system composed of four GNSS-RTK receivers to obtain accurate position and posture estimations for a vehicle in real-time. The four antennas of the receivers are placed so that every three-antennas combination is optimal to get the most precise 3D coordinates with respect to a global reference system. The redundancy provided by the fourth receiver allows to improve estimations even more and to maintain accuracy when one of the receivers fails. A mini computer with the Robotic Operating System is responsible for merging all the available measurements reliably. Successful experiments have been carried out with a ground rover on irregular terrain. Angular estimates similar to those of a high-performance IMU have been achieved in dynamic tests.

Development and Implementation of a Hybrid Wireless Sensor Network of Low Power and Long Range for Urban Environments.

The urban population, worldwide, is growing exponentially and with it the demand for information on pollution levels, vehicle traffic, or available parking, giving rise to citizens connected to their environment. This article presents an experimental long range (LoRa) and low power consumption network, with a combination of static and mobile wireless sensors (hybrid architecture) to tune and validate concentrator placement, to obtain a large coverage in an urban environment. A mobile node has been used, carrying a gateway and various sensors. The Activation By Personalization (ABP) mode has been used, justified for urban applications requiring multicasting. This allows to compare the coverage of each static gateway, being able to make practical decisions about its location. With this methodology, it has been possible to provide service to the city of Malaga, through a single concentrator node. The information acquired is synchronized in an external database, to monitor the data in real time, being able to geolocate the dataframes through web mapping services. This work presents the development and implementation of a hybrid wireless sensor network of long range and low power, configured and tuned to achieve efficient performance in a mid-size city, and tested in experiments in a real urban environment.

Statistical Study of the Performance of Recursive Bayesian Filters with Abnormal Observations from Range Sensors.

Range sensors are currently present in countless applications related to perception of the environment. In mobile robots, these devices constitute a key part of the sensory apparatus and enable essential operations, that are often addressed by applying methods grounded on probabilistic frameworks such as Bayesian filters. Unfortunately, modern mobile robots have to navigate within challenging environments from the perspective of their sensory devices, getting abnormal observations (e.g., biased, missing, etc.) that may compromise these operations. Although there exist previous contributions that either address filtering performance or identification of abnormal sensory observations, they do not provide a complete treatment of both problems at once. In this work we present a statistical approach that allows us to study and quantify the impact of abnormal observations from range sensors on the performance of Bayesian filters. For that, we formulate the estimation problem from a generic perspective (abstracting from concrete implementations), analyse the main limitations of common robotics range sensors, and define the factors that potentially affect the filtering performance. Rigorous statistical methods are then applied to a set of simulated experiments devised to reproduce a diversity of situations. The obtained results, which we also validate in a real environment, provide novel and relevant conclusions on the effect of abnormal range observations in these filters.

Proprioceptive Estimation of Forces Using Underactuated Fingers for Robot-Initiated pHRI.

In physical Human-Robot Interaction (pHRI), forces exerted by humans need to be estimated to accommodate robot commands to human constraints, preferences, and needs. This paper presents a method for the estimation of the interaction forces between a human and a robot using a gripper with proprioceptive sensing. Specifically, we measure forces exerted by a human limb grabbed by an underactuated gripper in a frontal plane using only the gripper's own sensors. This is achieved via a regression method, trained with experimental data from the values of the phalanx angles and actuator signals. The proposed method is intended for adaptive shared control in limb manipulation. Although adding force sensors provides better performance, the results obtained are accurate enough for this application. This approach requires no additional hardware: it relies uniquely on the gripper motor feedback-current, position and torque-and joint angles. Also, it is computationally cheap, so processing times are low enough to allow continuous human-adapted pHRI for shared control.

Using 3D Convolutional Neural Networks for Tactile Object Recognition with Robotic Palpation.

In this paper, a novel method of active tactile perception based on 3D neural networks and a high-resolution tactile sensor installed on a robot gripper is presented. A haptic exploratory procedure based on robotic palpation is performed to get pressure images at different grasping forces that provide information not only about the external shape of the object, but also about its internal features. The gripper consists of two underactuated fingers with a tactile sensor array in the thumb. A new representation of tactile information as 3D tactile tensors is described. During a squeeze-and-release process, the pressure images read from the tactile sensor are concatenated forming a tensor that contains information about the variation of pressure matrices along with the grasping forces. These tensors are used to feed a 3D Convolutional Neural Network (3D CNN) called 3D TactNet, which is able to classify the grasped object through active interaction. Results show that 3D CNN performs better, and provide better recognition rates with a lower number of training data.

Integration of a Mobile Node into a Hybrid Wireless Sensor Network for Urban Environments.

Robots, or in general, intelligent vehicles, require large amounts of data to adapt their behavior to the environment and achieve their goals. When their missions take place in large areas, using additional information to that gathered by the onboard sensors frequently offers a more efficient solution of the problem. The emergence of Cyber-Physical Systems and Cloud computing allows this approach, but integration of sensory information, and its effective availability for the robots or vehicles is challenging. This paper addresses the development and implementation of a modular mobile node of a Wireless Sensor Network (WSN), designed to be mounted onboard vehicles, and capable of using different sensors according to mission needs. The mobile node is integrated with an existing static network, transforming it into a Hybrid Wireless Sensor Network (H-WSN), and adding flexibility and range to it. The integration is achieved without the need for multi-hop routing. A database holds the data acquired by both mobile and static nodes, allowing access in real-time to the gathered information. A Human⁻Machine Interface (HMI) presents this information to users. Finally, the system is tested in real urban scenarios in a use-case of measurement of gas levels.

Enhancing Perception with Tactile Object Recognition in Adaptive Grippers for Human-Robot Interaction.

The use of tactile perception can help first response robotic teams in disaster scenarios, where visibility conditions are often reduced due to the presence of dust, mud, or smoke, distinguishing human limbs from other objects with similar shapes. Here, the integration of the tactile sensor in adaptive grippers is evaluated, measuring the performance of an object recognition task based on deep convolutional neural networks (DCNNs) using a flexible sensor mounted in adaptive grippers. A total of 15 classes with 50 tactile images each were trained, including human body parts and common environment objects, in semi-rigid and flexible adaptive grippers based on the fin ray effect. The classifier was compared against the rigid configuration and a support vector machine classifier (SVM). Finally, a two-level output network has been proposed to provide both object-type recognition and human/non-human classification. Sensors in adaptive grippers have a higher number of non-null tactels (up to 37% more), with a lower mean of pressure values (up to 72% less) than when using a rigid sensor, with a softer grip, which is needed in physical human-robot interaction (pHRI). A semi-rigid implementation with 95.13% object recognition rate was chosen, even though the human/non-human classification had better results (98.78%) with a rigid sensor.

Analysis of 3D Scan Measurement Distribution with Application to a Multi-Beam Lidar on a Rotating Platform.

Multi-beam lidar (MBL) rangefinders are becoming increasingly compact, light, and accessible 3D sensors, but they offer limited vertical resolution and field of view. The addition of a degree-of-freedom to build a rotating multi-beam lidar (RMBL) has the potential to become a common solution for affordable rapid full-3D high resolution scans. However, the overlapping of multiple-beams caused by rotation yields scanning patterns that are more complex than in rotating single beam lidar (RSBL). In this paper, we propose a simulation-based methodology to analyze 3D scanning patterns which is applied to investigate the scan measurement distribution produced by the RMBL configuration. With this purpose, novel contributions include: (i) the adaption of a recent spherical reformulation of Ripley's K function to assess 3D sensor data distribution on a hollow sphere simulation; (ii) a comparison, both qualitative and quantitative, between scan patterns produced by an ideal RMBL based on a Velodyne VLP-16 (Puck) and those of other 3D scan alternatives (i.e., rotating 2D lidar and MBL); and (iii) a new RMBL implementation consisting of a portable tilting platform for VLP-16 scanners, which is presented as a case study for measurement distribution analysis as well as for the discussion of actual scans from representative environments. Results indicate that despite the particular sampling patterns given by a RMBL, its homogeneity even improves that of an equivalent RSBL.

Voxel-Based Neighborhood for Spatial Shape Pattern Classification of Lidar Point Clouds with Supervised Learning.

Improving the effectiveness of spatial shape features classification from 3D lidar data is very relevant because it is largely used as a fundamental step towards higher level scene understanding challenges of autonomous vehicles and terrestrial robots. In this sense, computing neighborhood for points in dense scans becomes a costly process for both training and classification. This paper proposes a new general framework for implementing and comparing different supervised learning classifiers with a simple voxel-based neighborhood computation where points in each non-overlapping voxel in a regular grid are assigned to the same class by considering features within a support region defined by the voxel itself. The contribution provides offline training and online classification procedures as well as five alternative feature vector definitions based on principal component analysis for scatter, tubular and planar shapes. Moreover, the feasibility of this approach is evaluated by implementing a neural network (NN) method previously proposed by the authors as well as three other supervised learning classifiers found in scene processing methods: support vector machines (SVM), Gaussian processes (GP), and Gaussian mixture models (GMM). A comparative performance analysis is presented using real point clouds from both natural and urban environments and two different 3D rangefinders (a tilting Hokuyo UTM-30LX and a Riegl). Classification performance metrics and processing time measurements confirm the benefits of the NN classifier and the feasibility of voxel-based neighborhood.

Boresight calibration of construction misalignments for 3D scanners built with a 2D laser range finder rotating on its optical center.

Many applications, like mobile robotics, can profit from acquiring dense, wide-ranging and accurate 3D laser data. Off-the-shelf 2D scanners are commonly customized with an extra rotation as a low-cost, lightweight and low-power-demanding solution. Moreover, aligning the extra rotation axis with the optical center allows the 3D device to maintain the same minimum range as the 2D scanner and avoids offsets in computing Cartesian coordinates. The paper proposes a practical procedure to estimate construction misalignments based on a single scan taken from an arbitrary position in an unprepared environment that contains planar surfaces of unknown dimensions. Inherited measurement limitations from low-cost 2D devices prevent the estimation of very small translation misalignments, so the calibration problem reduces to obtaining boresight parameters. The distinctive approach with respect to previous plane-based intrinsic calibration techniques is the iterative maximization of both the flatness and the area of visible planes. Calibration results are presented for a case study. The method is currently being applied as the final stage in the production of a commercial 3D rangefinder.

Driver assistance system for passive multi-trailer vehicles with haptic steering limitations on the leading unit.

Driving vehicles with one or more passive trailers has difficulties in both forward and backward motion due to inter-unit collisions, jackknife, and lack of visibility. Consequently, advanced driver assistance systems (ADAS) for multi-trailer combinations can be beneficial to accident avoidance as well as to driver comfort. The ADAS proposed in this paper aims to prevent unsafe steering commands by means of a haptic handwheel. Furthermore, when driving in reverse, the steering-wheel and pedals can be used as if the vehicle was driven from the back of the last trailer with visual aid from a rear-view camera. This solution, which can be implemented in drive-by-wire vehicles with hitch angle sensors, profits from two methods previously developed by the authors: safe steering by applying a curvature limitation to the leading unit, and a virtual tractor concept for backward motion that includes the complex case of set-point propagation through on-axle hitches. The paper addresses system requirements and provides implementation details to tele-operate two different off- and on-axle combinations of a tracked mobile robot pulling and pushing two dissimilar trailers.

A large area tactile sensor patch based on commercial force sensors.

This paper reports the design of a tactile sensor patch to cover large areas of robots and machines that interact with human beings. Many devices have been proposed to meet such a demand. These realizations are mostly custom-built or developed in the lab. The sensor of this paper is implemented with commercial force sensors. This has the benefit of a more foreseeable response of the sensor if its behavior is understood as the aggregation of readings from all the individual force sensors in the array. A few reported large area tactile sensors are also based on commercial sensors. However, the one in this paper is the first of this kind based on the use of polymeric commercial force sensing resistors (FSR) as unit elements of the array or tactels, which results in a robust sensor. The paper discusses design issues related to some necessary modifications of the force sensor, its assembly in an array, and the signal conditioning. The patch has 16 × 9 force sensors mounted on a flexible printed circuit board with a spatial resolution of 18.5 mm. The force range of a tactel is 6 N and its sensitivity is 0.6 V/N. The array is read at a rate of 78 frames per second. Finally, two simple application examples are also carried out with the sensor mounted on the forearm of a rescue robot that communicates with the sensor through a CAN bus.

Nuevo brazo maestro para la resección transuretral mediante robot / New master arm for robotic assisted transurethral resection

OBJETIVO: En 1998 diseñamos un robot para la resección transuretral que constaba de un brazo maestro y un brazo esclavo que reproducía los movimientos del primero. En este artículo describimos el nuevo brazo maestro que hemos creado, que ha pasado de ser un mero "joystick" para convertirse en un auténtico elemento de trabajo. MÉTODOS: El brazo maestro se ha elaborado a partir de un elemento de trabajo convencional, integrado en una estructura articulada de metal y con sensores angulares en cada una de ellas. Cada sensor, a través del computador, ordena al brazo esclavo la posición y movimiento del resector. De esta manera podemos definir el espacio de trabajo del resector, que viene determinado por los cuatro movimientos básicos; desplazamiento horizontal, vertical, de penetración y rotación, más un quinto, el de corte del asa. Mediante programas informáticos, se ha dotado a este elemento de trabajo de un sistema capaz de eliminar los posibles errores del cirujano, e impedir movimientos fuera de campo de trabajo. RESULTADO: Se ha obtenido una herramienta de morfología y funcionamiento similar al resector conven cional, de manejo sencillo y que facilita el entrenamiento en la utilización del elemento de trabajo. Al tiempo que se beneficia de la tecnología punta, reduciendo el número de errores, ya sea por mala utilización como por cansancio o inexperiencia del cirujano. CONCLUSIONES: La nueva herramienta tiene un manejo idéntico al instrumento convencional, que permite que esta nueva tecnología no cambie los procedimientos habituales del cirujano durante la intervención. Además si se produce un uso inadecuado, el robot lo detecta e incluso se anticipa y actúa evitando que el error transcienda al paciente (AU)

[New master arm for transurethral resection with a robot]. / Nuevo brazo maestro para la resección transuretral mediante robot.

OBJECTIVE: In 1998, we designed a robot for transurethral resection that consisted of a master arm and a slave arm that reproduced the movements of the master. We describe the innovative master arm developed in two university departments that supersedes the mere "joystick" concept to become a true, operative surgical instrument. METHODS: The new master arm was developed from a conventional resectoscope working element integrated into a robotic manipulative device and fitted with angular displacement sensors. Each sensor, part of a computer-controlled feed back system commands the position and movements of the slave arm which, in turn, move the resectoscope. The surgeon freely chooses and manages precisely within a three-dimensional surgical field four basic resectoscope maneuvers; horizontal and vertical displacement, penetration-withdrawal, and rotation; he also controls the movements and operation of the resectoscope cutting loop. Additional computer-programs provide the surgeon with an error-free, fail-safe surgical instrument that denies any movements outside the planned surgical field. RESULTS: The fact that the new easily used instrument is similar in shape and works like conventional resectoscopes greatly helps familiarization and training. Advanced technology reduces the number of errors usually associated with fatigue or inexperience of the surgeon. CONCLUSIONS: The new surgical instrument handles just like conventional instruments so surgeons do not need to change their usual operating procedures. In addition, an early-warning program detects instantly and denies inappropriate maneuvers. This facility confers peace of mind to the surgeon and ensures patient satisfaction.