Increasing the Robustness of Deep Learning Models for Object Segmentation: A Framework for Blending Automatically Annotated Real and Synthetic Data.

Recent problems in robotics can sometimes only be tackled using machine learning technologies, particularly those that utilize deep learning (DL) with transfer learning. Transfer learning takes advantage of pretrained models, which are later fine-tuned using smaller task-specific datasets. The fine-tuned models must be robust against changes in environmental factors such as illumination since, often, there is no guarantee for them to be constant. Although synthetic data for pretraining has been shown to enhance DL model generalization, there is limited research on its application for fine-tuning. One limiting factor is that the generation and annotation of synthetic datasets can be cumbersome and impractical for the purpose of fine-tuning. To address this issue, we propose two methods for automatically generating annotated image datasets for object segmentation, one for real-world and another for synthetic images. We also introduce a novel domain adaptation approach called filling the reality gap (FTRG), which can blend elements from real-world and synthetic scenes in a single image to achieve domain adaptation. We demonstrate through experimentation on a representative robot application that FTRG outperforms other domain adaptation techniques, such as domain randomization or photorealistic synthetic images, in creating robust models. Furthermore, we evaluate the benefits of using synthetic data for fine-tuning in transfer learning and continual learning with experience replay using our proposed methods and FTRG. Our findings indicate that fine-tuning with synthetic data can produce superior results compared to solely using real-world data.

Computationally Relaxed Unscented Kalman Filter.

Advanced robotics and autonomous vehicles rely on filtering and sensor fusion techniques to a large extent. These mobile applications need to handle the computations onboard at high rates while the computing capacities are limited. Therefore, any improvement that lowers the CPU time of the filtering leads to more accurate control or longer battery operation. This article introduces a generic computational relaxation for the unscented transformation (UT) that is the key operation of the Unscented Kalman filter-based applications. The central idea behind the relaxation is to pull out the linear part of the filtering model and avoid the calculations for the kernel of the nonlinear part. The practical merit of the proposed relaxation is demonstrated through a simultaneous localization and mapping (SLAM) implementation that underpins the superior performance of the algorithm in the practically relevant cases, where the nonlinear dependencies influence only an affine subspace of the image space. The numerical examples show that the computational demand can be mitigated below 50% without decreasing the accuracy of the approximation. The method described in this article is implemented and published as an open-source C++ library RelaxedUnscentedTransformation on GitHub.

On the philosophical, cognitive and mathematical foundations of symbiotic autonomous systems.

Symbiotic autonomous systems (SAS) are advanced intelligent and cognitive systems that exhibit autonomous collective intelligence enabled by coherent symbiosis of human-machine interactions in hybrid societies. Basic research in the emerging field of SAS has triggered advanced general-AI technologies that either function without human intervention or synergize humans and intelligent machines in coherent cognitive systems. This work presents a theoretical framework of SAS underpinned by the latest advances in intelligence, cognition, computer, and system sciences. SAS are characterized by the composition of autonomous and symbiotic systems that adopt bio-brain-social-inspired and heterogeneously synergized structures and autonomous behaviours. This paper explores the cognitive and mathematical foundations of SAS. The challenges to seamless human-machine interactions in a hybrid environment are addressed. SAS-based collective intelligence is explored in order to augment human capability by autonomous machine intelligence towards the next generation of general AI, cognitive computers, and trustworthy mission-critical intelligent systems. Emerging paradigms and engineering applications of SAS are elaborated via autonomous knowledge learning systems that symbiotically work between humans and cognitive robots. This article is part of the theme issue 'Towards symbiotic autonomous systems'.

Asynchronous Sampled-Data Filtering Design for Fuzzy-Affine-Model-Based Stochastic Nonlinear Systems.

This article studies the asynchronous sampled-data filtering design problem for Itô stochastic nonlinear systems via Takagi-Sugeno fuzzy-affine models. The sample-and-hold behavior of the measurement output is described by an input delay method. Based on a novel piecewise quadratic Lyapunov-Krasovskii functional, some new results on the asynchronous sampled-data filtering design are proposed through a linearization procedure by using some convexification techniques. Simulation studies are given to illustrate the effectiveness of the proposed method.

Command Filter-Based Adaptive NN Control for MIMO Nonlinear Systems With Full-State Constraints and Actuator Hysteresis.

This article studies the issue of adaptive neural network (NN) control for strict-feedback multi-input and multioutput (MIMO) nonlinear systems with full-state constraints and actuator hysteresis. Radial basis function NNs (RBFNNs) are introduced to approximate unknown nonlinear functions. The command filter is adopted to solve the issue of "explosion of complexity." By applying a one-to-one nonlinear mapping, the strict-feedback system with full-state constraints is converted into a new pure-feedback system without state constraints, and a novel NN control method is proposed. The stability of the closed-loop system is proved via the Lyapunov stability theory, and the tracking errors converge to small residual sets. The simulation results are given to confirm the validity of the proposed method.

New MEMS Tweezers for the Viscoelastic Characterization of Soft Materials at the Microscale.

As many studies show, there is a relation between the tissue's mechanical characteristics and some specific diseases. Knowing this relationship would help early diagnosis or microsurgery. In this paper, a new method for measuring the viscoelastic properties of soft materials at the microscale is proposed. This approach is based on the adoption of a microsystem whose mechanical structure can be reduced to a compliant four bar linkage where the connecting rod is substituted by the tissue sample. A procedure to identify both stiffness and damping coefficients of the tissue is then applied to the developed hardware. Particularly, stiffness is calculated solving the static equations of the mechanism in a desired configuration, while the damping coefficient is inferred from the dynamic equations, which are written under the hypothesis that the sample tissue is excited by a variable compression force characterized by a suitable wave form. The whole procedure is implemented by making use of a control system.

Surface deformation and reaction force estimation of liver tissue based on a novel nonlinear mass-spring-damper viscoelastic model.

Rheological soft tissue models play an important role in designing control methods for modern teleoperation systems. In the meanwhile, these models are also essential for creating a realistic virtual environment for surgical training. The implementation of model-based control in teleoperation has been a frequently discussed topic in the past decades, offering solutions for the loss of stability caused by time delay, which is one of the major issues in long-distance force control. In this paper, mass-spring-damper soft tissue models are investigated, showing that the widely used linear models do not represent realistic behavior under surgical manipulations. A novel, nonlinear model is proposed, where mechanical parameters are estimated using curve fitting methods. Theoretical reaction force curves are estimated using the proposed model, and the results are verified using measurement results from uniaxial indentation. The model is extended with force estimation by nonuniform surface deformation, where the surface deformation function is approximated according to visual data. Results show that using the proposed nonlinear model, a good estimation of reaction force can be achieved within the range of 0-4 mm, provided that the tissue deformation shape function is appropriately approximated.