Learning 3D shape proprioception for continuum soft robots with multiple magnetic sensors.

Sensing the shape of continuum soft robots without obstructing their movements and modifying their natural softness requires innovative solutions. This letter proposes to use magnetic sensors fully integrated into the robot to achieve proprioception. Magnetic sensors are compact, sensitive, and easy to integrate into a soft robot. We also propose a neural architecture to make sense of the highly nonlinear relationship between the perceived intensity of the magnetic field and the shape of the robot. By injecting a priori knowledge from the kinematic model, we obtain an effective yet data-efficient learning strategy. We first demonstrate in simulation the value of this kinematic prior by investigating the proprioception behavior when varying the sensor configuration, which does not require us to re-train the neural network. We validate our approach in experiments involving one soft segment containing a cylindrical magnet and three magnetoresistive sensors. During the experiments, we achieve mean relative errors of 4.5%.

Closed-Loop Control Through Self-Sensing of a Joule-Heated Twisted and Coiled Polymer Muscle.

The twisted and coiled polymer muscle (TCPM) has two major benefits: low weight and low cost. Therefore, this new type of actuator is increasingly used in robotic applications where these benefits are relevant. Closed-loop control of these muscles, however, requires additional sensors that add weight and cost, negating the muscles' intrinsic benefits. Self-sensing enables feedback without added sensors. In this article, we investigate the feasibility of using self-sensing in closed-loop control of a Joule-heated muscle. We use a hardware module that is capable of driving the muscle, and simultaneously providing sensor measurements based on inductance. A mathematical model relates the measurements to the deflection. In combination with a simple force model, we can estimate both deflection and force, and control either of them. For a muscle that operates within deflections of [10, 30] mm and forces of [0.32, 0.51] N, our self-sensing method exhibited a 95% confidence interval of 2.14 mm around a mean estimation error of -0.27 mm and 29.0 mN around a mean estimation error of 7.5 mN, for the estimation of, respectively, deflection and force. We conclude that self-sensing in closed-loop control of Joule-heated TCPMs is feasible and may facilitate further deployment of such actuators in applications where low cost and weight are critical.

A Big Data Analysis Approach for Rail Failure Risk Assessment.

Railway infrastructure monitoring is a vital task to ensure rail transportation safety. A rail failure could result in not only a considerable impact on train delays and maintenance costs, but also on safety of passengers. In this article, the aim is to assess the risk of a rail failure by analyzing a type of rail surface defect called squats that are detected automatically among the huge number of records from video cameras. We propose an image processing approach for automatic detection of squats, especially severe types that are prone to rail breaks. We measure the visual length of the squats and use them to model the failure risk. For the assessment of the rail failure risk, we estimate the probability of rail failure based on the growth of squats. Moreover, we perform severity and crack growth analyses to consider the impact of rail traffic loads on defects in three different growth scenarios. The failure risk estimations are provided for several samples of squats with different crack growth lengths on a busy rail track of the Dutch railway network. The results illustrate the practicality and efficiency of the proposed approach.

Railway Track Circuit Fault Diagnosis Using Recurrent Neural Networks.

Timely detection and identification of faults in railway track circuits are crucial for the safety and availability of railway networks. In this paper, the use of the long-short-term memory (LSTM) recurrent neural network is proposed to accomplish these tasks based on the commonly available measurement signals. By considering the signals from multiple track circuits in a geographic area, faults are diagnosed from their spatial and temporal dependences. A generative model is used to show that the LSTM network can learn these dependences directly from the data. The network correctly classifies 99.7% of the test input sequences, with no false positive fault detections. In addition, the t-Distributed Stochastic Neighbor Embedding (t-SNE) method is used to examine the resulting network, further showing that it has learned the relevant dependences in the data. Finally, we compare our LSTM network with a convolutional network trained on the same task. From this comparison, we conclude that the LSTM network architecture is better suited for the railway track circuit fault detection and identification tasks than the convolutional network.

Approximate analytical solutions to the double-stance dynamics of the lossy spring-loaded inverted pendulum.

This paper introduces approximate time-domain solutions to the otherwise non-integrable double-stance dynamics of the 'bipedal' spring-loaded inverted pendulum (B-SLIP) in the presence of non-negligible damping. We first introduce an auxiliary system whose behavior under certain conditions is approximately equivalent to the B-SLIP in double-stance. Then, we derive approximate solutions to the dynamics of the new system following two different methods: (i) updated-momentum approach that can deal with both the lossy and lossless B-SLIP models, and (ii) perturbation-based approach following which we only derive a solution to the lossless case. The prediction performance of each method is characterized via a comprehensive numerical analysis. The derived representations are computationally very efficient compared to numerical integrations, and, hence, are suitable for online planning, increasing the autonomy of walking robots. Two application examples of walking gait control are presented. The proposed solutions can serve as instrumental tools in various fields such as control in legged robotics and human motion understanding in biomechanics.

Learning Sequential Composition Control.

Sequential composition is an effective supervisory control method for addressing control problems in nonlinear dynamical systems. It executes a set of controllers sequentially to achieve a control specification that cannot be realized by a single controller. As these controllers are designed offline, sequential composition cannot address unmodeled situations that might occur during runtime. This paper proposes a learning approach to augment the standard sequential composition framework by using online learning to handle unforeseen situations. New controllers are acquired via learning and added to the existing supervisory control structure. In the proposed setting, learning experiments are restricted to take place within the domain of attraction (DOA) of the existing controllers. This guarantees that the learning process is safe (i.e., the closed loop system is always stable). In addition, the DOA of the new learned controller is approximated after each learning trial. This keeps the learning process short as learning is terminated as soon as the DOA of the learned controller is sufficiently large. The proposed approach has been implemented on two nonlinear systems: 1) a nonlinear mass-damper system and 2) an inverted pendulum. The results show that in both cases a new controller can be rapidly learned and added to the supervisory control structure.

Reinforcement learning for port-hamiltonian systems.

Passivity-based control (PBC) for port-Hamiltonian systems provides an intuitive way of achieving stabilization by rendering a system passive with respect to a desired storage function. However, in most instances the control law is obtained without any performance considerations and it has to be calculated by solving a complex partial differential equation (PDE). In order to address these issues we introduce a reinforcement learning (RL) approach into the energy-balancing passivity-based control (EB-PBC) method, which is a form of PBC in which the closed-loop energy is equal to the difference between the stored and supplied energies. We propose a technique to parameterize EB-PBC that preserves the systems's PDE matching conditions, does not require the specification of a global desired Hamiltonian, includes performance criteria, and is robust. The parameters of the control law are found by using actor-critic (AC) RL, enabling the search for near-optimal control policies satisfying a desired closed-loop energy landscape. The advantage is that the solutions learned can be interpreted in terms of energy shaping and damping injection, which makes it possible to numerically assess stability using passivity theory. From the RL perspective, our proposal allows for the class of port-Hamiltonian systems to be incorporated in the AC framework, speeding up the learning thanks to the resulting parameterization of the policy. The method has been successfully applied to the pendulum swing-up problem in simulations and real-life experiments.

Parametric Bayesian filters for nonlinear stochastic dynamical systems: a survey.

Nonlinear stochastic dynamical systems are commonly used to model physical processes. For linear and Gaussian systems, the Kalman filter is optimal in minimum mean squared error sense. However, for nonlinear or non-Gaussian systems, the estimation of states or parameters is a challenging problem. Furthermore, it is often required to process data online. Therefore, apart from being accurate, the feasible estimation algorithm also needs to be fast. In this paper, we review Bayesian filters that possess the aforementioned properties. Each filter is presented in an easy way to implement algorithmic form. We focus on parametric methods, among which we distinguish three types of filters: filters based on analytical approximations (extended Kalman filter, iterated extended Kalman filter), filters based on statistical approximations (unscented Kalman filter, central difference filter, Gauss-Hermite filter), and filters based on the Gaussian sum approximation (Gaussian sum filter). We discuss each of these filters, and compare them with illustrative examples.

Efficient model learning methods for actor-critic control.

We propose two new actor-critic algorithms for reinforcement learning. Both algorithms use local linear regression (LLR) to learn approximations of the functions involved. A crucial feature of the algorithms is that they also learn a process model, and this, in combination with LLR, provides an efficient policy update for faster learning. The first algorithm uses a novel model-based update rule for the actor parameters. The second algorithm does not use an explicit actor but learns a reference model which represents a desired behavior, from which desired control actions can be calculated using the inverse of the learned process model. The two novel methods and a standard actor-critic algorithm are applied to the pendulum swing-up problem, in which the novel methods achieve faster learning than the standard algorithm.

Cross-entropy optimization of control policies with adaptive basis functions.

This paper introduces an algorithm for direct search of control policies in continuous-state discrete-action Markov decision processes. The algorithm looks for the best closed-loop policy that can be represented using a given number of basis functions (BFs), where a discrete action is assigned to each BF. The type of the BFs and their number are specified in advance and determine the complexity of the representation. Considerable flexibility is achieved by optimizing the locations and shapes of the BFs, together with the action assignments. The optimization is carried out with the cross-entropy method and evaluates the policies by their empirical return from a representative set of initial states. The return for each representative state is estimated using Monte Carlo simulations. The resulting algorithm for cross-entropy policy search with adaptive BFs is extensively evaluated in problems with two to six state variables, for which it reliably obtains good policies with only a small number of BFs. In these experiments, cross-entropy policy search requires vastly fewer BFs than value-function techniques with equidistant BFs, and outperforms policy search with a competing optimization algorithm called DIRECT.

Model-based pH monitor for sensor assessment.

Owing to the nature of the treatment processes, monitoring the processes based on individual online measurements is difficult or even impossible. However, the measurements (online and laboratory) can be combined with a priori process knowledge, using mathematical models, to objectively monitor the treatment processes and measurement devices. The pH measurement is a commonly used measurement at different stages in the drinking water treatment plant, although it is a unreliable instrument, requiring significant maintenance. It is shown that, using a grey-box model, it is possible to assess the measurement devices effectively, even if detailed information of the specific processes is unknown.

Computer-controlled mechanical simulation of the artificially ventilated human respiratory system.

A mechanical lung simulator can be used to simulate specific lung pathologies, to test lung-function equipment, and in instruction. A new approach to mechanical simulation of lung behavior is introduced that uses a computer-controlled active mechatronic system. The main advantage of this approach is that the static and dynamic properties of the simulator can easily be adjusted via the control software. A nonlinear single-compartment mathematical model of the artificially ventilated respiratory system has been derived and incorporated into the simulator control system. This model can capture both the static and dynamic compliance of the respiratory system as well as nonlinear flow-resistance properties. Parameters in this model can be estimated by using data from artificially ventilated patients. It is shown that the simulation model fits patient data well. This mathematical model of the respiratory system was then matched to a model of the available physical equipment (the simulator, actuators, and the interface electronics) in order to obtain the desired lung behavior. A significant time delay in the piston motion control loop has been identified, which can potentially cause oscillations or even instability for high compliance values. Therefore, a feedback controller based on the Smith-predictor scheme was developed to control the piston motion. The control system, implemented on a personal computer, also includes a user-friendly interface to allow easy parameter setting.

Estimation of expiratory time constants via fuzzy clustering.

OBJECTIVE: In mechanically ventilated patients the expiratory time constant provides information about respiratory mechanics. In the present study a new method, fuzzy clustering, is proposed to determine expiratory time constants. Fuzzy clustering differs from other methods since it neither interferes with expiration nor presumes any functional relationship between the variables analysed. Furthermore, time constant behaviour during expiration can be assessed, instead of an average time constant. The time constants obtained with fuzzy clustering are compared to time constants conventionally calculated from the same expirations. METHODS: 20 mechanically ventilated patients, including 10 patients with COPD, were studied. The data of flow, volume and pressure were sampled. From these data, four local linear models were detected by fuzzy clustering. The time constants (tau) of the local linear models (clusters) were calculated by a least-squares technique. Time constant behaviour was analysed. Time constants obtained with fuzzy clustering were compared to time constants calculated from flow-volume curves using a conventional method. RESULTS: Fuzzy clustering revealed two patterns of expiratory time constant behaviour. In the patients with COPD an initial low time constant was found (mean tau1: 0.33 s, SD 0.21) followed by higher time constants; mean tau2: 2.00 s (SD 0.91s), mean tau3: 3.45 s (SD 1.44) and mean tau4: 5.47 s (SD 2.93). In the other patients only minor changes in time constants were found; mean tau1: 0.74 s (SD 0.30), mean tau2: 0.90 s (SD 0.23), mean tau3: 1.04 s (SD 0.42) and mean tau4: 1.74 s (SD 0.78). Both the pattern of expiratory time constants, as well as the time constants calculated from the separate clusters, were significantly different between the patients with and without COPD. Time constants obtained with fuzzy clustering for cluster 2, 3 and 4 correlated well with time constants obtained from the flow-volume curves. CONCLUSIONS: In mechanically ventilated patients, expiratory time constant behaviour can be accurately assessed by fuzzy clustering. A good correlation was found between time constants obtained with fuzzy clustering and time constants obtained by conventional analysis. On the basis of the time constants obtained with fuzzy clustering, a clear distinction was made between patients with and without

Clavulanic acid degradation in Streptomyces clavuligerus fed-batch cultivations.

Clavulanic acid (CA) is an important antibiotic that is produced by Streptomyces clavuligerus. CA is unstable and product degradation has turned out to have a major impact on product titers in fed-batch cultivations. Three different types of experiments have been used to elucidate CA degradation under fed-batch cultivation conditions. First, the influence of individual medium compounds was examined. Second, degradation was monitored during the exponential growth phase in batch cultivations. Third, CA degradation was studied in the supernatant of samples taken during a fed-batch. In addition, data from six fed-batch cultivations were studied to derive information about CA degradation during the production phase. These cultivations were based on a mineral medium, containing glycerol, glutamate, ammonium, and phosphate as the main nutrients. The ammonium concentration had a large influence on the degradation rate constant. In addition, either changes in the substrate availability or high concentrations of ammonium or glycerol cause a major increase in the degradation rate constant. Finally, a linear and a fuzzy logic model were made to predict CA degradation rates in these fed-batches.