Towards multimodal graph neural networks for surgical instrument anticipation.

PURPOSE: Decision support systems and context-aware assistance in the operating room have emerged as the key clinical applications supporting surgeons in their daily work and are generally based on single modalities. The model- and knowledge-based integration of multimodal data as a basis for decision support systems that can dynamically adapt to the surgical workflow has not yet been established. Therefore, we propose a knowledge-enhanced method for fusing multimodal data for anticipation tasks. METHODS: We developed a holistic, multimodal graph-based approach combining imaging and non-imaging information in a knowledge graph representing the intraoperative scene of a surgery. Node and edge features of the knowledge graph are extracted from suitable data sources in the operating room using machine learning. A spatiotemporal graph neural network architecture subsequently allows for interpretation of relational and temporal patterns within the knowledge graph. We apply our approach to the downstream task of instrument anticipation while presenting a suitable modeling and evaluation strategy for this task. RESULTS: Our approach achieves an F1 score of 66.86% in terms of instrument anticipation, allowing for a seamless surgical workflow and adding a valuable impact for surgical decision support systems. A resting recall of 63.33% indicates the non-prematurity of the anticipations. CONCLUSION: This work shows how multimodal data can be combined with the topological properties of an operating room in a graph-based approach. Our multimodal graph architecture serves as a basis for context-sensitive decision support systems in laparoscopic surgery considering a comprehensive intraoperative operating scene.

Revitalizing Convolutional Network for Image Restoration.

Image restoration aims to reconstruct a high-quality image from its corrupted version, playing essential roles in many scenarios. Recent years have witnessed a paradigm shift in image restoration from convolutional neural networks (CNNs) to Transformerbased models due to their powerful ability to model long-range pixel interactions. In this paper, we explore the potential of CNNs for image restoration and show that the proposed simple convolutional network architecture, termed ConvIR, can perform on par with or better than the Transformer counterparts. By re-examing the characteristics of advanced image restoration algorithms, we discover several key factors leading to the performance improvement of restoration models. This motivates us to develop a novel network for image restoration based on cheap convolution operators. Comprehensive experiments demonstrate that our ConvIR delivers state-ofthe- art performance with low computation complexity among 20 benchmark datasets on five representative image restoration tasks, including image dehazing, image motion/defocus deblurring, image deraining, and image desnowing.

Grid codes vs. multi-scale, multi-field place codes for space.

Introduction: Recent work on bats flying over long distances has revealed that single hippocampal cells have multiple place fields of different sizes. At the network level, a multi-scale, multi-field place cell code outperforms classical single-scale, single-field place codes, yet the performance boundaries of such a code remain an open question. In particular, it is unknown how general multi-field codes compare to a highly regular grid code, in which cells form distinct modules with different scales. Methods: In this work, we address the coding properties of theoretical spatial coding models with rigorous analyses of comprehensive simulations. Starting from a multi-scale, multi-field network, we performed evolutionary optimization. The resulting multi-field networks sometimes retained the multi-scale property at the single-cell level but most often converged to a single scale, with all place fields in a given cell having the same size. We compared the results against a single-scale single-field code and a one-dimensional grid code, focusing on two main characteristics: the performance of the code itself and the dynamics of the network generating it. Results: Our simulation experiments revealed that, under normal conditions, a regular grid code outperforms all other codes with respect to decoding accuracy, achieving a given precision with fewer neurons and fields. In contrast, multi-field codes are more robust against noise and lesions, such as random drop-out of neurons, given that the significantly higher number of fields provides redundancy. Contrary to our expectations, the network dynamics of all models, from the original multi-scale models before optimization to the multi-field models that resulted from optimization, did not maintain activity bumps at their original locations when a position-specific external input was removed. Discussion: Optimized multi-field codes appear to strike a compromise between a place code and a grid code that reflects a trade-off between accurate positional encoding and robustness. Surprisingly, the recurrent neural network models we implemented and optimized for either multi- or single-scale, multi-field codes did not intrinsically produce a persistent "memory" of attractor states. These models, therefore, were not continuous attractor networks.

Context-Based Meta-Reinforcement Learning with Bayesian Nonparametric Models.

Deep reinforcement learning agents usually need to collect a large number of interactions to solve a single task. In contrast, meta-reinforcement learning (meta-RL) aims to quickly adapt to new tasks using a small amount of experience by leveraging the knowledge from training on a set of similar tasks. State-of-the-art context-based meta-RL algorithms use the context to encode the task information and train a policy conditioned on the inferred latent task encoding. However, most recent works are limited to parametric tasks, where a handful of variables control the full variation in the task distribution, and also failed to work in non-stationary environments due to the few-shot adaptation setting. To address those limitations, we propose MEta-reinforcement Learning with Task Self-discovery (MELTS), which adaptively learns qualitatively different nonparametric tasks and adapts to new tasks in a zero-shot manner. We introduce a novel deep clustering framework (DPMM-VAE) based on an infinite mixture of Gaussians, which combines the Dirichlet process mixture model (DPMM) and the variational autoencoder (VAE), to simultaneously learn task representations and cluster the tasks in a self-adaptive way. Integrating DPMM-VAE into MELTS enables it to adaptively discover the multi-modal structure of the nonparametric task distribution, which previous methods using isotropic Gaussian random variables cannot model. In addition, we propose a zero-shot adaptation mechanism and a recurrence-based context encoding strategy to improve the data efficiency and make our algorithm applicable in non-stationary environments. On various continuous control tasks with both parametric and nonparametric variations, our algorithm produces a more structured and self-adaptive task latent space and also achieves superior sample efficiency and asymptotic performance compared with state-of-the-art meta-RL algorithms.

Magnetic Soft Microrobot Design for Cell Grasping and Transportation.

Manipulating cells at a small scale is widely acknowledged as a complex and challenging task, especially when it comes to cell grasping and transportation. Various precise methods have been developed to remotely control the movement of microrobots. However, the manipulation of micro-objects necessitates the use of end-effectors. This paper presents a study on the control of movement and grasping operations of a magnetic microrobot, utilizing only 3 pairs of electromagnetic coils. A specially designed microgripper is employed on the microrobot for efficient cell grasping and transportation. To ensure precise grasping, a bending deformation model of the microgripper is formulated and subsequently validated. To achieve precise and reliable transportation of cells to specific positions, an approach that combines an extended Kalman filter with a model predictive control method is adopted to accomplish the trajectory tracking task. Through experiments, we observe that by applying the proposed control strategy, the mean absolute error of path tracking is found to be less than 0.155 mm. Remarkably, this value accounts for only 1.55% of the microrobot's size, demonstrating the efficacy and accuracy of our control strategy. Furthermore, an experiment involving the grasping and transportation of a zebrafish embryonic cell (diameter: 800 µm) is successfully conducted. The results of this experiment not only validate the precision and effectiveness of the proposed microrobot and its associated models but also highlight its tremendous potential for cell manipulation in vitro and in vivo.

Toward coordinated planning and hierarchical optimization control for highly redundant mobile manipulator.

This paper represents a constraint planning and optimization control scheme for a highly redundant mobile manipulator considering a complex indoor environment. Compared with the traditional optimization solution of a redundant manipulator, infinity norm and slack variable are additionally introduced and leveraged by the optimization algorithm. The former takes into account the joint limits effectively by considering individual joint velocities and the latter relaxes the equality constraint by decreasing the infeasible solution area. By using derived kinematic equations, the tracking control problem is expressed as an optimization problem and converted into a new quadratic programming (QP) problem. To address the optimization problem, the two-timescale recurrent neural networks optimization scheme is proposed and tested with a 9 DOFs nonholonomic mobile-based manipulator. Additionally, the BI2RRT∗ path-planning algorithm incorporates path planning in the complex environment where different obstacles are positioned. To test and evaluate the proposed optimization scheme, both predefined and generated paths are tested in the Neurorobotics Platform (NRP) 2which is open access and open source integrative simulation framework powered by Gazebo and developed by our team.

Image Restoration via Frequency Selection.

Image restoration aims to reconstruct the latent sharp image from its corrupted counterpart. Besides dealing with this long-standing task in the spatial domain, a few approaches seek solutions in the frequency domain by considering the large discrepancy between spectra of sharp/degraded image pairs. However, these algorithms commonly utilize transformation tools, e.g., wavelet transform, to split features into several frequency parts, which is not flexible enough to select the most informative frequency component to recover. In this paper, we exploit a multi-branch and content-aware module to decompose features into separate frequency subbands dynamically and locally, and then accentuate the useful ones via channel-wise attention weights. In addition, to handle large-scale degradation blurs, we propose an extremely simple decoupling and modulation module to enlarge the receptive field via global and window-based average pooling. Furthermore, we merge the paradigm of multi-stage networks into a single U-shaped network to pursue multi-scale receptive fields and improve efficiency. Finally, integrating the above designs into a convolutional backbone, the proposed Frequency Selection Network (FSNet) performs favorably against state-of-the-art algorithms on 20 different benchmark datasets for 6 representative image restoration tasks, including single-image defocus deblurring, image dehazing, image motion deblurring, image desnowing, image deraining, and image denoising.

Hybrid Residual Multiexpert Reinforcement Learning for Spatial Scheduling of High-Density Parking Lots.

Industries, such as manufacturing, are accelerating their embrace of the metaverse to achieve higher productivity, especially in complex industrial scheduling. In view of the growing parking challenges in large cities, high-density vehicle spatial scheduling is one of the potential solutions. Stack-based parking lots utilize parking robots to densely park vehicles in the vertical stacks like container stacking, which greatly reduces the aisle area in the parking lot, but requires complex scheduling algorithms to park and take out the vehicles. The existing high-density parking (HDP) scheduling algorithms are mainly heuristic methods, which only contain simple logic and are difficult to utilize information effectively. We propose a hybrid residual multiexpert (HIRE) reinforcement learning (RL) approach, a method for interactive learning in the digital industrial metaverse, which efficiently solves the HDP batch space scheduling problem. In our proposed framework, each heuristic scheduling method is considered as an expert. The neural network trained by RL assigns the expert strategy according to the current parking lot state. Furthermore, to avoid being limited by heuristic expert performance, the proposed hierarchical network framework also sets up a residual output channel. Experiments show that our proposed algorithm outperforms various advanced heuristic methods and the end-to-end RL method in the number of vehicle maneuvers, and has good robustness to the parking lot size and the estimation accuracy of vehicle exit time. We believe that the proposed HIRE RL method can be effectively and conveniently applied to practical application scenarios, which can be regarded as a key step for RL to enter the practical application stage of the industrial metaverse.

Dual-domain strip attention for image restoration.

Image restoration aims to reconstruct a latent high-quality image from a degraded observation. Recently, the usage of Transformer has significantly advanced the state-of-the-art performance of various image restoration tasks due to its powerful ability to model long-range dependencies. However, the quadratic complexity of self-attention hinders practical applications. Moreover, sufficiently leveraging the huge spectral disparity between clean and degraded image pairs can also be conducive to image restoration. In this paper, we develop a dual-domain strip attention mechanism for image restoration by enhancing representation learning, which consists of spatial and frequency strip attention units. Specifically, the spatial strip attention unit harvests the contextual information for each pixel from its adjacent locations in the same row or column under the guidance of the learned weights via a simple convolutional branch. In addition, the frequency strip attention unit refines features in the spectral domain via frequency separation and modulation, which is implemented by simple pooling techniques. Furthermore, we apply different strip sizes for enhancing multi-scale learning, which is beneficial for handling degradations of various sizes. By employing the dual-domain attention units in different directions, each pixel can implicitly perceive information from an expanded region. Taken together, the proposed dual-domain strip attention network (DSANet) achieves state-of-the-art performance on 12 different datasets for four image restoration tasks, including image dehazing, image desnowing, image denoising, and image defocus deblurring. The code and models are available at https://github.com/c-yn/DSANet.

Lateral flexion of a compliant spine improves motor performance in a bioinspired mouse robot.

A flexible spine is critical to the motion capability of most animals and plays a pivotal role in their agility. Although state-of-the-art legged robots have already achieved very dynamic and agile movement solely relying on their legs, they still exhibit the type of stiff movement that compromises movement efficiency. The integration of a flexible spine thus appears to be a promising approach to improve their agility, especially for small and underactuated quadruped robots that are underpowered because of size limitations. Here, we show that the lateral flexion of a compliant spine can promote both walking speed and maneuver agility for a neurorobotic mouse (NeRmo). We present NeRmo as a biomimetic robotic mouse that mimics the morphology of biological mice and their muscle-tendon actuation system. First, by leveraging the lateral flexion of the compliant spine, NeRmo can greatly increase its static stability in an initially unstable configuration by adjusting its posture. Second, the lateral flexion of the spine can also effectively extend the stride length of a gait and therefore improve the walking speeds of NeRmo. Finally, NeRmo shows agile maneuvers that require both a small turning radius and fast walking speed with the help of the spine. These results advance our understanding of spine-based quadruped locomotion skills and highlight promising design concepts to develop more agile legged robots.

A human-centered safe robot reinforcement learning framework with interactive behaviors.

Deployment of Reinforcement Learning (RL) algorithms for robotics applications in the real world requires ensuring the safety of the robot and its environment. Safe Robot RL (SRRL) is a crucial step toward achieving human-robot coexistence. In this paper, we envision a human-centered SRRL framework consisting of three stages: safe exploration, safety value alignment, and safe collaboration. We examine the research gaps in these areas and propose to leverage interactive behaviors for SRRL. Interactive behaviors enable bi-directional information transfer between humans and robots, such as conversational robot ChatGPT. We argue that interactive behaviors need further attention from the SRRL community. We discuss four open challenges related to the robustness, efficiency, transparency, and adaptability of SRRL with interactive behaviors.

Embodied bidirectional simulation of a spiking cortico-basal ganglia-cerebellar-thalamic brain model and a mouse musculoskeletal body model distributed across computers including the supercomputer Fugaku.

Embodied simulation with a digital brain model and a realistic musculoskeletal body model provides a means to understand animal behavior and behavioral change. Such simulation can be too large and complex to conduct on a single computer, and so distributed simulation across multiple computers over the Internet is necessary. In this study, we report our joint effort on developing a spiking brain model and a mouse body model, connecting over the Internet, and conducting bidirectional simulation while synchronizing them. Specifically, the brain model consisted of multiple regions including secondary motor cortex, primary motor and somatosensory cortices, basal ganglia, cerebellum and thalamus, whereas the mouse body model, provided by the Neurorobotics Platform of the Human Brain Project, had a movable forelimb with three joints and six antagonistic muscles to act in a virtual environment. Those were simulated in a distributed manner across multiple computers including the supercomputer Fugaku, which is the flagship supercomputer in Japan, while communicating via Robot Operating System (ROS). To incorporate models written in C/C++ in the distributed simulation, we developed a C++ version of the rosbridge library from scratch, which has been released under an open source license. These results provide necessary tools for distributed embodied simulation, and demonstrate its possibility and usefulness toward understanding animal behavior and behavioral change.

The Bcm rule allows a spinal cord model to learn rhythmic movements.

Currently, it is accepted that animal locomotion is controlled by a central pattern generator in the spinal cord. Experiments and models show that rhythm generating neurons and genetically determined network properties could sustain oscillatory output activity suitable for locomotion. However, current central pattern generator models do not explain how a spinal cord circuitry, which has the same basic genetic plan across species, can adapt to control the different biomechanical properties and locomotion patterns existing in these species. Here we demonstrate that rhythmic and alternating movements in pendulum models can be learned by a monolayer spinal cord circuitry model using the Bienenstock-Cooper-Munro learning rule, which has been previously proposed to explain learning in the visual cortex. These results provide an alternative theory to central pattern generator models, because rhythm generating neurons and genetically defined connectivity are not required in our model. Though our results are not in contradiction to current models, as existing neural mechanism and structures, not used in our model, can be expected to facilitate the kind of learning demonstrated here. Therefore, our model could be used to augment existing models.

Meta-Reinforcement Learning in Nonstationary and Nonparametric Environments.

Recent state-of-the-art artificial agents lack the ability to adapt rapidly to new tasks, as they are trained exclusively for specific objectives and require massive amounts of interaction to learn new skills. Meta-reinforcement learning (meta-RL) addresses this challenge by leveraging knowledge learned from training tasks to perform well in previously unseen tasks. However, current meta-RL approaches limit themselves to narrow parametric and stationary task distributions, ignoring qualitative differences and nonstationary changes between tasks that occur in the real world. In this article, we introduce a Task-Inference-based meta-RL algorithm using explicitly parameterized Gaussian variational autoencoders (VAEs) and gated Recurrent units (TIGR), designed for nonparametric and nonstationary environments. We employ a generative model involving a VAE to capture the multimodality of the tasks. We decouple the policy training from the task-inference learning and efficiently train the inference mechanism on the basis of an unsupervised reconstruction objective. We establish a zero-shot adaptation procedure to enable the agent to adapt to nonstationary task changes. We provide a benchmark with qualitatively distinct tasks based on the half-cheetah environment and demonstrate the superior performance of TIGR compared with state-of-the-art meta-RL approaches in terms of sample efficiency (three to ten times faster), asymptotic performance, and applicability in nonparametric and nonstationary environments with zero-shot adaptation. Videos can be viewed at https://videoviewsite.wixsite.com/tigr.

Versatile end effector for laparoscopic robotic scrub nurse.

PURPOSE: Integrating robotic scrub nurses in the operating room has the potential to help overcome staff shortages and limited use of available operating capacities in hospitals. Existing approaches of robotic scrub nurses are mainly focused on open surgical procedures, neglecting laparoscopic procedures. Laparoscopic interventions offer great potential for the context-sensitive integration of robotic systems due to possible standardization. However, the first step is to ensure the safe manipulation of laparoscopic instruments. METHODS: A robotic platform with a universal gripper system was designed to pick up and place laparoscopic as well as da Vinci[Formula: see text] instruments in an efficient workflow. The robustness of the gripper system was studied using a test protocol, which included a force absorption test to determine the operational safety limits of the design and a grip test to determine the system performance. RESULTS: The test protocol shows results regarding force and torque absorption capabilities of the end effector, which are essential when transferring an instrument to the surgeon to enable a robust handover. The grip tests show that the laparoscopic instruments can be safely picked up, manipulated and returned independent of unexpected positional deviations. The gripper system also enables the manipulation of da Vinci[Formula: see text] instruments, opening the door for robot-robot interaction. CONCLUSION: Our evaluation tests have shown that our robotic scrub nurse with the universal gripper system can safely and robustly manipulate laparoscopic and da Vinci[Formula: see text] instruments. The system design will continue with the integration of context-sensitive capabilities.

Overtone Mass Sensitivity of Quartz Crystal Microbalance Sensor with Asymmetric N-M Type Electrode Configuration.

Sensing sensitivity is one of the crucial parameters for quartz crystal microbalance (QCM) sensors. Herein, we study the overtone mass sensitivity of a QCM sensor with an asymmetric N-M type electrode configuration. In order to overcome the deficiency that the sensitivity of the QCM sensor with an asymmetric electrode cannot be calculated by Sauerbrey's equation, we design the electrochemical electrodeposition experiments to measure it. The measurement results of overtone mass sensitivities of three 3.1-5.1 and three 4.1-5.1 QCMs are 5.418, 5.629, and 5.572 Hz/ng and 4.155, 4.456, and 3.982 Hz/ng in the third overtone mode and 9.208, 9.474, and 9.243 Hz/ng and 6.811, 7.604, and 6.588 Hz/ng in the fifth overtone mode, respectively. The overtone mass sensitivities of three 5.1-5.1 QCMs are 3.210, 3.439, and 3.540 Hz/ng in the third overtone mode and 5.396, 5.010, and 5.707 Hz/ng in the fifth overtone mode, respectively. These results show that the overtone mass sensitivity of the N-M type QCM is larger than that of QCMs with symmetric electrodes, and the fifth overtone mass sensitivity is higher than the third overtone mass sensitivity for the same type of QCM. The above results strongly confirm that the overtone mass sensitivity of a QCM sensor with an asymmetric N-M electrode structure significantly enhances its sensing performance, and it will greatly meet the demands for high precision measurement of QCM sensors in applications.

Robotic Manipulation in Dynamic Scenarios via Bounding-Box-Based Hindsight Goal Generation.

By relabeling past experience with heuristic or curriculum goals, state-of-the-art reinforcement learning (RL) algorithms such as hindsight experience replay (HER), hindsight goal generation (HGG), and graph-based HGG (G-HGG) have been able to solve challenging robotic manipulation tasks in multigoal settings with sparse rewards. HGG outperforms HER in challenging tasks in which goals are difficult to explore by learning from a curriculum, in which intermediate goals are selected based on the Euclidean distance to target goals. G-HGG enhances HGG by selecting intermediate goals from a precomputed graph representation of the environment, which enables its applicability in an environment with stationary obstacles. However, G-HGG is not applicable to manipulation tasks with dynamic obstacles, since its graph representation is only valid in static scenarios and fails to provide any correct information to guide the exploration. In this article, we propose bounding-box-based HGG (Bbox-HGG), an extension of G-HGG selecting hindsight goals with the help of image observations of the environment, which makes it applicable to tasks with dynamic obstacles. We evaluate Bbox-HGG on four challenging manipulation tasks, where significant enhancements in both sample efficiency and overall success rate are shown over state-of-the-art algorithms. The videos can be viewed at https://videoviewsite.wixsite.com/bbhgg.

Meta-Reinforcement Learning in Non-Stationary and Dynamic Environments.

In recent years, the subject of deep reinforcement learning (DRL) has developed very rapidly, and is now applied in various fields, such as decision making and control tasks. However, artificial agents trained with RL algorithms require great amounts of training data, unlike humans that are able to learn new skills from very few examples. The concept of meta-reinforcement learning (meta-RL) has been recently proposed to enable agents to learn similar but new skills from a small amount of experience by leveraging a set of tasks with a shared structure. Due to the task representation learning strategy with few-shot adaptation, most recent work is limited to narrow task distributions and stationary environments, where tasks do not change within episodes. In this work, we address those limitations and introduce a training strategy that is applicable to non-stationary environments, as well as a task representation based on Gaussian mixture models to model clustered task distributions. We evaluate our method on several continuous robotic control benchmarks. Compared with state-of-the-art literature that is only applicable to stationary environments with few-shot adaption, our algorithm first achieves competitive asymptotic performance and superior sample efficiency in stationary environments with zero-shot adaption. Second, our algorithm learns to perform successfully in non-stationary settings as well as a continual learning setting, while learning well-structured task representations. Last, our algorithm learns basic distinct behaviors and well-structured task representations in task distributions with multiple qualitatively distinct tasks.