Deep Neural Network-Based Visual Feedback System for Nasopharyngeal Swab Sampling.

During the 2019 coronavirus disease pandemic, robotic-based systems for swab sampling were developed to reduce burdens on healthcare workers and their risk of infection. Teleoperated sampling systems are especially appreciated as they fundamentally prevent contact with suspected COVID-19 patients. However, the limited field of view of the installed cameras prevents the operator from recognizing the position and deformation of the swab inserted into the nasal cavity, which highly decreases the operating performance. To overcome this limitation, this study proposes a visual feedback system that monitors and reconstructs the shape of an NP swab using augmented reality (AR). The sampling device contained three load cells and measured the interaction force applied to the swab, while the shape information was captured using a motion-tracking program. These datasets were used to train a one-dimensional convolution neural network (1DCNN) model, which estimated the coordinates of three feature points of the swab in 2D X-Y plane. Based on these points, the virtual shape of the swab, reflecting the curvature of the actual one, was reconstructed and overlaid on the visual display. The accuracy of the 1DCNN model was evaluated on a 2D plane under ten different bending conditions. The results demonstrate that the x-values of the predicted points show errors of under 0.590 mm from P0, while those of P1 and P2 show a biased error of about -1.5 mm with constant standard deviations. For the y-values, the error of all feature points under positive bending is uniformly estimated with under 1 mm of difference, when the error under negative bending increases depending on the amount of deformation. Finally, experiments using a collaborative robot validate its ability to visualize the actual swab's position and deformation on the camera image of 2D and 3D phantoms.

Machine-learning-based coordination of powered ankle-foot orthosis and functional electrical stimulation for gait control.

This study proposes a novel gait rehabilitation method that uses a hybrid system comprising a powered ankle-foot orthosis (PAFO) and FES, and presents its coordination control. The developed system provides assistance to the ankle joint in accordance with the degree of volitional participation of patients with post-stroke hemiplegia. The PAFO adopts the desired joint angle and impedance profile obtained from biomechanical simulation. The FES patterns of the tibialis anterior and soleus muscles are derived from predetermined electromyogram patterns of healthy individuals during gait and personalized stimulation parameters. The CNN-based estimation model predicts the volitional joint torque from the electromyogram of the patient, which is used to coordinate the contributions of the PAFO and FES. The effectiveness of the developed hybrid system was tested on healthy individuals during treadmill walking with and without considering the volitional muscle activity of the individual. The results showed that consideration of the volitional muscle activity significantly lowers the energy consumption by the PAFO and FES while providing adaptively assisted ankle motion depending on the volitional muscle activities of the individual. The proposed system has potential use as an assist-as-needed rehabilitation system, where it can improve the outcome of gait rehabilitation by inducing active patient participation depending on the stage of rehabilitation.

Standing Balance Control of a Bipedal Robot Based on Behavior Cloning.

Bipedal robots have gained increasing attention for their human-like mobility which allows them to work in various human-scale environments. However, their inherent instability makes it difficult to control their balance while they are physically interacting with the environment. This study proposes a novel balance controller for bipedal robots based on a behavior cloning model as one of the machine learning techniques. The behavior cloning model employs two deep neural networks (DNNs) trained on human-operated balancing data, so that the trained model can predict the desired wrench required to maintain the balance of the bipedal robot. Based on the prediction of the desired wrench, the joint torques for both legs are calculated using robot dynamics. The performance of the developed balance controller was validated with a bipedal lower-body robotic system through simulation and experimental tests by providing random perturbations in the frontal plane. The developed balance controller demonstrated superior performance with respect to resistance to balance loss compared to the conventional balance control method, while generating a smoother balancing movement for the robot.

Development of a Novel Robotic Rehabilitation System With Muscle-to-Muscle Interface.

In this study, we developed a novel robotic system with a muscle-to-muscle interface to enhance rehabilitation of post-stroke patients. The developed robotic rehabilitation system was designed to provide patients with stage appropriate physical rehabilitation exercise and muscular stimulation. Unlike the position-based control of conventional bimanual robotic therapies, the developed system stimulates the activities of the target muscles, as well as the joint movements of the paretic limb. The robot-assisted motion and the electrical stimulation on the muscles of the paretic side are controlled by on-line comparison of the motion and the muscle activities between the paretic and unaffected sides. With the developed system, the rehabilitation exercise can be customized and modulated depending on the patient's stage of motor recovery after stroke. The system can be operated in three different modes allowing both passive and active exercises. The effectiveness of the developed system was verified with healthy human subjects, where the subjects were paired to serve as the unaffected side and the paretic side of a hemiplegic patient.

Development of Gait Rehabilitation System Capable of Assisting Pelvic Movement of Normal Walking.

Gait rehabilitation training with robotic exoskeleton is drawing attention as a method for more advanced gait rehabilitation training. However, most of the rehabilitation robots are mainly focused on locomotion training in the sagittal plane. This study introduces a novel gait rehabilitation system with actuated pelvic motion to generate natural gait motion. The rehabilitation robot developed in this study, COWALK, is a lower-body exoskeleton system with 15 degrees of freedom (DoFs). The COWALK can generate multi-DoF pelvic movement along with leg movements. To produce natural gait patterns, the actuation of pelvic movement is essential. In the COWALK, the pelvic movement mechanism is designed to help hemiplegic patients regain gait balance during gait training. To verify the effectiveness of the developed system, the gait patterns with and without pelvic movement were compared to the normal gait on a treadmill. The experimental results show that the active control of pelvic movement combined with the active control of leg movement can make the gait pattern much more natural.