Feasibility Study: Towards a Robot-Assisted Gait Training in Ophthalmological Rehabilitation.

The idea of using mobile assistance robots for gait training in rehabilitation has been increasingly explored in recent years due to the associated benefits. This paper describes how the previous results of research and praxis on gait training with a mobile assistance robot in orthopedic rehabilitation can be transferred to ophthalmic-related orientation and mobility training for blind and visually impaired people. To this end, the specific requirements for such orientation and mobility training are presented from a therapeutic perspective. Using sensory data, it is investigated how the analysis of training errors can be automated and transferred back to the training person. These pre-examinations are the prerequisite for any form of robot-assisted mobile gait training in ophthamological rehabilitation, which does not exist so far and which is expected to be of great benefit to these patients.

OHO: A Multi-Modal, Multi-Purpose Dataset for Human-Robot Object Hand-Over.

In the context of collaborative robotics, handing over hand-held objects to a robot is a safety-critical task. Therefore, a robust distinction between human hands and presented objects in image data is essential to avoid contact with robotic grippers. To be able to develop machine learning methods for solving this problem, we created the OHO (Object Hand-Over) dataset of tools and other everyday objects being held by human hands. Our dataset consists of color, depth, and thermal images with the addition of pose and shape information about the objects in a real-world scenario. Although the focus of this paper is on instance segmentation, our dataset also enables training for different tasks such as 3D pose estimation or shape estimation of objects. For the instance segmentation task, we present a pipeline for automated label generation in point clouds, as well as image data. Through baseline experiments, we show that these labels are suitable for training an instance segmentation to distinguish hands from objects on a per-pixel basis. Moreover, we present qualitative results for applying our trained model in a real-world application.

Laser-Based Door Localization for Autonomous Mobile Service Robots.

For autonomous mobile service robots, closed doors that are in their way are restricting obstacles. In order to open doors with on-board manipulation skills, a robot needs to be able to localize the door's key features, such as the hinge and handle, as well as the current opening angle. While there are vision-based approaches for detecting doors and handles in images, we concentrate on analyzing 2D laser range scans. This requires less computational effort, and laser-scan sensors are available on most mobile robot platforms. Therefore, we developed three different machine learning approaches and a heuristic method based on line fitting able to extract the required position data. The algorithms are compared with respect to localization accuracy with help of a dataset containing laser range scans of doors. Our LaserDoors dataset is publicly available for academic use. Pros and cons of the individual methods are discussed; basically, the machine learning methods could outperform the heuristic method, but require special training data when applied in a real application.

Few-Shot Object Detection: A Comprehensive Survey.

Humans are able to learn to recognize new objects even from a few examples. In contrast, training deep-learning-based object detectors requires huge amounts of annotated data. To avoid the need to acquire and annotate these huge amounts of data, few-shot object detection (FSOD) aims to learn from few object instances of new categories in the target domain. In this survey, we provide an overview of the state of the art in FSOD. We categorize approaches according to their training scheme and architectural layout. For each type of approach, we describe the general realization as well as concepts to improve the performance on novel categories. Whenever appropriate, we give short takeaways regarding these concepts in order to highlight the best ideas. Eventually, we introduce commonly used datasets and their evaluation protocols and analyze the reported benchmark results. As a result, we emphasize common challenges in evaluation and identify the most promising current trends in this emerging field of FSOD.

Digital media in intergenerational communication: Status quo and future scenarios for the grandparent-grandchild relationship.

Communication technologies play an important role in maintaining the grandparent-grandchild (GP-GC) relationship. Based on Media Richness Theory, this study investigates the frequency of use (RQ1) and perceived quality (RQ2) of established media as well as the potential use of selected innovative media (RQ3) in GP-GC relationships with a particular focus on digital media. A cross-sectional online survey and vignette experiment were conducted in February 2021 among N = 286 university students in Germany (mean age 23 years, 57% female) who reported on the direct and mediated communication with their grandparents. In addition to face-to-face interactions, non-digital and digital established media (such as telephone, texting, video conferencing) and innovative digital media, namely augmented reality (AR)-based and social robot-based communication technologies, were covered. Face-to-face and phone communication occurred most frequently in GP-GC relationships: 85% of participants reported them taking place at least a few times per year (RQ1). Non-digital established media were associated with higher perceived communication quality than digital established media (RQ2). Innovative digital media received less favorable quality evaluations than established media. Participants expressed doubts regarding the technology competence of their grandparents, but still met innovative media with high expectations regarding improved communication quality (RQ3). Richer media, such as video conferencing or AR, do not automatically lead to better perceived communication quality, while leaner media, such as letters or text messages, can provide rich communication experiences. More research is needed to fully understand and systematically improve the utility, usability, and joy of use of different digital communication technologies employed in GP-GC relationships.

Can Communication Technologies Reduce Loneliness and Social Isolation in Older People? A Scoping Review of Reviews.

BACKGROUND: Loneliness and social isolation in older age are considered major public health concerns and research on technology-based solutions is growing rapidly. This scoping review of reviews aims to summarize the communication technologies (CTs) (review question RQ1), theoretical frameworks (RQ2), study designs (RQ3), and positive effects of technology use (RQ4) present in the research field. METHODS: A comprehensive multi-disciplinary, multi-database literature search was conducted. Identified reviews were analyzed according to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) framework. A total of N = 28 research reviews that cover 248 primary studies spanning 50 years were included. RESULTS: The majority of the included reviews addressed general internet and computer use (82% each) (RQ1). Of the 28 reviews, only one (4%) worked with a theoretical framework (RQ2) and 26 (93%) covered primary studies with quantitative-experimental designs (RQ3). The positive effects of technology use were shown in 55% of the outcome measures for loneliness and 44% of the outcome measures for social isolation (RQ4). CONCLUSION: While research reviews show that CTs can reduce loneliness and social isolation in older people, causal evidence is limited and insights on innovative technologies such as augmented reality systems are scarce.

Robot-Assisted Gait Self-Training: Assessing the Level Achieved.

This paper presents the technological status of robot-assisted gait self-training under real clinical environment conditions. A successful rehabilitation after surgery in hip endoprosthetics comprises self-training of the lessons taught by physiotherapists. While doing this, immediate feedback to the patient about deviations from the expected physiological gait pattern during training is important. Hence, the Socially Assistive Robot (SAR) developed for this type of training employs task-specific, user-centered navigation and autonomous, real-time gait feature classification techniques to enrich the self-training through companionship and timely corrective feedback. The evaluation of the system took place during user tests in a hospital from the point of view of technical benchmarking, considering the therapists' and patients' point of view with regard to training motivation and from the point of view of initial findings on medical efficacy as a prerequisite from an economic perspective. In this paper, the following research questions were primarily considered: Does the level of technology achieved enable autonomous use in everyday clinical practice? Has the gait pattern of patients who used additional robot-assisted gait self-training for several days been changed or improved compared to patients without this training? How does the use of a SAR-based self-training robot affect the motivation of the patients?

Point Cloud Hand-Object Segmentation Using Multimodal Imaging with Thermal and Color Data for Safe Robotic Object Handover.

This paper presents an application of neural networks operating on multimodal 3D data (3D point cloud, RGB, thermal) to effectively and precisely segment human hands and objects held in hand to realize a safe human-robot object handover. We discuss the problems encountered in building a multimodal sensor system, while the focus is on the calibration and alignment of a set of cameras including RGB, thermal, and NIR cameras. We propose the use of a copper-plastic chessboard calibration target with an internal active light source (near-infrared and visible light). By brief heating, the calibration target could be simultaneously and legibly captured by all cameras. Based on the multimodal dataset captured by our sensor system, PointNet, PointNet++, and RandLA-Net are utilized to verify the effectiveness of applying multimodal point cloud data for hand-object segmentation. These networks were trained on various data modes (XYZ, XYZ-T, XYZ-RGB, and XYZ-RGB-T). The experimental results show a significant improvement in the segmentation performance of XYZ-RGB-T (mean Intersection over Union: 82.8% by RandLA-Net) compared with the other three modes (77.3% by XYZ-RGB, 35.7% by XYZ-T, 35.7% by XYZ), in which it is worth mentioning that the Intersection over Union for the single class of hand achieves 92.6%.

Mobile Robot-Based Gait Training after Total Hip Arthroplasty (THA) Improves Walking in Biomechanical Gait Analysis.

There are multiple attempts to decrease costs in the healthcare system while maintaining a high treatment quality. Digital therapies receive increasing attention in clinical practice, mainly relating to home-based exercises supported by mobile devices, eventually in combination with wearable sensors. The aim of this study was to determine if patients following total hip arthroplasty (THA) could benefit from gait training on crutches conducted by a mobile robot in a clinical setting. METHOD: This clinical trial was conducted with 30 patients following total hip arthroplasty. Fifteen patients received the conventional physiotherapy program in the clinic (including 5 min of gait training supported by a physiotherapist). The intervention group of 15 patients passed the same standard physiotherapy program, but the 5-min gait training supported by a physiotherapist was replaced by 2 × 5 min of gait training conducted by the robot. Length of stay of the patients was set to five days. Biomechanical gait parameters of the patients were assessed pre-surgery and upon patient discharge. RESULTS: While before surgery no significant difference in gait parameters was existent, patients from the intervention group showed a significant higher absolute walking speed (0.83 vs. 0.65 m/s, p = 0.029), higher relative walking speed (0.2 vs. 0.16 m/s, p = 0.043) or shorter relative cycle time (3.35 vs. 3.68 s, p = 0.041) than the patients from the control group. CONCLUSION: The significant higher walking speed of patients indicates that such robot-based gait training on crutches may shorten length of stay (LOS) in acute clinics. However, the number of patients involved was rather small, thus calling for further studies.

A Multi-Modal Person Perception Framework for Socially Interactive Mobile Service Robots.

In order to meet the increasing demands of mobile service robot applications, a dedicated perception module is an essential requirement for the interaction with users in real-world scenarios. In particular, multi sensor fusion and human re-identification are recognized as active research fronts. Through this paper we contribute to the topic and present a modular detection and tracking system that models position and additional properties of persons in the surroundings of a mobile robot. The proposed system introduces a probability-based data association method that besides the position can incorporate face and color-based appearance features in order to realize a re-identification of persons when tracking gets interrupted. The system combines the results of various state-of-the-art image-based detection systems for person recognition, person identification and attribute estimation. This allows a stable estimate of a mobile robot's user, even in complex, cluttered environments with long-lasting occlusions. In our benchmark, we introduce a new measure for tracking consistency and show the improvements when face and appearance-based re-identification are combined. The tracking system was applied in a real world application with a mobile rehabilitation assistant robot in a public hospital. The estimated states of persons are used for the user-centered navigation behaviors, e.g., guiding or approaching a person, but also for realizing a socially acceptable navigation in public environments.

May I Keep an Eye on Your Training? Gait Assessment Assisted by a Mobile Robot.

A successful rehabilitation after surgery in hip endoprosthetics comprises self-training of the lessons taught by physiotherapists. While doing this, immediate feedback to the patient about deviations from physiological gait patterns during training is important. Such immediate feedback also concerns the correct usage of forearm crutches in three-point gait. In the project ROGER, a mobile Socially Assistive Robot (SAR) to support patients after surgery in hip endoprosthetics is going to be developed. The current implementation status of the robotic application developed for the use in a real-world scenario is presented below.

Robotic Companions for Older People: A Case Study in the Wild.

Older people tend to have difficulties using unknown technical devices and are less willing to accept technical shortcomings. Therefore, a robot that is supposed to support older people in managing daily life has to adapt to the users' needs and capabilities that are very heterogeneous within the target group. The aim of the presented case study was to provide in-depth insights on individual usage patterns and acceptance of a mobile service robot in real live environments (i.e. in the users' homes). Results from three cases (users aged 67, 78 and 85 living in their own apartments) are reported. Findings on usability and user experience illustrate that the robot has considerable potential to be accepted to support daily living at home.