Immersive participatory design of assistive robots to support older adults.

Assistive robots have the potential to support independence, enhance safety, and lower healthcare costs for older adults, as well as alleviate the demands of their care partners. However, ensuring that these robots will effectively and reliably address end-user needs in the long term requires user-specific design factors to be considered during the robot development process. To identify these design factors, we embedded Stretch, a mobile manipulator created by Hello Robot Inc., in the home of an older adult with motor impairments and his care partner for four weeks to support them with everyday activities. An occupational therapist and a robotics engineer lived with them during this period, employing an immersive participatory design approach to co-design and customise the robot with them. We highlight the benefits of this immersive participatory design experience and provide insights into robot design that can be applied broadly to other assistive technologies.

BodyPressure - Inferring Body Pose and Contact Pressure From a Depth Image.

Contact pressure between the human body and its surroundings has important implications. For example, it plays a role in comfort, safety, posture, and health. We present a method that infers contact pressure between a human body and a mattress from a depth image. Specifically, we focus on using a depth image from a downward facing camera to infer pressure on a body at rest in bed occluded by bedding, which is directly applicable to the prevention of pressure injuries in healthcare. Our approach involves augmenting a real dataset with synthetic data generated via a soft-body physics simulation of a human body, a mattress, a pressure sensing mat, and a blanket. We introduce a novel deep network that we trained on an augmented dataset and evaluated with real data. The network contains an embedded human body mesh model and uses a white-box model of depth and pressure image generation. Our network successfully infers body pose, outperforming prior work. It also infers contact pressure across a 3D mesh model of the human body, which is a novel capability, and does so in the presence of occlusion from blankets.

The Design of Stretch: A Compact, Lightweight Mobile Manipulator for Indoor Human Environments.

Mobile manipulators for indoor human environments can serve as versatile devices that perform a variety of tasks, yet adoption of this technology has been limited. Reducing size, weight, and cost could facilitate adoption, but risks restricting capabilities. We present a novel design that reduces size, weight, and cost, while supporting a variety of tasks. The core design consists of a two-wheeled differential-drive mobile base, a lift, and a telescoping arm configured to achieve Cartesian motion at the end of the arm. Design extensions include a 1 degree-of-freedom (DOF) wrist to stow a tool, a 2-DOF dexterous wrist to pitch and roll a tool, and a compliant gripper. We justify our design with anthropometry and mathematical models of static stability. We also provide empirical support from teleoperating and autonomously controlling a commercial robot based on our design (the Stretch RE1 from Hello Robot Inc.) to perform tasks in real homes.

Material Recognition via Heat Transfer Given Ambiguous Initial Conditions.

Humans and robots can recognize materials with distinct thermal effusivities by making physical contact and observing temperatures during heat transfer. This works well with room temperature materials, yet research has shown that contact with distinct materials can result in similar temperatures and confusion when one material is heated or cooled. To thoroughly investigate this form of ambiguity, we designed a psychophysical experiment in which a participant discriminates between two materials given initial conditions that result in similar temperatures (i.e., ambiguous initial conditions). In this article, we conducted a study with 32 human participants and a robot. Humans and the robot confused the materials. We also found that robots can overcome this ambiguity using two temperature sensors with different temperatures prior to contact. We support this conclusion based on a mathematical proof using a heat transfer model and empirical results in which a robot achieved 100% accuracy compared to 5% human accuracy. Our results also indicate that robots with a single temperature sensor can use subtle cues to outperform humans. Overall, our work provides insights into challenging conditions for material recognition via heat transfer, and suggests methods by which robots can overcome these challenges.

A system for bedside assistance that integrates a robotic bed and a mobile manipulator.

Various situations, such as injuries or long-term disabilities, can result in people receiving physical assistance while in bed. We present a robotic system for bedside assistance that consists of a robotic bed and a mobile manipulator (i.e., a wheeled robot with arms) that work together to provide better assistance. Many assistive tasks depend on moving with respect to the person's body, and the complementary physical and perceptual capabilities of the two robots help with respect to this general goal. The system provides autonomy for common tasks, as well as an interface for direct teleoperation of the two robots. Autonomy handles coarse motions of the robots by estimating the person's pose using a pressure sensing mat and then moving the robots to configurations optimized for the task. After completing these motions, the user is given fine control of the robots to complete the task. In an evaluation using a medical mannequin, we found that the robotic bed's motion and perception each improved the assistive robotic system's performance. The system achieved 100% success over 9 trials involving 3 tasks. Using the system with the bed movement or the body pose estimation capabilities turned off resulted in success in only 33% or 78% of the trials, respectively. We also evaluated our system with Henry Evans, a person with severe quadriplegia, in his home. In a formal test, Henry successfully used the bedside-assistance system to perform 3 different tasks, 5 times each, without any failures. Henry's feedback on the system was positive regarding usefulness and ease of use, and he noted benefits of using our system over fully manual teleoperation. Overall, our results suggest that a robotic bed and a mobile manipulator can work collaboratively to provide effective personal assistance and that the combination of the two robots is beneficial.

Multidimensional Capacitive Sensing for Robot-Assisted Dressing and Bathing.

Robotic assistance presents an opportunity to benefit the lives of many people with physical disabilities, yet accurately sensing the human body and tracking human motion remain difficult for robots. We present a multidimensional capacitive sensing technique that estimates the local pose of a human limb in real time. A key benefit of this sensing method is that it can sense the limb through opaque materials, including fabrics and wet cloth. Our method uses a multielectrode capacitive sensor mounted to a robot's end effector. A neural network model estimates the position of the closest point on a person's limb and the orientation of the limb's central axis relative to the sensor's frame of reference. These pose estimates enable the robot to move its end effector with respect to the limb using feedback control. We demonstrate that a PR2 robot can use this approach with a custom six electrode capacitive sensor to assist with two activities of daily living- dressing and bathing. The robot pulled the sleeve of a hospital gown onto able-bodied participants' right arms, while tracking human motion. When assisting with bathing, the robot moved a soft wet washcloth to follow the contours of able-bodied participants' limbs, cleaning their surfaces. Overall, we found that multidimensional capacitive sensing presents a promising approach for robots to sense and track the human body during assistive tasks that require physical human-robot interaction.

In-home and remote use of robotic body surrogates by people with profound motor deficits.

By controlling robots comparable to the human body, people with profound motor deficits could potentially perform a variety of physical tasks for themselves, improving their quality of life. The extent to which this is achievable has been unclear due to the lack of suitable interfaces by which to control robotic body surrogates and a dearth of studies involving substantial numbers of people with profound motor deficits. We developed a novel, web-based augmented reality interface that enables people with profound motor deficits to remotely control a PR2 mobile manipulator from Willow Garage, which is a human-scale, wheeled robot with two arms. We then conducted two studies to investigate the use of robotic body surrogates. In the first study, 15 novice users with profound motor deficits from across the United States controlled a PR2 in Atlanta, GA to perform a modified Action Research Arm Test (ARAT) and a simulated self-care task. Participants achieved clinically meaningful improvements on the ARAT and 12 of 15 participants (80%) successfully completed the simulated self-care task. Participants agreed that the robotic system was easy to use, was useful, and would provide a meaningful improvement in their lives. In the second study, one expert user with profound motor deficits had free use of a PR2 in his home for seven days. He performed a variety of self-care and household tasks, and also used the robot in novel ways. Taking both studies together, our results suggest that people with profound motor deficits can improve their quality of life using robotic body surrogates, and that they can gain benefit with only low-level robot autonomy and without invasive interfaces. However, methods to reduce the rate of errors and increase operational speed merit further investigation.

Understanding healthcare providers' perceptions of a personal assistant robot.

To successfully deploy a robot into a healthcare setting, it must be accepted by the end users. This study explored healthcare providers' perceptions of a mobile manipulator class personal robot assisting with caregiving tasks for older adult patients. Participants were 14 healthcare providers with an average of 12 years of continuous work experience with older patients. Quantitative and qualitative methods were used. Participants indicated a willingness to use a mobile manipulator robot as an assistant, yet they expressed discretion in their acceptance for different tasks. Benefits of robot assistance noted by participants included saving time, being accurate when conducting medical tasks, and enabling them to be more productive. Participants expressed concern about robots being unreliable, hazardous to patients, and inappropriate for performing some tasks (e.g., those that involve close patient contact). These findings provide insights into healthcare providers' attitudes and preferences for assistance from a mobile manipulator robot.

Older adults' acceptance of a robot for partner dance-based exercise.

Partner dance has been shown to be beneficial for the health of older adults. Robots could potentially facilitate healthy aging by engaging older adults in partner dance-based exercise. However, partner dance involves physical contact between the dancers, and older adults would need to be accepting of partner dancing with a robot. Using methods from the technology acceptance literature, we conducted a study with 16 healthy older adults to investigate their acceptance of robots for partner dance-based exercise. Participants successfully led a human-scale wheeled robot with arms (i.e., a mobile manipulator) in a simple, which we refer to as the Partnered Stepping Task (PST). Participants led the robot by maintaining physical contact and applying forces to the robot's end effectors. According to questionnaires, participants were generally accepting of the robot for partner dance-based exercise, tending to perceive it as useful, easy to use, and enjoyable. Participants tended to perceive the robot as easier to use after performing the PST with it. Through a qualitative data analysis of structured interview data, we also identified facilitators and barriers to acceptance of robots for partner dance-based exercise. Throughout the study, our robot used admittance control to successfully dance with older adults, demonstrating the feasibility of this method. Overall, our results suggest that robots could successfully engage older adults in partner dance-based exercise.

Small forces that differ with prior motor experience can communicate movement goals during human-human physical interaction.

BACKGROUND: Physical interactions between two people are ubiquitous in our daily lives, and an integral part of many forms of rehabilitation. However, few studies have investigated forces arising from physical interactions between humans during a cooperative motor task, particularly during overground movements. As such, the direction and magnitude of interaction forces between two human partners, how those forces are used to communicate movement goals, and whether they change with motor experience remains unknown. A better understanding of how cooperative physical interactions are achieved in healthy individuals of different skill levels is a first step toward understanding principles of physical interactions that could be applied to robotic devices for motor assistance and rehabilitation. METHODS: Interaction forces between expert and novice partner dancers were recorded while performing a forward-backward partnered stepping task with assigned "leader" and "follower" roles. Their position was recorded using motion capture. The magnitude and direction of the interaction forces were analyzed and compared across groups (i.e. expert-expert, expert-novice, and novice-novice) and across movement phases (i.e. forward, backward, change of direction). RESULTS: All dyads were able to perform the partnered stepping task with some level of proficiency. Relatively small interaction forces (10-30N) were observed across all dyads, but were significantly larger among expert-expert dyads. Interaction forces were also found to be significantly different across movement phases. However, interaction force magnitude did not change as whole-body synchronization between partners improved across trials. CONCLUSIONS: Relatively small interaction forces may communicate movement goals (i.e. "what to do and when to do it") between human partners during cooperative physical interactions. Moreover, these small interactions forces vary with prior motor experience, and may act primarily as guiding cues that convey information about movement goals rather than providing physical assistance. This suggests that robots may be able to provide meaningful physical interactions for rehabilitation using relatively small force levels.

Older Users' Acceptance of an Assistive Robot: Attitudinal Changes Following Brief Exposure.

INTRODUCTION: Many older adults wish to age-in-place. Robot assistance at home may be beneficial for older adults who are experiencing limitations in performing home activities. In this study we investigate older Americans' robot acceptance before and after exposure to a domestic mobile manipulator, with an emphasis on understanding trialability (i.e., "trying out" a robot for a short time period) and result demonstrability (i.e., observing the results of the robot's functionality). METHOD: Older adult participants observed a mobile manipulator robot autonomously demonstrating three tasks: delivering medication, learning to turn off a light switch, and organizing home objects. We administered pre and post exposure questionnaires about participants' opinions and attitudes toward the robot, as well as a semi-structured interview about each demonstration. RESULTS: We found that demonstration of a mobile manipulator assistive robot did, in fact, influence older adults' acceptance. There was a significant increase, pre vs. post, in positive perceptions of robot usefulness and ease of use for 8 of the 12 Robot Opinions Questionnaire items. Furthermore, in the Assistance Preference Checklist, eighteen tasks significantly differed between pre and post exposure, with older adults showing a greater openness to robot assistance after exposure to the robot. CONCLUSION: Thus, demonstration of robot capability positively affected older adults' preferences for robot assistance for tasks in the home. Interview data suggest that the robot's capability and reliability influenced older adults' first impressions of the robot.

Evaluation by Expert Dancers of a Robot That Performs Partnered Stepping via Haptic Interaction.

Our long-term goal is to enable a robot to engage in partner dance for use in rehabilitation therapy, assessment, diagnosis, and scientific investigations of two-person whole-body motor coordination. Partner dance has been shown to improve balance and gait in people with Parkinson's disease and in older adults, which motivates our work. During partner dance, dance couples rely heavily on haptic interaction to convey motor intent such as speed and direction. In this paper, we investigate the potential for a wheeled mobile robot with a human-like upper-body to perform partnered stepping with people based on the forces applied to its end effectors. Blindfolded expert dancers (N=10) performed a forward/backward walking step to a recorded drum beat while holding the robot's end effectors. We varied the admittance gain of the robot's mobile base controller and the stiffness of the robot's arms. The robot followed the participants with low lag (M=224, SD=194 ms) across all trials. High admittance gain and high arm stiffness conditions resulted in significantly improved performance with respect to subjective and objective measures. Biomechanical measures such as the human hand to human sternum distance, center-of-mass of leader to center-of-mass of follower (CoM-CoM) distance, and interaction forces correlated with the expert dancers' subjective ratings of their interactions with the robot, which were internally consistent (Cronbach's α=0.92). In response to a final questionnaire, 1/10 expert dancers strongly agreed, 5/10 agreed, and 1/10 disagreed with the statement "The robot was a good follower." 2/10 strongly agreed, 3/10 agreed, and 2/10 disagreed with the statement "The robot was fun to dance with." The remaining participants were neutral with respect to these two questions.

Domestic Robots for Older Adults: Attitudes, Preferences, and Potential.

The population of older adults in America is expected to reach an unprecedented level in the near future. Some of them have difficulties with performing daily tasks and caregivers may not be able to match pace with the increasing need for assistance. Robots, especially mobile manipulators, have the potential for assisting older adults with daily tasks enabling them to live independently in their homes. However, little is known about their views of robot assistance in the home. Twenty-one independently living older Americans (65-93 years old) were asked about their preferences for and attitudes toward robot assistance via a structured group interview and questionnaires. In the group interview, they generated a diverse set of 121 tasks they would want a robot to assist them with in their homes. These data, along with their questionnaire responses, suggest that the older adults were generally open to robot assistance but were discriminating in their acceptance of assistance for different tasks. They preferred robot assistance over human assistance for tasks related to chores, manipulating objects, and information management. In contrast, they preferred human assistance to robot assistance for tasks related to personal care and leisure activities. Our study provides insights into older adults' attitudes and preferences for robot assistance with everyday living tasks in the home which may inform the design of robots that will be more likely accepted by older adults.

Identifying the Potential for Robotics to Assist Older Adults in Different Living Environments.

As the older adult population grows and becomes more diverse, so will their needs and preferences for living environments. Many adults over 65 years of age require some assistance [1, 2]; yet it is important for their feelings of well-being that the assistance not restrict their autonomy [3]. Not only is autonomy correlated with quality of life [4], autonomy enhancement may improve functionality [2, 5]. The goal of this paper is to provide guidance for the development of technology to enhance autonomy and quality of life for older adults. We explore the potential for robotics to meet these needs. We evaluated older adults' diverse living situations and the predictors of residential moves to higher levels of care in the United States. We also examined older adults' needs for assistance with activities of daily living (ADLs), instrumental activities of daily living (IADLs), and medical conditions when living independently or in a long-term care residence. By providing support for older adults, mobile manipulator robots may reduce need-driven, undesired moves from residences with lower levels of care (i.e., private homes, assisted living) to those with higher levels of care (i.e., skilled nursing).

Whole-arm tactile sensing for beneficial and acceptable contact during robotic assistance.

Many assistive tasks involve manipulation near the care-receiver's body, including self-care tasks such as dressing, feeding, and personal hygiene. A robot can provide assistance with these tasks by moving its end effector to poses near the care-receiver's body. However, perceiving and maneuvering around the care-receiver's body can be challenging due to a variety of issues, including convoluted geometry, compliant materials, body motion, hidden surfaces, and the object upon which the body is resting (e.g., a wheelchair or bed). Using geometric simulations, we first show that an assistive robot can achieve a much larger percentage of end-effector poses near the care-receiver's body if its arm is allowed to make contact. Second, we present a novel system with a custom controller and whole-arm tactile sensor array that enables a Willow Garage PR2 to regulate contact forces across its entire arm while moving its end effector to a commanded pose. We then describe tests with two people with motor impairments, one of whom used the system to grasp and pull a blanket over himself and to grab a cloth and wipe his face, all while in bed at his home. Finally, we describe a study with eight able-bodied users in which they used the system to place objects near their bodies. On average, users perceived the system to be safe and comfortable, even though substantial contact occurred between the robot's arm and the user's body.

Older Adults' Medication Management in the Home: How can Robots Help?

Successful management of medications is critical to maintaining healthy and independent living for older adults. However, medication non-adherence is a common problem with a high risk for severe consequences [5], which can jeopardize older adults' chances to age in place [1]. Well-designed robots assisting with medication management tasks could support older adults' independence. Design of successful robots will be enhanced through understanding concerns, attitudes, and preferences for medication assistance tasks. We assessed older adults' reactions to medication hand-off from a mobile manipulator robot with 12 participants (68-79 yrs). We identified factors that affected their attitudes toward a mobile manipulator for supporting general medication management tasks in the home. The older adults were open to robot assistance; however, their preferences varied depending on the nature of the medication management task. For instance, they preferred a robot (over a human) to remind them to take medications, but preferred human assistance for deciding what medication to take and for administering the medication. Factors such as perceptions of one's own capability and robot reliability influenced their attitudes.

Investigating Healthcare Providers' Acceptance of Personal Robots for Assisting with Daily Caregiving Tasks.

Robots have potential to provide assistance to healthcare providers in daily caregiving tasks. The healthcare providers' acceptance of assistive robots will mediate the success or failure of implementation of robotic systems in care settings. It is essential to understand why and how providers would accept implementation of a robot in their daily work routines. We identified caregiving tasks with which healthcare providers would or would not accept assistance from a personal robot (Willow Garage's PR2). We also explored preferences for human or robot assistance. The healthcare providers we interviewed were quite open to the idea of receiving robot assistance for certain tasks.

Dusty: an assistive mobile manipulator that retrieves dropped objects for people with motor impairments.

People with physical disabilities have ranked object retrieval as a high-priority task for assistive robots. We have developed Dusty, a teleoperated mobile manipulator that fetches objects from the floor and delivers them to users at a comfortable height. In this paper, we first demonstrate the robot's high success rate (98.4%) when autonomously grasping 25 objects considered being important by people with amyotrophic lateral sclerosis (ALS). We tested the robot with each object in five different configurations on five types of flooring. We then present the results of an experiment in which 20 people with ALS operated Dusty. Participants teleoperated Dusty to move around an obstacle, pick up an object and deliver the object to themselves. They successfully completed this task in 59 out of 60 trials (3 trials each) with a mean completion time of 61.4 SD = 20.5 seconds), and reported high overall satisfaction using Dusty (7-point Likert scale; 6.8 SD = 0.6). Participants rated Dusty to be significantly easier to use than their own hands, asking family members, and using mechanical reachers (p < 0.03, paired t-tests). Fourteen of the 20 participants reported that they would prefer using Dusty over their current methods. [Box: see text].

The Domesticated Robot: Design Guidelines for Assisting Older Adults to Age in Place.

Many older adults wish to remain in their own homes as they age [16]. However, challenges in performing home upkeep tasks threaten an older adult's ability to age in place. Even healthy independently living older adults experience challenges in maintaining their home [13]. Challenges with home tasks can be compensated through technology, such as home robots. However, for home robots to be adopted by older adult users, they must be designed to meet older adults' needs for assistance and the older users must be amenable to robot assistance for those needs. We conducted a needs assessment to (1) assess older adults' openness to assistance from robots; and (2) understand older adults' opinions about using an assistive robot to help around the home. We administered questionnaires and conducted structured group interviews with 21 independently living older adults (ages 65-93). The questionnaire data suggest that older adults prefer robot assistance for cleaning and fetching/organizing tasks overall. However their assistance preferences discriminated between tasks. The interview data provided insight as to why they hold such preferences. Older adults reported benefits of robot assistance (e.g., the robot compensating for limitations, saving them time and effort, completing undesirable tasks, and performing tasks at a high level of performance). Participants also reported concerns such as the robot damaging the environment, being unreliable at or incapable of doing a task, doing tasks the older adult would rather do, or taking up too much space/storage. These data, along with specific comments from participant interviews, provide the basis for preliminary recommendations for designing mobile manipulator robots to support aging in place.

"Commanding Your Robot" Older Adults' Preferences for Methods of Robot Control.

Home robots have the potential to assist older adults in maintaining their independence. However, robots deployed in older adults' homes will be required to interact with untrained, novice users. The way untrained users, such as older adults, provide commands or control the robot (i.e., "method of robot control") will likely impact the ease of use and adoption of the robot. The current study explored older adults' preferences for controlling robots. Twelve independently-living older adults (ages 68-79) observed a functioning personal robot in a home setting, and were interviewed about their opinions regarding specific methods of robot control (i.e., laser pointer, physical manipulation, and devices). The older adults perceived advantages and disadvantages of these specific methods, including 'specificity in command', 'accurate robot performance', 'limitations in their own physical capability', and 'challenges in using control device.' The older adults also completed a questionnaire measuring their willingness to use 10 different types of methods of robot control. These data revealed that older adults were willing to use a variety of methods. Although older adults were limited in their spontaneous ideas about robot control (i.e., limited to voice command), once exposed to other options they were willing and open to a variety of control methods.