Every "Body" Gets a Say: An Augmented Optimization Metric to Preserve Body Pose during Avatar Adaptation in Mixed/Augmented Reality.

User-Avatar interaction within augmented reality applications is rapidly increasing in frequency. Applications routinely place users in rooms with other, remote users embodied by photorealistic avatars, or require users to work with an avatar of a remote user to complete a task. During these types of interactions, it is often required to modify or redirect the posture of an avatar to achieve goals such as contact with or pointing at an object or maintaining eye gaze with the local user. A key limitation of modern redirection techniques is successfully preserving body posture, a critical component of nonverbal communication. This paper presents a new pose-preserving objective function to be used in the multi-objective optimization of an avatar's kinematic configuration. This objective function not only mimics the correct placement of body joints, but also preserves their orientation in space. We have tested this approach against several commonly used and current state-of-the-art redirection techniques and have found that our new approach achieves a significant reduction in targeted redirection error while simultaneously reducing body posture error. Additionally, human subject testing has shown that our new technique provides both a significantly more natural looking redirection and a significantly more realistic and believable overall body posture.

Feasibility of wearable devices and machine learning for sleep classification in children with Rett syndrome: A pilot study.

Sleep is vital to many processes involved in the well-being and health of children; however, it is estimated that 80% of children with Rett syndrome suffer from sleep disorders. Caregiver reports and questionnaires, which are the current method of studying sleep, are prone to observer bias and missed information. Polysomnography is considered the gold standard for sleep analysis but is labor and cost-intensive and limits the frequency of data collection for sleep disorder studies. Wearable digital health technologies, such as actigraphy devices, have shown potential and feasibility as a method for sleep analysis in Rett syndrome, but have not been validated against polysomnography. Furthermore, the collected accelerometer data has limitations due to the rigidity, periodic limb movement, and involuntary muscle contractions prevalent in Rett syndrome. Heart rate and electrodermal activity, along with other physiological signals, have been linked to sleep stages and can be utilized with machine learning to provide better resistance to noise and false positives than actigraphy. This research aims to address the gap in Rett syndrome sleep analysis by comparing the performance of a machine learning model utilizing both accelerometer data and physiological data features to the gold-standard polysomnography for sleep analysis in Rett syndrome. Our analytical validation pilot study (n = 7) found that using physiological and accelerometer features, our machine learning models can differentiate between awake, non-rapid eye movement sleep, and rapid eye movement sleep in Rett syndrome children with an accuracy of 85.1% when using an individual model. Additionally, this work demonstrates that it is feasible to use digital health technologies in Rett syndrome, even at a young age, without data loss or interference from repetitive movements that are characteristic of Rett syndrome.

SAR-Connect: A Socially Assistive Robotic System to Support Activity and Social Engagement of Older Adults.

Multi-domain activities that incorporate physical, cognitive, and social stimuli can enhance older adults' overall health and quality of life. Several robotic platforms have been developed to provide these therapies in a quantifiable manner to complement healthcare personnel in resource-strapped long-term care settings. However, these platforms are primarily limited to one-to-one human robot interaction (HRI) and thus do not enhance social interaction. In this paper, we present a novel HRI framework and a realized platform called SAR-Connect to foster robot-mediated social interaction among older adults through carefully designed tasks that also incorporate physical and cognitive stimuli. SAR-Connect seamlessly integrates a humanoid robot with a virtual reality-based activity platform and a multimodal data acquisition module including game interaction, audio, visual and electroencephalography responses of the participants. Results from a laboratory-based user study with older adults indicates the potential of SAR-Connect that showed this system could 1) involve one or multiple older adults to perform multi-domain activities and provide dynamic guidance, 2) engage them in the robot-mediated task and foster human-human interaction, and 3) quantify their social and activity engagement from multiple sensory modalities.

An Immersive Computer-Mediated Caregiver-Child Interaction System for Young Children With Autism Spectrum Disorder.

Autism Spectrum Disorder (ASD) affects 1 in 54 children in the United States. A core social communication skill negatively impacted by ASD is joint attention (JA), which influences the development of language, cognitive, and social skills from infancy onward. Although several technology-based JA studies have shown potential, they primarily focus on response to joint attention (RJA). The other important component of JA, the initiation of joint attention (IJA), has received less attention from a technology-based intervention perspective. In this work, we present an immersive Computer-mediated Caregiver-Child Interaction (C3I) system to help children with ASD practice IJA skills. C3I is a novel computerized intervention system that integrates a caregiver in the teaching loop, thereby preserving the advantages of both human and computer-administered intervention. A feasibility study with 6 dyads (caregiver-child with ASD) was conducted. A near significant increase with medium effect size on IJA performance was observed. Meanwhile, physiology-based stress analysis showed that C3I did not increase stress of the caregivers over the course of the study. To the best of our knowledge, this is the first autonomous system designed for teaching IJA skills to children with ASD incorporating caregivers within the loop to enhance the potential for generalization in real-world.

Field Testing of Ro-Tri, a Robot-Mediated Triadic Interaction for Older Adults.

Older adults residing in long term care (LTC) settings commonly experience apathy, a neuropsychiatric condition with adverse consequences of increased morbidity and mortality. Activities that combine social, physical and cognitive stimuli are most effective in engaging older adults with apathy but are time consuming and require significant staff resources. We present the results from an initial pilot field study of our socially assistive robotic (SAR) system, Ro-Tri, capable of multi-modal interventions to foster social interaction between pairs of older adults. Seven paired participants attended two sessions a week for three weeks. Sessions consisted of robot-mediated triadic interactions with three types of activities repeated once over the 3 weeks. Ro-Tri gathered quantitative interaction data, head pose, vocal sound, and physiological signals to automatically evaluate older adults' activity and social engagement. Ro-Tri functioned smoothly without any technical issues. Older adults had > 90% attendance and 100% completion rate and remained engaged with the system throughout the study duration. Participants' visual attention toward the SAR system and their partners increased 7.2% and 4.7%, respectively, with their interaction effort showing an increase of 2.9%. Older adults and LTC staff had positive perceptions with the system. These initial results demonstrate Ro-Tri's ability to engage older adults, encourage social human-to-human interaction, and assess the changes using quantitative metrics. Future studies will determine SAR's impact on apathy in LTC older adults.