Wearable multimode sensor with a seamless integrated structure for recognition of different joint motion states with the assistance of a deep learning algorithm.

Accurate motion feature extraction and recognition provide critical information for many scientific problems. Herein, a new paradigm for a wearable seamless multimode sensor with the ability to decouple pressure and strain stimuli and recognize the different joint motion states is reported. This wearable sensor is integrated into a unique seamless structure consisting of two main parts (a resistive component and a capacitive component) to decouple the different stimuli by an independent resistance-capacitance sensing mechanism. The sensor exhibits both high strain sensitivity (GF = 7.62, 0-140% strain) under the resistance mechanism and high linear pressure sensitivity (S = 3.4 kPa-1, 0-14 kPa) under the capacitive mechanism. The sensor can differentiate the motion characteristics of the positions and states of different joints with precise recognition (97.13%) with the assistance of machine learning algorithms. The unique integrated seamless structure is achieved by developing a layer-by-layer casting process that is suitable for large-scale manufacturing. The proposed wearable seamless multimode sensor and the convenient process are expected to contribute significantly to developing essential components in various emerging research fields, including soft robotics, electronic skin, health care, and innovative sports systems applications.

A Preliminary Study on Realizing Human-Robot Mental Comforting Dialogue via Sharing Experience Emotionally.

Mental health issues are receiving more and more attention in society. In this paper, we introduce a preliminary study on human-robot mental comforting conversation, to make an android robot (ERICA) present an understanding of the user's situation by sharing similar emotional experiences to enhance the perception of empathy. Specifically, we create the emotional speech for ERICA by using CycleGAN-based emotional voice conversion model, in which the pitch and spectrogram of the speech are converted according to the user's mental state. Then, we design dialogue scenarios for the user to talk about his/her predicament with ERICA. In the dialogue, ERICA shares other people's similar predicaments and adopts a low-spirit voice to express empathy to the interlocutor's situation. At the end of the dialogue, ERICA tries to encourage with a positive voice. Subsequently, questionnaire-based evaluation experiments were conducted with the recorded conversation. In the questionnaire, we use the Big Five scale to evaluate ERICA's personality. In addition, the perception of emotion, empathy, and encouragement in the dialogue are evaluated. The results show that the proposed emotional expression strategy helps the android robot better present low-spirit emotion, empathy, the personality of extroversion, while making the user better feel the encouragement.