Seeing through a robot's eyes: A cross-sectional exploratory study in developing a robotic screening technology for autism.

The present exploratory cross-sectional case-control study sought to develop a reliable and scalable screening tool for autism using a social robot. The robot HUMANE, installed with computer vision and linked with recognition technology, detected the direction of eye gaze of children. Children aged 3-8 (M = 5.52; N = 199) participated, 87 of whom had been confirmed with autism, 55 of whom were suspected to have autism, and 57 of whom were not considered to cause any concern for having autism. Before a session, a human experimenter instructed HUMANE to narrate a story to a child. HUMANE prompted the child to return his/her eye gaze to the robot if the child looked away, and praised the child when it re-established its eye gaze quickly after a prompt. The reliability of eye gaze detection was checked across all pairs of human raters and HUMANE and reached 0.90, indicating excellent interrater agreement. Using the pre-specified reference standard (Autism Spectrum Quotient), the sensitivity and specificity of the index tests (i.e., the number of robot prompts and duration of inattentiveness) reached 0.88 or above and the Diagnostic Odds Ratios were beyond 190. These results show that social robots may detect atypical eye patterns, suggesting a potential future for screening autism using social robots.

Examining Phenotypical Heterogeneity and its Underlying Factors in Gesture Skills of Chinese Autistic Children: Clustering Analysis.

PURPOSE: The heterogeneity of autism is well documented, but few studies have studied the heterogeneity of gesture production ability in autistic children. The present study aimed to identify subgroups of autistic children who displayed heterogeneous gesture production abilities and explore the underlying factors, including autism characteristics, intellectual ability, and language ability, that were associated with the heterogeneity. METHODS: A total of 65 Chinese autistic children (mean age = 5;3) participated. Their autism characteristics and intellectual ability were assessed by standardized measurements. Language output and gesture production were captured from a parent-child interaction task. RESULTS: We conducted a hierarchical cluster analysis and identified four distinct clusters. Cluster 1 and Cluster 2 both had low gesture production whereas Cluster 3 and Cluster 4 had high gesture production. Both Clusters 1 and 2 had relatively strong autism characteristics, in comparison to Clusters 3 and 4. CONCLUSIONS: Our findings revealed that children with stronger autism characteristics may gesture less often than those with weaker characteristics. However, the relationship between language ability and intellectual ability and gesture production was not clear. These findings shed light on the directions of intervention on gesture production for autistic children, especially those with stronger autism characteristics.

Comparing the effectiveness of robot-based to human-based intervention in improving joint attention in autistic children.

Background: Children with autism have impairments in initiation of joint attention (IJA) and response to joint attention (RJA). Aims: The present study compared the learning effectiveness of robot-based intervention (RBI) with that of content-matched human-based intervention (HBI) in improving joint attention (JA). We examined whether RBI would enhance RJA, in comparison to HBI. We also examined whether RBI would increase IJA, in comparison to HBI. Methods and procedures: Thirty-eight Chinese-speaking children with autism aged 6 to 9 years were randomly assigned to RBI and HBI groups. Before intervention, their autism severity, cognitive abilities, and language skills were assessed. Each child received six 30-min training sessions over 3 weeks. During training, he/she watched one or two robot/human dramas twice where two robot/human actors demonstrated eye contact and RJA. Outcomes and results: Children in the RBI (but not HBI) group produced more RJA and IJA behaviors in the delayed post-test than in the pre-test. Parents of the RBI children rated the program more positively than those of the HBI children. Conclusions and implications: RBI may be more effective than HBI in promoting JA in autistic children with high support needs. Our findings shed light on the application of robot dramas in enhancing social communication skills.

Robot dramas may improve joint attention of Chinese-speaking low-functioning children with autism: stepped wedge trials.

INTRODUCTION: Children with autism spectrum disorder (ASD), especially those with low cognitive functioning, have deficits in joint attention. Previous research has found that these children are interested in engaging with social robots. PURPOSE: In the present study, we designed a robot drama intervention for promoting responses to joint attention abilities (RJA) of children with low-functioning autism (IQs < 70). MATERIALS AND METHODS: Using a stepped wedge design, Chinese-speaking children aged six to eight were randomly assigned to three tiers (N = 18). Children of all three tiers had comparable autism severity, language and cognitive function, and joint attention abilities. Tier 1 first received intervention, followed by Tiers 2 and 3. They watched six dramas in which social robots demonstrated RJA behaviours. RESULTS: The RJA of children of all tiers improved after intervention and such improvement was maintained over time. Despite initiation of joint attention (IJA) not being explicitly taught, IJA of all children was found to improve after intervention. CONCLUSIONS: It was, therefore, concluded that a robot drama could enhance the joint attention of children with low-functioning ASD.Implications for rehabilitationIn comparison to typically developing children, children with autism spectrum disorder (ASD), especially those with low cognitive functioning, have deficits in joint attention.Robot-based drama intervention program was developed to promote responses to joint attention (RJA) abilities of children with low-functioning autism (IQs < 70).Modelling RJA in robot dramas can promote RJA abilities in these children.An improvement in RJA also yielded an increase in the initiation of joint attention (IJA) abilities.Children with low-functioning autism might be able to extract the initiation of joint attention skills from the drama, even though these behaviours were not explicitly taught.

Whose Gestures are More Predictive of Expressive Language Abilities among Chinese-Speaking Children with Autism? A Comparison of Caregivers' and Children's Gestures.

In spite of the close relationship between gestures and expressive language, little research has examined the roles of the parents' and children's gestures in the development of expressive language abilities in autistic children. Previous findings are also inconclusive. In the present study, we coded the gestures produced by the parents and their autistic children in parent-child interactions and compared the influence of their gestures on the children's expressive language abilities (N = 35; M = 4;10). Autistic children's deictic gestures positively predicted their Mean Length Utterance (MLU), word types, and word tokens whereas parents' deictic gesture inputs negatively predicted MLU and word types. The findings shed light on the importance of the gestures made by autistic children, which may trigger parents' gesture-to-word translation.

What affects gestural learning in children with and without Autism? The role of prior knowledge and imitation.

The present study examined whether prior knowledge to the learning target and imitation during learning affected learning outcomes in preschool children with autism spectrum disorder (ASD, N = 22) compared to their typically developing (TD, N = 15) peers. Children's gestural skills in recognizing and producing the target gestures before and after the training, as well as their imitative behavior during the training were coded. Results showed that consistent prior knowledge benefited gestural learning in both groups. Besides, only children with ASD were hindered by inconsistent prior knowledge. Notably, the effect of imitation was not significant in the ASD group. In conclusion, the learning process in children with ASD may differ from those with typical development, suggesting special-designed interventions are required.

Gesture Development in Chinese-Speaking Preschool Children With Autism and the Roles of Parental Input and Child-Based Factors.

PURPOSE: Children with autism are found to have delayed and heterogeneous gesture abilities. It is important to understand the growth of gesture abilities and the underlying factors affecting its growth. Addressing these issues can help to design effective intervention programs. METHOD: Thirty-five Chinese-speaking preschoolers with autism spectrum disorder (M age = 4.89 years, SD = 0.91; four girls) participated in four play sessions with their parents over 9 months. Their child-based factors including autism severity, intellectual functioning, and expressive language abilities were assessed. The gestures (deictic, iconic, and conventional) of the children and their parents were coded. Growth curve analyses were conducted to examine individual growth trajectories and the roles of child-based factors and parental input in shaping the children's gesture development. RESULTS: Child-based factors and parental input predicted gesture development differently. Parents' gestures positively predicted their children's gestures of the same type. Autism severity negatively predicted iconic and conventional gestures. Overall growth was found in deictic rather than iconic and conventional gestures. Subgroup variation was also found. Specifically, children with better expressive language ability showed a decrease in deictic gestures. An increase in iconic and conventional gestures was found in children with more severe autism and those with poorer expressive language ability and intellectual functioning, respectively. CONCLUSIONS: Different types of gestures may have different growth trajectories and be predicted by different child-based factors. Particular attention should be given to children who never produced iconic gestures, which is more challenging and may not develop over a short period, and hence require direct instruction.

Examining Phenotypical Heterogeneity in Language Abilities in Chinese-Speaking Children with Autism: A Naturalistic Sampling Approach.

Phenotypical heterogeneity in language abilities is a hallmark of autism but remains poorly understood. The present study collected naturalistic language samples from parent-child interactions. We quantified verbal abilities (mean length of utterance, tokens, types) of 50 Chinese-speaking children (M = 5; 6) and stratified subgroups based on their autism traits, IQ, and language abilities. Using hierarchical cluster analysis, four groups were identified. Group 1, the least affected group, had mild autism, the highest IQ, and the strongest verbal abilities. Group 2, the severely affected group, had the lowest IQ, most severe autism symptoms, and weakest verbal abilities. Group 3 and Group 4 displayed average levels of verbal abilities and IQ. These findings may characterize the heterogeneous profiles of verbal abilities in Chinese-speaking children.

Conversation Skills in Chinese-Speaking Preschoolers with Autism: The Contributing Role of Parents' Verbal Responsiveness.

Children with autism spectrum disorder (ASD) have conversation deficits, yet the growth of conversation abilities is understudied, especially in Chinese-speaking populations. Little is known about whether their parents' verbal responsiveness and redirectives are related to their conversation skills. Children with ASD (N = 37; M = 5;5) and their parents contributed their language samples. These children interacted with their parents at four time points over nine months. The number of conversational turns and the proportion of child-initiated conversation (but not the proportion of children's appropriate responses) grew over nine months. After controlling for time, autism severity, and language skills, parents' verbal responsiveness positively predicted children's appropriate responses. Parents' redirectives negatively predicted the proportion of children's appropriate responses and the number of conversational turns.

The influence of child-based factors and parental inputs on expressive language abilities in children with autism spectrum disorder.

LAY ABSTRACT: Language impairment is one of the early signs of Autism Spectrum Disorder (ASD) that alerts parents to take their children for early diagnosis and intervention. Little is known about how children's autism traits, IQ, initial language abilities and parental inputs influence their language abilities. In addition, only a few studies have compared the relative influence of these factors. The present study addressed these issues by examining the structural language in parent-child spontaneous interactions. Forty-two Cantonese (Chinese)-speaking autistic children aged four to eight were recruited. Their expressive language skills grew rapidly more than 9 months, but their development trajectories varied. Initial expressive language ability is the only significant predictor of child language outcomes and language growth trajectories. In contrast, nonverbal cognition, autism traits, and parents' input do not affect language outcomes in children with ASD. Therefore, early language intervention is crucial for autistic children at all severity and IQ levels.

Gestures in Storytelling by Preschool Chinese-Speaking Children With and Without Autism.

Previous findings on gestural impairment in autism are inconsistent, while scant evidence came from Chinese-speaking individuals. In the present study, preschool Chinese-speaking children with typical development and with autism were asked to generate stories from a set of wordless Cartoon pictures. Two groups were matched in chronological age and language developmental age. Their speech and gestures were coded. Compared to children with typical development, children with autism produced fewer gestures and showed lower gesture rate. Besides, children with autism produced fewer emblems and fewer supplementary gestures compared to their TD peers. Unlike children with typical development, children with autism tend to produce emblems for reinforcing, rather than supplementing information not conveyed in speech. Results showed the impairments in integrating the cross-modal semantic information in children with autism.

A Robot-Based Play-Drama Intervention May Improve the Joint Attention and Functional Play Behaviors of Chinese-Speaking Preschoolers with Autism Spectrum Disorder: A Pilot Study.

Children with autism spectrum disorder (ASD) have deficits in joint attention and play behaviors. We examined whether a robot-based play-drama intervention would promote these skills. Chinese-speaking preschool children were randomly assigned to an intervention group (N = 12) and a waitlist control group (N = 11). Children in the intervention group watched three robot dramas and engaged in role-plays with both robots and human experimenters over the course of 9 weeks. There were significant improvements in joint attention initiations and functional play behaviors in the intervention group. Parents of this group of children also reported less severe social impairments. It was therefore concluded that a robot-based play-drama intervention can enhance the joint attention and play behaviors of children with ASD.

Robot-based play-drama intervention may improve the narrative abilities of Chinese-speaking preschoolers with autism spectrum disorder.

BACKGROUND: Children with autism spectrum disorder (ASD) have deficits in their narrative skills and gestural communication. Very few intervention studies have been conducted with the aim of improving these skills. AIMS: We examined whether children with ASD who received the robot-based drama intervention had better narrative abilities and gestured more often than their peers who did not receive the intervention. METHODS AND PROCEDURES: Preschool children were randomly assigned to the intervention group (N = 13) and waitlist control group (N = 13). Children in the intervention group watched three robot dramas and engaged in roleplays with both robots and human experimenters. Children in both groups took the pre-tests, immediate post-tests, and, two week later, delayed post-tests, in which they narrated three stories. OUTCOMES AND RESULTS: There were significant improvements in various narrative measures, including narrative length, syntactic complexity, narrative structure, and cognitive inferences, in the intervention group. There was also an improvement in the average number of overall gestures per clause in this condition. These learning outcomes were maintained in the delayed post-test. These patterns were not found in the waitlist control group. CONCLUSIONS AND IMPLICATIONS: A robot-based play-drama intervention can enhance the narrative abilities and gestural communication of children with ASD.

Who is a better teacher for children with autism? Comparison of learning outcomes between robot-based and human-based interventions in gestural production and recognition.

BACKGROUND: Individuals with autism spectrum disorder (ASD) tend to show deficits in engaging with humans. Previous findings have shown that robot-based training improves the gestural recognition and production of children with ASD. It is not known whether social robots perform better than human therapists in teaching children with ASD. AIMS: The present study aims to compare the learning outcomes in children with ASD and intellectual disabilities from robot-based intervention on gestural use to those from human-based intervention. METHODS AND PROCEDURES: Children aged six to 12 with low-functioning autism were randomly assigned to the robot group (N = 12) and human group (N = 11). In both groups, human experimenters or social robots engaged in daily life conversations and demonstrated to children 14 intransitive gestures in a highly-structured and standardized intervention protocol. OUTCOMES AND RESULTS: Children with ASD in the human group were as likely to recognize gestures and produce them accurately as those in the robot group in both training and new conversations. Their learning outcomes maintained for at least two weeks. CONCLUSIONS AND IMPLICATIONS: The social cues found in the human-based intervention might not influence gestural learning. It does not matter who serves as teaching agents when the lessons are highly structured.

A Feel for Numbers: The Changing Role of Gesture in Manipulating the Mental Representation of an Abacus Among Children at Different Skill Levels.

Abacus mental arithmetic involves the skilled acquisition of a set of gestures representing mathematical algorithms to properly manipulate an imaginary abacus. The present study examined how the beneficial effect of abacus co-thought gestures varied at different skill and problem difficulty levels. We compared the mental arithmetic performance of 6- to 8-year-old beginning (N = 57), intermediate (N = 65), and advanced (N = 54) learners under three conditions: a physical abacus, hands-free (spontaneous gesture) mental arithmetic, and hands-restricted mental arithmetic. We adopted a mixed-subject design, with level of difficulty and skill level as the within-subject independent variables and condition as the between-subject independent variable. Our results showed a clear contrast in calculation performance and gesture accuracy among learners at different skill levels. Learners first mastered how to calculate using a physical abacus and later benefitted from using abacus gestures to aid mental arithmetic. Hand movement and gesture accuracy indicated that the beneficial effect of gestures may be related to motor learning. Beginners were proficient with a physical abacus, but performed poorly and had low gesture accuracy during mental arithmetic. Intermediates relied on gestures to do mental arithmetic and had accurate hand movements, but performed more poorly when restricted from gesturing. Advanced learners could perform mental arithmetic with accurate gestures and scored just as well without gesturing. These findings suggest that for intermediate and advanced learners, motor-spatial representation through abacus co-thought gestures may complement visual-spatial representation of a mental abacus to reduce working memory load.

Neural evidence for reduced automaticity in processing emotional prosody among men with high levels of autistic traits.

This study aimed to examine individual differences in the integration of emotional prosody when processing semantic meaning in speech among men with high and low levels of autistic traits, as measured by the Autism Spectrum Quotient (AQ). The behavioral and neural responses of high- and low-AQ men during semantic valence judgment were compared. The stimuli were positive or negative words spoken with either happy or sad prosody; in other words, the prosody was either congruous or incongruous to the valence of meaning. Participants were required to judge the (positive vs. negative) valence of word meaning as accurately and as quickly as possible while ignoring emotional prosody. Behavioral results showed that high-AQ men responded significantly more slowly than low-AQ men in all stimulus conditions, indicating lower automaticity in processing emotional speech. Neural data revealed that low-AQ men (but not high-AQ men) had significantly increased N200 and N400 amplitudes for incongruous (compared to congruous) stimuli spoken with happy prosody. Our findings supported our hypotheses that high levels of autistic traits are associated with reduced behavioral automaticity and less differential neural resources allocated to processing emotional speech stimuli with different cognitive demands.

Robot-based intervention may reduce delay in the production of intransitive gestures in Chinese-speaking preschoolers with autism spectrum disorder.

Background: Past studies have shown that robot-based intervention was effective in improving gestural use in children with autism spectrum disorders (ASD). The present study examined whether children with ASD could catch up to the level of gestural production found in age-matched children with typical development and whether they showed an increase in verbal imitation after the completion of robot-based training. We also explored the cognitive and motor skills associated with gestural learning. Methods: Children with ASD were randomly assigned to two groups. Four- to 6-year-old children with ASD in the intervention group (N = 15) received four 30-min robot-based gestural training sessions. In each session, a social robot, NAO, narrated five stories and gestured (e.g., both hands clapping for an awesome expression). Children with ASD were told to imitate the gestures during training. Age-matched children with ASD in the wait-list control group (N = 15) and age-matched children with typical development (N = 15) received the gestural training after the completion of research. Standardized pretests and posttests (both immediate and delayed) were administered to assess the accuracy and appropriateness of gestural production in both training and novel stories. Children's language and communication abilities, gestural recognition skills, fine motor proficiencies, and attention skills were also examined. Results: Children with ASD in the intervention condition were more likely to produce accurate or appropriate intransitive gestures in training and novel stories than those in the wait-list control. The positive learning outcomes were maintained in the delayed posttests. The level of gestural production accuracy in children with ASD in the delayed posttest of novel stories was comparable to that in children with typical development, suggesting that children with ASD could catch up to the level of gestural production found in children with typical development. Children with ASD in the intervention condition were also more likely to produce verbal markers while gesturing than those in the wait-list control. Gestural recognition skills were found to significantly predict the learning of gestural production accuracy in the children with ASD, with such relation partially mediated via spontaneous imitation. Conclusions: Robot-based intervention may reduce the gestural delay in children with ASD in their early childhood.

Using a social robot to teach gestural recognition and production in children with autism spectrum disorders.

While it has been argued that children with autism spectrum disorders are responsive to robot-like toys, very little research has examined the impact of robot-based intervention on gesture use. These children have delayed gestural development. We used a social robot in two phases to teach them to recognize and produce eight pantomime gestures that expressed feelings and needs. Compared to the children in the wait-list control group (N = 6), those in the intervention group (N = 7) were more likely to recognize gestures and to gesture accurately in trained and untrained scenarios. They also generalized the acquired recognition (but not production) skills to human-to-human interaction. The benefits and limitations of robot-based intervention for gestural learning were highlighted. Implications for Rehabilitation Compared to typically-developing children, children with autism spectrum disorders have delayed development of gesture comprehension and production. Robot-based intervention program was developed to teach children with autism spectrum disorders recognition (Phase I) and production (Phase II) of eight pantomime gestures that expressed feelings and needs. Children in the intervention group (but not in the wait-list control group) were able to recognize more gestures in both trained and untrained scenarios and generalize the acquired gestural recognition skills to human-to-human interaction. Similar findings were reported for gestural production except that there was no strong evidence showing children in the intervention group could produce gestures accurately in human-to-human interaction.

Absence of delay in spontaneous use of gestures in spoken narratives among children with Autism Spectrum Disorders.

BACKGROUND: Gestures are spontaneous hand movements produced when speaking. Despite gestures being of communicative significance, little is known about the gestural production in spoken narratives in six- to 12-year-old children with Autism Spectrum Disorders (ASD). AIMS: The present study examined whether six- to 12-year-old children with ASD have a delay in gestural production in a spoken narrative task, in comparison to their typically-developing (TD) peers. METHODS AND PROCEDURES: Six- to-12-year-old children with ASD (N=14) and their age- and IQ-matched TD peers (N=12) narrated a story, which could elicit spontaneous speech and gestures. Their speech and gestures were then transcribed and coded. OUTCOMES AND RESULTS: Both groups of children had comparable expressive language skills. Children with ASD produced a similar number of pointing and marker gestures to TD children and significantly more iconic gestures in their spoken narratives. While children with ASD produced more reinforcing gestures than their TD counterparts, both groups of children produced comparable numbers of disambiguating and supplementary gestures. CONCLUSIONS: Our findings indicate that children with ASD may be as capable as TD children in gestural production when they engage in spoken narratives, which gives them spontaneity in producing gestures.