Computer Vision Analysis for Quantification of Autism Risk Behaviors.

Observational behavior analysis plays a key role for the discovery and evaluation of risk markers for many neurodevelopmental disorders. Research on autism spectrum disorder (ASD) suggests that behavioral risk markers can be observed at 12 months of age or earlier, with diagnosis possible at 18 months. To date, these studies and evaluations involving observational analysis tend to rely heavily on clinical practitioners and specialists who have undergone intensive training to be able to reliably administer carefully designed behavioural-eliciting tasks, code the resulting behaviors, and interpret such behaviors. These methods are therefore extremely expensive, time-intensive, and are not easily scalable for large population or longitudinal observational analysis. We developed a self-contained, closed-loop, mobile application with movie stimuli designed to engage the child's attention and elicit specific behavioral and social responses, which are recorded with a mobile device camera and then analyzed via computer vision algorithms. Here, in addition to presenting this paradigm, we validate the system to measure engagement, name-call responses, and emotional responses of toddlers with and without ASD who were presented with the application. Additionally, we show examples of how the proposed framework can further risk marker research with fine-grained quantification of behaviors. The results suggest these objective and automatic methods can be considered to aid behavioral analysis, and can be suited for objective automatic analysis for future studies.

A Scalable Off-the-Shelf Framework for Measuring Patterns of Attention in Young Children and its Application in Autism Spectrum Disorder.

Autism spectrum disorder (ASD) is associated with deficits in the processing of social information and difficulties in social interaction, and individuals with ASD exhibit atypical attention and gaze. Traditionally, gaze studies have relied upon precise and constrained means of monitoring attention using expensive equipment in laboratories. In this work we develop a low-cost off-the-shelf alternative for measuring attention that can be used in natural settings. The head and iris positions of 104 16-31 months children, an age range appropriate for ASD screening and diagnosis, 22 of them diagnosed with ASD, were recorded using the front facing camera in an iPad while they watched on the device screen a movie displaying dynamic stimuli, social stimuli on the left and nonsocial stimuli on the right. The head and iris position were then automatically analyzed via computer vision algorithms to detect the direction of attention. Children in the ASD group paid less attention to the movie, showed less attention to the social as compared to the nonsocial stimuli, and often fixated their attention to one side of the screen. The proposed method provides a low-cost means of monitoring attention to properly designed stimuli, demonstrating that the integration of stimuli design and automatic response analysis results in the opportunity to use off-the-shelf cameras to assess behavioral biomarkers.

A Six-Minute Measure of Vocalizations in Toddlers with Autism Spectrum Disorder.

To improve early identification of autism spectrum disorder (ASD), we need objective, reliable, and accessible measures. To that end, a previous study demonstrated that a tablet-based application (app) that assessed several autism risk behaviors distinguished between toddlers with ASD and non-ASD toddlers. Using vocal data collected during this study, we investigated whether vocalizations uttered during administration of this app can distinguish among toddlers aged 16-31 months with typical development (TD), language or developmental delay (DLD), and ASD. Participant's visual and vocal responses were recorded using the camera and microphone in a tablet while toddlers watched movies designed to elicit behaviors associated with risk for ASD. Vocalizations were then coded offline. Results showed that (a) children with ASD and DLD were less likely to produce words during app administration than TD participants; (b) the ratio of syllabic vocalizations to all vocalizations was higher among TD than ASD or DLD participants; and (c) the rates of nonsyllabic vocalizations were higher in the ASD group than in either the TD or DLD groups. Those producing more nonsyllabic vocalizations were 24 times more likely to be diagnosed with ASD. These results lend support to previous findings that early vocalizations might be useful in identifying risk for ASD in toddlers and demonstrate the feasibility of using a scalable tablet-based app for assessing vocalizations in the context of a routine pediatric visit. LAY SUMMARY: Although parents often report symptoms of autism spectrum disorder (ASD) in infancy, we are not yet reliably diagnosing ASD until much later in development. A previous study tested a tablet-based application (app) that recorded behaviors we know are associated with ASD to help identify children at risk for the disorder. Here we measured how children vocalize while they watched the movies presented on the tablet. Children with ASD were less likely to produce words, less likely to produce speechlike sounds, and more likely to produce atypical sounds while watching these movies. These measures, combined with other behaviors measured by the app, might help identify which children should be evaluated for ASD. Autism Res 2020, 13: 1373-1382. © 2020 International Society for Autism Research, Wiley Periodicals, Inc.

Author Correction: Atypical postural control can be detected via computer vision analysis in toddlers with autism spectrum disorder.

An amendment to this paper has been published and can be accessed via a link at the top of the paper.

Computer vision and behavioral phenotyping: an autism case study.

Despite significant recent advances in molecular genetics and neuroscience, behavioral ratings based on clinical observations are still the gold standard for screening, diagnosing, and assessing outcomes in neurodevelopmental disorders, including autism spectrum disorder. Such behavioral ratings are subjective, require significant clinician expertise and training, typically do not capture data from the children in their natural environments such as homes or schools, and are not scalable for large population screening, low-income communities, or longitudinal monitoring, all of which are critical for outcome evaluation in multisite studies and for understanding and evaluating symptoms in the general population. The development of computational approaches to standardized objective behavioral assessment is, thus, a significant unmet need in autism spectrum disorder in particular and developmental and neurodegenerative disorders in general. Here, we discuss how computer vision, and machine learning, can develop scalable low-cost mobile health methods for automatically and consistently assessing existing biomarkers, from eye tracking to movement patterns and affect, while also providing tools and big data for novel discovery.

Computer vision analysis captures atypical attention in toddlers with autism.

To demonstrate the capability of computer vision analysis to detect atypical orienting and attention behaviors in toddlers with autism spectrum disorder. One hundered and four toddlers of 16-31 months old (mean = 22) participated in this study. Twenty-two of the toddlers had autism spectrum disorder and 82 had typical development or developmental delay. Toddlers watched video stimuli on a tablet while the built-in camera recorded their head movement. Computer vision analysis measured participants' attention and orienting in response to name calls. Reliability of the computer vision analysis algorithm was tested against a human rater. Differences in behavior were analyzed between the autism spectrum disorder group and the comparison group. Reliability between computer vision analysis and human coding for orienting to name was excellent (intra-class coefficient 0.84, 95% confidence interval 0.67-0.91). Only 8% of toddlers with autism spectrum disorder oriented to name calling on >1 trial, compared to 63% of toddlers in the comparison group (p = 0.002). Mean latency to orient was significantly longer for toddlers with autism spectrum disorder (2.02 vs 1.06 s, p = 0.04). Sensitivity for autism spectrum disorder of atypical orienting was 96% and specificity was 38%. Older toddlers with autism spectrum disorder showed less attention to the videos overall (p = 0.03). Automated coding offers a reliable, quantitative method for detecting atypical social orienting and reduced sustained attention in toddlers with autism spectrum disorder.

Atypical postural control can be detected via computer vision analysis in toddlers with autism spectrum disorder.

Evidence suggests that differences in motor function are an early feature of autism spectrum disorder (ASD). One aspect of motor ability that develops during childhood is postural control, reflected in the ability to maintain a steady head and body position without excessive sway. Observational studies have documented differences in postural control in older children with ASD. The present study used computer vision analysis to assess midline head postural control, as reflected in the rate of spontaneous head movements during states of active attention, in 104 toddlers between 16-31 months of age (Mean = 22 months), 22 of whom were diagnosed with ASD. Time-series data revealed robust group differences in the rate of head movements while the toddlers watched movies depicting social and nonsocial stimuli. Toddlers with ASD exhibited a significantly higher rate of head movement as compared to non-ASD toddlers, suggesting difficulties in maintaining midline position of the head while engaging attentional systems. The use of digital phenotyping approaches, such as computer vision analysis, to quantify variation in early motor behaviors will allow for more precise, objective, and quantitative characterization of early motor signatures and potentially provide new automated methods for early autism risk identification.

Motivational valence alters memory formation without altering exploration of a real-life spatial environment.

Volitional exploration and learning are key to adaptive behavior, yet their characterization remains a complex problem for cognitive science. Exploration has been posited as a mechanism by which motivation promotes memory, but this relationship is not well-understood, in part because novel stimuli that motivate exploration also reliably elicit changes in neuromodulatory brain systems that directly alter memory formation, via effects on neural plasticity. To deconfound interrelationships between motivation, exploration, and memory formation we manipulated motivational state prior to entering a spatial context, measured exploratory responses to the context and novel stimuli within it, and then examined motivation and exploration as predictors of memory outcomes. To elicit spontaneous exploration, we used the physical space of an art exhibit with affectively rich content; we expected motivated exploration and memory to reflect multiple factors, including not only motivational valence, but also individual differences. Motivation was manipulated via an introductory statement framing exhibit themes in terms of Promotion- or Prevention-oriented goals. Participants explored the exhibit while being tracked by video. They returned 24 hours later for recall and spatial memory tests, followed by measures of motivation, personality, and relevant attitude variables. Promotion and Prevention condition participants did not differ in terms of group-level exploration time or memory metrics, suggesting similar motivation to explore under both framing contexts. However, exploratory behavior and memory outcomes were significantly more closely related under Promotion than Prevention, indicating that Prevention framing disrupted expected depth-of-encoding effects. Additionally, while trait measures predicted exploration similarly across framing conditions, traits interacted with motivational framing context and facial affect to predict memory outcomes. This novel characterization of motivated learning implies that dissociable behavioral and biological mechanisms, here varying as a function of valence, contribute to memory outcomes in complex, real-life environments.

Automatic emotion and attention analysis of young children at home: a ResearchKit autism feasibility study.

Current tools for objectively measuring young children's observed behaviors are expensive, time-consuming, and require extensive training and professional administration. The lack of scalable, reliable, and validated tools impacts access to evidence-based knowledge and limits our capacity to collect population-level data in non-clinical settings. To address this gap, we developed mobile technology to collect videos of young children while they watched movies designed to elicit autism-related behaviors and then used automatic behavioral coding of these videos to quantify children's emotions and behaviors. We present results from our iPhone study Autism & Beyond, built on ResearchKit's open-source platform. The entire study-from an e-Consent process to stimuli presentation and data collection-was conducted within an iPhone-based app available in the Apple Store. Over 1 year, 1756 families with children aged 12-72 months old participated in the study, completing 5618 caregiver-reported surveys and uploading 4441 videos recorded in the child's natural settings. Usable data were collected on 87.6% of the uploaded videos. Automatic coding identified significant differences in emotion and attention by age, sex, and autism risk status. This study demonstrates the acceptability of an app-based tool to caregivers, their willingness to upload videos of their children, the feasibility of caregiver-collected data in the home, and the application of automatic behavioral encoding to quantify emotions and attention variables that are clinically meaningful and may be refined to screen children for autism and developmental disorders outside of clinical settings. This technology has the potential to transform how we screen and monitor children's development.

Use of a Digital Modified Checklist for Autism in Toddlers - Revised with Follow-up to Improve Quality of Screening for Autism.

OBJECTIVES: To assess changes in quality of care for children at risk for autism spectrum disorders (ASD) due to process improvement and implementation of a digital screening form. STUDY DESIGN: The process of screening for ASD was studied in an academic primary care pediatrics clinic before and after implementation of a digital version of the Modified Checklist for Autism in Toddlers - Revised with Follow-up with automated risk assessment. Quality metrics included accuracy of documentation of screening results and appropriate action for positive screens (secondary screening or referral). Participating physicians completed pre- and postintervention surveys to measure changes in attitudes toward feasibility and value of screening for ASD. Evidence of change was evaluated with statistical process control charts and χ2 tests. RESULTS: Accurate documentation in the electronic health record of screening results increased from 54% to 92% (38% increase, 95% CI 14%-64%) and appropriate action for children screening positive increased from 25% to 85% (60% increase, 95% CI 35%-85%). A total of 90% of participating physicians agreed that the transition to a digital screening form improved their clinical assessment of autism risk. CONCLUSIONS: Implementation of a tablet-based digital version of the Modified Checklist for Autism in Toddlers - Revised with Follow-up led to improved quality of care for children at risk for ASD and increased acceptability of screening for ASD. Continued efforts towards improving the process of screening for ASD could facilitate rapid, early diagnosis of ASD and advance the accuracy of studies of the impact of screening.

Computer vision tools for low-cost and noninvasive measurement of autism-related behaviors in infants.

The early detection of developmental disorders is key to child outcome, allowing interventions to be initiated which promote development and improve prognosis. Research on autism spectrum disorder (ASD) suggests that behavioral signs can be observed late in the first year of life. Many of these studies involve extensive frame-by-frame video observation and analysis of a child's natural behavior. Although nonintrusive, these methods are extremely time-intensive and require a high level of observer training; thus, they are burdensome for clinical and large population research purposes. This work is a first milestone in a long-term project on non-invasive early observation of children in order to aid in risk detection and research of neurodevelopmental disorders. We focus on providing low-cost computer vision tools to measure and identify ASD behavioral signs based on components of the Autism Observation Scale for Infants (AOSI). In particular, we develop algorithms to measure responses to general ASD risk assessment tasks and activities outlined by the AOSI which assess visual attention by tracking facial features. We show results, including comparisons with expert and nonexpert clinicians, which demonstrate that the proposed computer vision tools can capture critical behavioral observations and potentially augment the clinician's behavioral observations obtained from real in-clinic assessments.

A tetraploid intermediate precedes aneuploid formation in yeasts exposed to fluconazole.

Candida albicans, the most prevalent human fungal pathogen, is generally diploid. However, 50% of isolates that are resistant to fluconazole (FLC), the most widely used antifungal, are aneuploid and some aneuploidies can confer FLC resistance. To ask if FLC exposure causes or only selects for aneuploidy, we analyzed diploid strains during exposure to FLC using flow cytometry and epifluorescence microscopy. FLC exposure caused a consistent deviation from normal cell cycle regulation: nuclear and spindle cycles initiated prior to bud emergence, leading to "trimeras," three connected cells composed of a mother, daughter, and granddaughter bud. Initially binucleate, trimeras underwent coordinated nuclear division yielding four daughter nuclei, two of which underwent mitotic collapse to form a tetraploid cell with extra spindle components. In subsequent cell cycles, the abnormal number of spindles resulted in unequal DNA segregation and viable aneuploid progeny. The process of aneuploid formation in C. albicans is highly reminiscent of early stages in human tumorigenesis in that aneuploidy arises through a tetraploid intermediate and subsequent unequal DNA segregation driven by multiple spindles coupled with a subsequent selective advantage conferred by at least some aneuploidies during growth under stress. Finally, trimera formation was detected in response to other azole antifungals, in related Candida species, and in an in vivo model for Candida infection, suggesting that aneuploids arise due to azole treatment of several pathogenic yeasts and that this can occur during the infection process.