ABSTRACT
This study focused on the development and initial psychometric evaluation of a set of online, webcam-collected, and artificial intelligence-derived patient performance measures for neurodevelopmental genetic syndromes (NDGS). Initial testing and qualitative input was used to develop four stimulus paradigms capturing social and cognitive processes, including social attention, receptive vocabulary, processing speed, and single-word reading. The paradigms were administered to a sample of 375 participants, including 163 with NDGS, 56 with idiopathic neurodevelopmental disability (NDD), and 156 neurotypical controls. Twelve measures were created from the four stimulus paradigms. Valid completion rates varied from 87 to 100% across measures, with lower but adequate completion rates in participants with intellectual disability. Adequate to excellent internal consistency reliability (α = 0.67 to 0.95) was observed across measures. Test-retest reproducibility at 1-month follow-up and stability at 4-month follow-up was fair to good (r = 0.40-0.73) for 8 of the 12 measures. All gaze-based measures showed evidence of convergent and discriminant validity with parent-report measures of other cognitive and behavioral constructs. Comparisons across NDGS groups revealed distinct patterns of social and cognitive functioning, including people with PTEN mutations showing a less impaired overall pattern and people with SYNGAP1 mutations showing more attentional, processing speed, and social processing difficulties relative to people with NFIX mutations. Webcam-collected performance measures appear to be a reliable and potentially useful method for objective characterization and monitoring of social and cognitive processes in NDGS and idiopathic NDD. Additional validation work, including more detailed convergent and discriminant validity analyses and examination of sensitivity to change, is needed to replicate and extend these observations.
Subject(s)
Artificial Intelligence , Intellectual Disability , Humans , Reproducibility of Results , Intelligence , PsychometricsABSTRACT
There are few well-validated measures that are appropriate for assessing the full range of neurobehavioral presentations in PTEN hamartoma tumor syndrome (PHTS) and other neurodevelopmental genetic syndromes (NDGS). As potential therapeutics are developed, having reliable, valid, free, and easily accessible measures to track a range of neurobehavioral domains will be crucial for future clinical trials. This study focused on the development and initial psychometric evaluation of a set of freely available informant-report survey scales for PHTS-the Neurobehavioral Evaluation Tool (NET). Concept elicitation, quantitative ratings, and cognitive interviewing processes were conducted with stakeholders and clinician-scientist experts, used to identify the most important neurobehavioral domains for this population, and to ensure items were appropriate for the full range of individuals with PHTS. Results of this process identified a PHTS neurobehavioral impact model with 11 domains. The final NET scales assessing these domains were administered to a sample of 384 participants (median completion time = 20.6 min), including 32 people with PHTS, 141 with other NDGS, 47 with idiopathic neurodevelopmental disorder (NDD), and 164 neurotypical controls. Initial psychometric results for the total scores of each scale indicated very good model (ω = 0.83-0.99) and internal consistency reliability (α = 0.82-0.98) as well as excellent test-retest reproducibility at 1-month follow-up (r = 0.78-0.98) and stability at 4-month follow-up (r = 0.76-0.96). Conditional reliability estimates indicated very strong measurement precision in key score ranges for assessing PHTS and other people with NDGS and/or idiopathic NDD. Comparisons across domains between PHTS and the other groups revealed specific patterns of symptoms and functioning, including lower levels of challenging behavior and more developed daily living and executive functioning skills relative to other NDGS. The NET appears to be a reliable and potentially useful tool for clinical characterization and monitoring of neurobehavioral symptoms in PHTS and may also have utility in the assessment of other NDGS and idiopathic NDD. Additional validation work, including convergent and discriminant validity analyses, are needed to replicate and extend these observations.