Unlocking the full potential of digital endpoints for decision making: a novel modular evidence concept enabling re-use and advancing collaboration.

INTRODUCTION: Over the last decade increasing examples indicate opportunities to measure patient functioning and its relevance for clinical and regulatory decision making via endpoints collected through digital health technologies. More recently, we have seen such measures support primary study endpoints and enable smaller trials. The field is advancing fast: validation requirements have been proposed in the literature and regulators are releasing new guidances to review these endpoints. Pharmaceutical companies are embracing collaborations to develop them and working with academia and patient organizations in their development. However, the road to validation and regulatory acceptance is lengthy. The full value of digital endpoints cannot be unlocked until better collaboration and modular evidence frameworks are developed enabling re-use of evidence and repurposing of digital endpoints. AREAS COVERED: This paper proposes a solution by presenting a novel modular evidence framework -the Digital Evidence Ecosystem and Protocols (DEEP)- enabling repurposing of measurement solutions, re-use of evidence, application of standards and also facilitates collaboration with health technology assessment bodies. EXPERT OPINION: The integration of digital endpoints in healthcare, essential for personalized and remote care, requires harmonization and transparency. The proposed novel stack model offers a modular approach, fostering collaboration and expediting the adoption in patient care.

The Reliability and Clinical Validation of Automatically-Derived Verbal Memory Features of the Verbal Learning Test in Early Diagnostics of Cognitive Impairment.

BACKGROUND: Previous research has shown that verbal memory accurately measures cognitive decline in the early phases of neurocognitive impairment. Automatic speech recognition from the verbal learning task (VLT) can potentially be used to differentiate between people with and without cognitive impairment. OBJECTIVE: Investigate whether automatic speech recognition (ASR) of the VLT is reliable and able to differentiate between subjective cognitive decline (SCD) and mild cognitive impairment (MCI). METHODS: The VLT was recorded and processed via a mobile application. Following, verbal memory features were automatically extracted. The diagnostic performance of the automatically derived features was investigated by training machine learning classifiers to distinguish between participants with SCD versus MCI/dementia. RESULTS: The ICC for inter-rater reliability between the clinical and automatically derived features was 0.87 for the total immediate recall and 0.94 for the delayed recall. The full model including the total immediate recall, delayed recall, recognition count, and the novel verbal memory features had an AUC of 0.79 for distinguishing between participants with SCD versus MCI/dementia. The ten best differentiating VLT features correlated low to moderate with other cognitive tests such as logical memory tasks, semantic verbal fluency, and executive functioning. CONCLUSIONS: The VLT with automatically derived verbal memory features showed in general high agreement with the clinical scoring and distinguished well between SCD and MCI/dementia participants. This might be of added value in screening for cognitive impairment.

Validation of an Automated Speech Analysis of Cognitive Tasks within a Semiautomated Phone Assessment.

Introduction: We studied the accuracy of the automatic speech recognition (ASR) software by comparing ASR scores with manual scores from a verbal learning test (VLT) and a semantic verbal fluency (SVF) task in a semiautomated phone assessment in a memory clinic population. Furthermore, we examined the differentiating value of these tests between participants with subjective cognitive decline (SCD) and mild cognitive impairment (MCI). We also investigated whether the automatically calculated speech and linguistic features had an additional value compared to the commonly used total scores in a semiautomated phone assessment. Methods: We included 94 participants from the memory clinic of the Maastricht University Medical Center+ (SCD N = 56 and MCI N = 38). The test leader guided the participant through a semiautomated phone assessment. The VLT and SVF were audio recorded and processed via a mobile application. The recall count and speech and linguistic features were automatically extracted. The diagnostic groups were classified by training machine learning classifiers to differentiate SCD and MCI participants. Results: The intraclass correlation for inter-rater reliability between the manual and the ASR total word count was 0.89 (95% CI 0.09-0.97) for the VLT immediate recall, 0.94 (95% CI 0.68-0.98) for the VLT delayed recall, and 0.93 (95% CI 0.56-0.97) for the SVF. The full model including the total word count and speech and linguistic features had an area under the curve of 0.81 and 0.77 for the VLT immediate and delayed recall, respectively, and 0.61 for the SVF. Conclusion: There was a high agreement between the ASR and manual scores, keeping the broad confidence intervals in mind. The phone-based VLT was able to differentiate between SCD and MCI and can have opportunities for clinical trial screening.

The Accuracy of Speech and Linguistic Analysis in Early Diagnostics of Neurocognitive Disorders in a Memory Clinic Setting.

OBJECTIVE: To investigate whether automatic analysis of the Semantic Verbal Fluency test (SVF) is reliable and can extract additional information that is of value for identifying neurocognitive disorders. In addition, the associations between the automatically derived speech and linguistic features and other cognitive domains were explored. METHOD: We included 135 participants from the memory clinic of the Maastricht University Medical Center+ (with Subjective Cognitive Decline [SCD; N = 69] and Mild Cognitive Impairment [MCI]/dementia [N = 66]). The SVF task (one minute, category animals) was recorded and processed via a mobile application, and speech and linguistic features were automatically extracted. The diagnostic performance of the automatically derived features was investigated by training machine learning classifiers to differentiate SCD and MCI/dementia participants. RESULTS: The intraclass correlation for interrater reliability between the clinical total score (golden standard) and automatically derived total word count was 0.84. The full model including the total word count and the automatically derived speech and linguistic features had an Area Under the Curve (AUC) of 0.85 for differentiating between people with SCD and MCI/dementia. The model with total word count only and the model with total word count corrected for age showed an AUC of 0.75 and 0.81, respectively. Semantic switching correlated moderately with memory as well as executive functioning. CONCLUSION: The one-minute SVF task with automatically derived speech and linguistic features was as reliable as the manual scoring and differentiated well between SCD and MCI/dementia. This can be considered as a valuable addition in the screening of neurocognitive disorders and in clinical practice.

Feasibility Study of an Internet-Based Platform for Tele-Neuropsychological Assessment of Elderly in Remote Areas.

Today, in rural isolated areas or so-called 'medical deserts', access to diagnosis and care is very limited. With the current pandemic crisis, now even more than ever, telemedicine platforms are gradually more employed for remote medical assessment. Only a few are tailored to comprehensive teleneuropsychological assessment of older adults. Hence, our study focuses on evaluating the feasibility of performing a remote neuropsychological assessment of older adults suffering from a cognitive complaint. 50 participants (aged 55 and older) were recruited at the local hospital of Digne-les-Bains, France. A brief neuropsychological assessment including a short clinical interview and several validated neuropsychological tests was administered in two conditions, once by Teleneuropsychology (TNP) and once by Face-to-Face (FTF) in a crossover design. Acceptability and user experience was assessed through questionnaires. Results show high agreement in most tests between the FTF and TNP conditions. The TNP was overall well accepted by the participants. However, differences in test performances were observed, which urges the need to validate TNP tests with broader samples with normative data.

Remote data collection speech analysis and prediction of the identification of Alzheimer's disease biomarkers in people at risk for Alzheimer's disease dementia: the Speech on the Phone Assessment (SPeAk) prospective observational study protocol.

INTRODUCTION: Identifying cost-effective, non-invasive biomarkers of Alzheimer's disease (AD) is a clinical and research priority. Speech data are easy to collect, and studies suggest it can identify those with AD. We do not know if speech features can predict AD biomarkers in a preclinical population. METHODS AND ANALYSIS: The Speech on the Phone Assessment (SPeAk) study is a prospective observational study. SPeAk recruits participants aged 50 years and over who have previously completed studies with AD biomarker collection. Participants complete a baseline telephone assessment, including spontaneous speech and cognitive tests. A 3-month visit will repeat the cognitive tests with a conversational artificial intelligence bot. Participants complete acceptability questionnaires after each visit. Participants are randomised to receive their cognitive test results either after each visit or only after they have completed the study. We will combine SPeAK data with AD biomarker data collected in a previous study and analyse for correlations between extracted speech features and AD biomarkers. The outcome of this analysis will inform the development of an algorithm for prediction of AD risk based on speech features. ETHICS AND DISSEMINATION: This study has been approved by the Edinburgh Medical School Research Ethics Committee (REC reference 20-EMREC-007). All participants will provide informed consent before completing any study-related procedures, participants must have capacity to consent to participate in this study. Participants may find the tests, or receiving their scores, causes anxiety or stress. Previous exposure to similar tests may make this more familiar and reduce this anxiety. The study information will include signposting in case of distress. Study results will be disseminated to study participants, presented at conferences and published in a peer reviewed journal. No study participants will be identifiable in the study results.

Remote cognitive assessment of older adults in rural areas by telemedicine and automatic speech and video analysis: protocol for a cross-over feasibility study.

INTRODUCTION: Early detection of cognitive impairments is crucial for the successful implementation of preventive strategies. However, in rural isolated areas or so-called 'medical deserts', access to diagnosis and care is very limited. With the current pandemic crisis, now even more than ever, remote solutions such as telemedicine platforms represent great potential and can help to overcome this barrier. Moreover, current advances made in voice and image analysis can help overcome the barrier of physical distance by providing additional information on a patients' emotional and cognitive state. Therefore, the aim of this study is to evaluate the feasibility and reliability of a videoconference system for remote cognitive testing empowered by automatic speech and video analysis. METHODS AND ANALYSIS: 60 participants (aged 55 and older) with and without cognitive impairment will be recruited. A complete neuropsychological assessment including a short clinical interview will be administered in two conditions, once by telemedicine and once by face-to-face. The order of administration procedure will be counterbalanced so half of the sample starts with the videoconference condition and the other half with the face-to-face condition. Acceptability and user experience will be assessed among participants and clinicians in a qualitative and quantitative manner. Speech and video features will be extracted and analysed to obtain additional information on mood and engagement levels. In a subgroup, measurements of stress indicators such as heart rate and skin conductance will be compared. ETHICS AND DISSEMINATION: The procedures are not invasive and there are no expected risks or burdens to participants. All participants will be informed that this is an observational study and their consent taken prior to the experiment. Demonstration of the effectiveness of such technology makes it possible to diffuse its use across all rural areas ('medical deserts') and thus, to improve the early diagnosis of neurodegenerative pathologies, while providing data crucial for basic research. Results from this study will be published in peer-reviewed journals.

Pilot Study to Assess the Feasibility of a Mobile Unit for Remote Cognitive Screening of Isolated Elderly in Rural Areas.

BACKGROUND: Given the current COVID-19 pandemic situation, now more than ever, remote solutions for assessing and monitoring individuals with cognitive impairment are urgently needed. Older adults in particular, living in isolated rural areas or so-called 'medical deserts', are facing major difficulties in getting access to diagnosis and care. Telemedical approaches to assessments are promising and seem well accepted, reducing the burden of bringing patients to specialized clinics. However, many older adults are not yet adequately equipped to allow for proper implementation of this technology. A potential solution could be a mobile unit in the form of a van, equipped with the telemedical system which comes to the patients' home. The aim of this proof-of-concept study is to evaluate the feasibility and reliability of such mobile unit settings for remote cognitive testing. Methods and analysis: eight participants (aged between 69 and 86 years old) from the city of Digne-Les-Bains volunteered for this study. A basic neuropsychological assessment, including a short clinical interview, is administered in two conditions, by telemedicine in a mobile clinic (equipped van) at a participants' home and face to face in a specialized clinic. The administration procedure order is randomized, and the results are compared with each other. Acceptability and user experience are assessed among participants and clinicians in a qualitative and quantitative manner. Measurements of stress indicators were collected for comparison. RESULTS: The analysis revealed no significant differences in test results between the two administration procedures. Participants were, overall, very satisfied with the mobile clinic experience and found the use of the telemedical system relatively easy. CONCLUSION: A mobile unit equipped with a telemedical service could represent a solution for remote cognitive testing overcoming barriers in rural areas to access specialized diagnosis and care.