Generalizing Parkinson's disease detection using keystroke dynamics: a self-supervised approach.

OBJECTIVE: Passive monitoring of touchscreen interactions generates keystroke dynamic signals that can be used to detect and track neurological conditions such as Parkinson's disease (PD) and psychomotor impairment with minimal burden on the user. However, this typically requires datasets with clinically confirmed labels collected in standardized environments, which is challenging, especially for a large subject pool. This study validates the efficacy of a self-supervised learning method in reducing the reliance on labels and evaluates its generalizability. MATERIALS AND METHODS: We propose a new type of self-supervised loss combining Barlow Twins loss, which attempts to create similar feature representations with reduced feature redundancy for samples coming from the same subject, and a Dissimilarity loss, which promotes uncorrelated features for samples generated by different subjects. An encoder is first pre-trained using this loss on unlabeled data from an uncontrolled setting, then fine-tuned with clinically validated data. Our experiments test the model generalizability with controls and subjects with PD on 2 independent datasets. RESULTS: Our approach showed better generalization compared to previous methods, including a feature engineering strategy, a deep learning model pre-trained on Parkinsonian signs, and a traditional supervised model. DISCUSSION: The absence of standardized data acquisition protocols and the limited availability of annotated datasets compromise the generalizability of supervised models. In these contexts, self-supervised models offer the advantage of learning more robust patterns from the data, bypassing the need for ground truth labels. CONCLUSION: This approach has the potential to accelerate the clinical validation of touchscreen typing software for neurodegenerative diseases.

Exploring Asymmetric Fine Motor Impairment Trends in Early Parkinson's Disease via Keystroke Typing.

Background: The nQiMechPD algorithm transforms natural typing data into a numerical index that characterizes motor impairment in people with Parkinson's Disease (PwPD). Objectives: Use nQiMechPD to compare asymmetrical progression of PD-related impairment in dominant (D-PD) versus non-dominant side onset (ND-PD) de-novo patients. Methods: Keystroke data were collected from 53 right-handed participants (15 D-PD, 13 ND-PD, 25 controls). We apply linear mixed effects modeling to evaluate participants' right, left, and both hands nQiMechPD relative change by group. Results: The 6-month nQiMechPD trajectories of right (**P = 0.002) and both (*P = 0.043) hands showed a significant difference in nQiMechPD trends between D-PD and ND-PD participants. Left side trends were not significantly different between these two groups (P = 0.328). Conclusions: Significant differences between D-PD and ND-PD groups were observed, likely driven by contrasting dominant hand trends. Our findings suggest disease onset side may influence motor impairment progression, medication response, and functional outcomes in PwPD.

Keystroke-Dynamics for Parkinson's Disease Signs Detection in an At-Home Uncontrolled Population: A New Benchmark and Method.

Parkinson's disease (PD) is the second most prevalent neurodegenerative disease disorder in the world. A prompt diagnosis would enable clinical trials for disease-modifying neuroprotective therapies. Recent research efforts have unveiled imaging and blood markers that have the potential to be used to identify PD patients promptly, however, the idiopathic nature of PD makes these tests very hard to scale to the general population. To this end, we need an easily deployable tool that would enable screening for PD signs in the general population. In this work, we propose a new set of features based on keystroke dynamics, i.e., the time required to press and release keyboard keys during typing, and used to detect PD in an ecologically valid data acquisition setup at the subject's homes, without requiring any pre-defined task. We compare and contrast existing models presented in the literature and present a new model that combines a new type of keystroke dynamics signal representation using hold time and flight time series as a function of key types and asymmetry in the time series using a convolutional neural network. We show how this model achieves an Area Under the Receiving Operating Characteristic curve ranging from 0.80 to 0.83 on a dataset of subjects who actively interacted with their computers for at least 5 months and positively compares against other state-of-the-art approaches previously tested on keystroke dynamics data acquired with mechanical keyboards.

A novel framework to estimate cognitive impairment via finger interaction with digital devices.

Measuring cognitive function is essential for characterizing brain health and tracking cognitive decline in Alzheimer's Disease and other neurodegenerative conditions. Current tools to accurately evaluate cognitive impairment typically rely on a battery of questionnaires administered during clinical visits which is essential for the acquisition of repeated measurements in longitudinal studies. Previous studies have shown that the remote data collection of passively monitored daily interaction with personal digital devices can measure motor signs in the early stages of synucleinopathies, as well as facilitate longitudinal patient assessment in the real-world scenario with high patient compliance. This was achieved by the automatic discovery of patterns in the time series of keystroke dynamics, i.e. the time required to press and release keys, by machine learning algorithms. In this work, our hypothesis is that the typing patterns generated from user-device interaction may reflect relevant features of the effects of cognitive impairment caused by neurodegeneration. We use machine learning algorithms to estimate cognitive performance through the analysis of keystroke dynamic patterns that were extracted from mechanical and touchscreen keyboard use in a dataset of cognitively normal (n = 39, 51% male) and cognitively impaired subjects (n = 38, 60% male). These algorithms are trained and evaluated using a novel framework that integrates items from multiple neuropsychological and clinical scales into cognitive subdomains to generate a more holistic representation of multifaceted clinical signs. In our results, we see that these models based on typing input achieve moderate correlations with verbal memory, non-verbal memory and executive function subdomains [Spearman's ρ between 0.54 (P < 0.001) and 0.42 (P < 0.001)] and a weak correlation with language/verbal skills [Spearman's ρ 0.30 (P < 0.05)]. In addition, we observe a moderate correlation between our typing-based approach and the Total Montreal Cognitive Assessment score [Spearman's ρ 0.48 (P < 0.001)]. Finally, we show that these machine learning models can perform better by using our subdomain framework that integrates the information from multiple neuropsychological scales as opposed to using the individual items that make up these scales. Our results support our hypothesis that typing patterns are able to reflect the effects of neurodegeneration in mild cognitive impairment and Alzheimer's disease and that this new subdomain framework both helps the development of machine learning models and improves their interpretability.

Detection of Mental Fatigue in the General Population: Feasibility Study of Keystroke Dynamics as a Real-world Biomarker.

BACKGROUND: Mental fatigue is a common and potentially debilitating state that can affect individuals' health and quality of life. In some cases, its manifestation can precede or mask early signs of other serious mental or physiological conditions. Detecting and assessing mental fatigue can be challenging nowadays as it relies on self-evaluation and rating questionnaires, which are highly influenced by subjective bias. Introducing more objective, quantitative, and sensitive methods to characterize mental fatigue could be critical to improve its management and the understanding of its connection to other clinical conditions. OBJECTIVE: This paper aimed to study the feasibility of using keystroke biometrics for mental fatigue detection during natural typing. As typing involves multiple motor and cognitive processes that are affected by mental fatigue, our hypothesis was that the information captured in keystroke dynamics can offer an interesting mean to characterize users' mental fatigue in a real-world setting. METHODS: We apply domain transformation techniques to adapt and transform TypeNet, a state-of-the-art deep neural network, originally intended for user authentication, to generate a network optimized for the fatigue detection task. All experiments were conducted using 3 keystroke databases that comprise different contexts and data collection protocols. RESULTS: Our preliminary results showed area under the curve performances ranging between 72.2% and 80% for fatigue versus rested sample classification, which is aligned with previously published models on daily alertness and circadian cycles. This demonstrates the potential of our proposed system to characterize mental fatigue fluctuations via natural typing patterns. Finally, we studied the performance of an active detection approach that leverages the continuous nature of keystroke biometric patterns for the assessment of users' fatigue in real time. CONCLUSIONS: Our results suggest that the psychomotor patterns that characterize mental fatigue manifest during natural typing, which can be quantified via automated analysis of users' daily interaction with their device. These findings represent a step towards the development of a more objective, accessible, and transparent solution to monitor mental fatigue in a real-world environment.

Remote Monitoring of Treatment Response in Parkinson's Disease: The Habit of Typing on a Computer.

OBJECTIVE: The recent advances in technology are opening a new opportunity to remotely evaluate motor features in people with Parkinson's disease (PD). We hypothesized that typing on an electronic device, a habitual behavior facilitated by the nigrostriatal dopaminergic pathway, could allow for objectively and nonobtrusively monitoring parkinsonian features and response to medication in an at-home setting. METHODS: We enrolled 31 participants recently diagnosed with PD who were due to start dopaminergic treatment and 30 age-matched controls. We remotely monitored their typing pattern during a 6-month (24 weeks) follow-up period before and while dopaminergic medications were being titrated. The typing data were used to develop a novel algorithm based on recursive neural networks and detect participants' responses to medication. The latter were defined by the Unified Parkinson's Disease Rating Scale-III (UPDRS-III) minimal clinically important difference. Furthermore, we tested the accuracy of the algorithm to predict the final response to medication as early as 21 weeks prior to the final 6-month clinical outcome. RESULTS: The score on the novel algorithm based on recursive neural networks had an overall moderate kappa agreement and fair area under the receiver operating characteristic (ROC) curve with the time-coincident UPDRS-III minimal clinically important difference. The participants classified as responders at the final visit (based on the UPDRS-III minimal clinically important difference) had higher scores on the novel algorithm based on recursive neural networks when compared with the participants with stable UPDRS-III, from the third week of the study onward. CONCLUSIONS: This preliminary study suggests that remotely gathered unsupervised typing data allows for the accurate detection and prediction of drug response in PD. © 2019 International Parkinson and Movement Disorder Society.

Detecting Motor Impairment in Early Parkinson's Disease via Natural Typing Interaction With Keyboards: Validation of the neuroQWERTY Approach in an Uncontrolled At-Home Setting.

BACKGROUND: Parkinson's disease (PD) is the second most prevalent neurodegenerative disease and one of the most common forms of movement disorder. Although there is no known cure for PD, existing therapies can provide effective symptomatic relief. However, optimal titration is crucial to avoid adverse effects. Today, decision making for PD management is challenging because it relies on subjective clinical evaluations that require a visit to the clinic. This challenge has motivated recent research initiatives to develop tools that can be used by nonspecialists to assess psychomotor impairment. Among these emerging solutions, we recently reported the neuroQWERTY index, a new digital marker able to detect motor impairment in an early PD cohort through the analysis of the key press and release timing data collected during a controlled in-clinic typing task. OBJECTIVE: The aim of this study was to extend the in-clinic implementation to an at-home implementation by validating the applicability of the neuroQWERTY approach in an uncontrolled at-home setting, using the typing data from subjects' natural interaction with their laptop to enable remote and unobtrusive assessment of PD signs. METHODS: We implemented the data-collection platform and software to enable access and storage of the typing data generated by users while using their computer at home. We recruited a total of 60 participants; of these participants 52 (25 people with Parkinson's and 27 healthy controls) provided enough data to complete the analysis. Finally, to evaluate whether our in-clinic-built algorithm could be used in an uncontrolled at-home setting, we compared its performance on the data collected during the controlled typing task in the clinic and the results of our method using the data passively collected at home. RESULTS: Despite the randomness and sparsity introduced by the uncontrolled setting, our algorithm performed nearly as well in the at-home data (area under the receiver operating characteristic curve [AUC] of 0.76 and sensitivity/specificity of 0.73/0.69) as it did when used to evaluate the in-clinic data (AUC 0.83 and sensitivity/specificity of 0.77/0.72). Moreover, the keystroke metrics presented a strong correlation between the 2 typing settings, which suggests a minimal influence of the in-clinic typing task in users' normal typing. CONCLUSIONS: The finding that an algorithm trained on data from an in-clinic setting has comparable performance with that tested on data collected through naturalistic at-home computer use reinforces the hypothesis that subtle differences in motor function can be detected from typing behavior. This work represents another step toward an objective, user-convenient, and quasi-continuous monitoring tool for PD.

Detection of Motor Impairment in Parkinson's Disease Via Mobile Touchscreen Typing.

Mobile technology is opening a wide range of opportunities for transforming the standard of care for chronic disorders. Using smartphones as tools for longitudinally tracking symptoms could enable personalization of drug regimens and improve patient monitoring. Parkinson's disease (PD) is an ideal candidate for these tools. At present, evaluation of PD signs requires trained experts to quantify motor impairment in the clinic, limiting the frequency and quality of the information available for understanding the status and progression of the disease. Mobile technology can help clinical decision making by completing the information of motor status between hospital visits. This paper presents an algorithm to detect PD by analyzing the typing activity on smartphones independently of the content of the typed text. We propose a set of touchscreen typing features based on a covariance, skewness, and kurtosis analysis of the timing information of the data to capture PD motor signs. We tested these features, both independently and in a multivariate framework, in a population of 21 PD and 23 control subjects, achieving a sensitivity/specificity of 0.81/0.81 for the best performing feature and 0.73/0.84 for the best multivariate method. The results of the alternating finger-tapping, an established motor test, measured in our cohort are 0.75/0.78. This paper contributes to the development of a home-based, high-compliance, and high-frequency PD motor test by analysis of routine typing on touchscreens.