Measuring the prediction difficulty of individual cases in a dataset using machine learning.

Different levels of prediction difficulty are one of the key factors that researchers encounter when applying machine learning to data. Although previous studies have introduced various metrics for assessing the prediction difficulty of individual cases, these metrics require specific dataset preconditions. In this paper, we propose three novel metrics for measuring the prediction difficulty of individual cases using fully-connected feedforward neural networks. The first metric is based on the complexity of the neural network needed to make a correct prediction. The second metric employs a pair of neural networks: one makes a prediction for a given case, and the other predicts whether the prediction made by the first model is likely to be correct. The third metric assesses the variability of the neural network's predictions. We investigated these metrics using a variety of datasets, visualized their values, and compared them to fifteen existing metrics from the literature. The results demonstrate that the proposed case difficulty metrics were better able to differentiate various levels of difficulty than most of the existing metrics and show constant effectiveness across diverse datasets. We expect our metrics will provide researchers with a new perspective on understanding their datasets and applying machine learning in various fields.

Sensitivity and specificity of alternative screening methods for systematic reviews using text mining tools.

OBJECTIVES: To evaluate the impact of text mining (TM) on the sensitivity and specificity of title and abstract screening strategies for systematic reviews (SRs). STUDY DESIGN AND SETTING: Twenty reviewers each evaluated a 500-citation set. We compared five screening methods: conventional double screen (CDS), single screen, double screen with TM, combined double screen and single screen with TM, and single screen with TM. Rayyan, Abstrackr, and SWIFT-Review were used for each TM method. The results of a published SR were used as the reference standard. RESULTS: The mean sensitivity and specificity achieved by CDS were 97.0% (95% confidence interval [CI]: 94.7, 99.3) and 95.0% (95% CI: 93.0, 97.1). When compared with single screen, CDS provided a greater sensitivity without a decrease in specificity. Rayyan, Abstrackr, and SWIFT-Review identified all relevant studies. Specificity was often higher for TM-assisted methods than that for CDS, although with mean differences of only one-to-two percentage points. For every 500 citations not requiring manual screening, 216 minutes (95% CI: 169, 264) could be saved. CONCLUSION: TM-assisted screening methods resulted in similar sensitivity and modestly improved specificity as compared to CDS. The time saved with TM makes this a promising new tool for SR.

Association Between Frailty and Antiseizure Medication Tolerability in Older Adults With Epilepsy.

BACKGROUND AND OBJECTIVE: Frailty is an important aspect of biological aging, referring to the increased vulnerability of individuals with frailty to physical and psychological stressors. While older adults with epilepsy are an important and distinct clinical group, there are no data on frailty in this population. We hypothesize that frailty will correlate with the seizure frequency and especially the tolerability of antiseizure medications (ASMs) in older adults with epilepsy. METHODS: We recruited individuals aged 60 years or older with active epilepsy from 4 Canadian hospital centers. We reported the seizure frequency in the 3 months preceding the interview, while ASM tolerability was quantified using the Liverpool Adverse Events Profile (LAEP). We applied 3 measures of frailty: grip strength as a measure of physical frailty, 1 self-reported score (Edmonton frail score [EFS]), and 1 scale completed by a healthcare professional (clinical frailty scale [CFS]). We also administered standardized questionnaires measuring levels of anxiety, depression, functional disability, and quality of life and obtained relevant clinical and demographic data. RESULTS: Forty-three women and 43 men aged 60-93 years were recruited, 87% of whom had focal epilepsy, with an average frequency of 3.4 seizures per month. Multiple linear regression and zero-inflated negative binomial regression models showed that EFS and CFS scores were associated with decreased ASM tolerability, each point increase leading to 1.83 (95% CI: 0.67-4.30) and 2.49 (95% CI: 1.27-2.39) point increases on the LAEP scale, respectively. Neither the EFS and CFS scores nor grip strength were significantly associated with seizure frequency. The EFS was moderately correlated with depression, anxiety, quality of life, and functional disability, demonstrating the best construct validity among the 3 tested measures of frailty. DISCUSSION: The EFS was significantly, both statistically and clinically, associated with ASM tolerability. It also showed multiple advantages in performance while assessing for frailty in older adults with epilepsy, when compared with the 2 other measures of frailty that we tested. Future studies must focus on what role the EFS during epilepsy diagnosis may play in ASM selection among older adults with epilepsy.

Effect of Training on Visual Identification of High Frequency Oscillations-A Delphi-Style Intervention.

OBJECTIVE: We examined the effect of a simple Delphi-method feedback on visual identification of high frequency oscillations (HFOs) in the ripple (80-250 Hz) band, and assessed the impact of this training intervention on the interrater reliability and generalizability of HFO evaluations. METHODS: We employed a morphology detector to identify potential HFOs at two thresholds and presented them to visual reviewers to assess the probability of each epoch containing an HFO. We recruited 19 board-certified epileptologists with various levels of experience to complete a series of HFO evaluations during three sessions. A Delphi-style intervention was used to provide feedback on the performance of each reviewer relative to their peers. A delayed-intervention paradigm was used, in which reviewers received feedback either before or after the second session. ANOVAs were used to assess the effect of the intervention on the reviewers' evaluations. Generalizability theory was used to assess the interrater reliability before and after the intervention. RESULTS: The intervention, regardless of when it occurred, resulted in a significant reduction in the variability between reviewers in both groups (p GroupDI = 0.037, p GroupEI = 0.003). Prior to the delayed-intervention, the group receiving the early intervention showed a significant reduction in variability (p GroupEI = 0.041), but the delayed-intervention group did not (p GroupDI = 0.414). Following the intervention, the projected number of reviewers required to achieve strong generalizability decreased from 35 to 16. SIGNIFICANCE: This study shows a robust effect of a Delphi-style intervention on the interrater variability, reliability, and generalizability of HFO evaluations. The observed decreases in HFO marking discrepancies across 14 of the 15 reviewers are encouraging: they are necessarily associated with an increase in interrater reliability, and therefore with a corresponding decrease in the number of reviewers required to achieve strong generalizability. Indeed, the reliability of all reviewers following the intervention was similar to that of experienced reviewers prior to intervention. Therefore, a Delphi-style intervention could be implemented either to sufficiently train any reviewer, or to further refine the interrater reliability of experienced reviewers. In either case, a Delphi-style intervention would help facilitate the standardization of HFO evaluations and its implementation in clinical care.

Repeated measures discriminant analysis using multivariate generalized estimation equations.

Discriminant analysis procedures that assume parsimonious covariance and/or means structures have been proposed for distinguishing between two or more populations in multivariate repeated measures designs. However, these procedures rely on the assumptions of multivariate normality which is not tenable in multivariate repeated measures designs which are characterized by binary, ordinal, or mixed types of response distributions. This study investigates the accuracy of repeated measures discriminant analysis (RMDA) based on the multivariate generalized estimating equations (GEE) framework for classification in multivariate repeated measures designs with the same or different types of responses repeatedly measured over time. Monte Carlo methods were used to compare the accuracy of RMDA procedures based on GEE, and RMDA based on maximum likelihood estimators (MLE) under diverse simulation conditions, which included number of repeated measure occasions, number of responses, sample size, correlation structures, and type of response distribution. RMDA based on GEE exhibited higher average classification accuracy than RMDA based on MLE especially in multivariate non-normal distributions. Three repeatedly measured responses namely severity of epilepsy, current number of anti-epileptic drugs, and parent-reported quality of life in children with epilepsy were used to demonstrate the application of these procedures.

Outcomes of stereoelectroencephalography exploration at an epilepsy surgery center.

OBJECTIVES: Epilepsy surgery is offered in resistant focal epilepsy. Non-invasive investigations like scalp video EEG monitoring (SVEM) help delineate epileptogenic zone. Complex cases may require intracranial video EEG monitoring (IVEM). Stereoelectroencephalography (SEEG)-based intracerebral electrode implantation has better spatial resolution, lower morbidity, better tolerance, and superiority in sampling deep structures. Our objectives were to assess IVEM using SEEG with regard to reasoning behind implantation, course, surgical interventions, and outcomes. MATERIALS AND METHODS: Seventy-two admissions for SEEG from January 2014 to December 2018 were included in the study. Demographic and clinical data were retrospectively collected. RESULTS: The cohort comprised of 69 adults of which 34 (47%) had lesional MRI. Reasons for SEEG considering all cases included non-localizing ictal onset (76%), ictal-interictal discordance (21%), discordant semiology (17%), proximity to eloquent cortex (33%), nuclear imaging discordance (34%), and discordance with neuropsychology (19%). Among lesional cases, additional reasons included SVEM discordance (68%) and dual or multiple pathology (47%). Forty-eight patients (67%) were offered resective surgery, and 41 underwent it. Twenty-three (56%) had at least one year post-surgical follow-up of which 14 (61%) had Engels class I outcome. Of the remaining 23 who were continued on medical management, 4 (17%) became seizure-free and 12 (51%) had reduction in seizure frequency. CONCLUSION: SEEG monitoring is an important and safe tool for presurgical evaluation with good surgical and non-surgical outcomes. Whether seizure freedom following non-surgical management could be related to SEEG implantation, medication change, or natural course needs to be determined.

Cannabis use is both independently associated with and mediates worse psychosocial health in patients with epilepsy.

OBJECTIVE: To examine whether cannabis use is associated with or mediates psychosocial health in people with epilepsy. METHODS: Consecutive adult epilepsy patients visiting the Calgary Comprehensive Epilepsy Programme clinic were administered validated patient-reported outcome measures (PROMs) including the Neurological Disorders Depression Inventory for Epilepsy (NDDI-E), Quality of Life in Epilepsy (QOLIE-10-P), EuroQOL five dimensions five level scale (EQ-5D-5L), Global Assessment of Severity of Epilepsy Scale, Global Assessment of Disability Associated with Seizures Scale and the Treatment Satisfaction Questionnaire for Medication scale. We used multiple regression analyses to investigate associations between cannabis use and PROMs. Mediation analyses were performed to determine the degree to which cannabis modulated the associations between current or past psychiatric disorders, monthly seizure frequency, and 1-year seizure freedom on psychosocial health. RESULTS: Of 337 consecutive patients, 71 (21%) reported cannabis use. Cannabis use was independently associated with depression (NDDI-E score≥14; OR 3.90; 95% CI 2.01 to 7.59; p<0.001), lower quality of life (ß=-16.73, 95% CI - 26.26 to - 7.20; p=0.001), worse epilepsy-related disability (OR 2.23, 95% CI 1.19 to 4.17; p=0.01) and lower satisfaction with antiepileptic medication (OR 0.41, 95% CI 0.23 to 0.72; p=0.002). Cannabis use mediates 7%-12% of the effect of a psychiatric history on depression, worse quality of life and worse health valuation. CONCLUSIONS: There is a strong and independent association between cannabis use and poor psychosocial health, and it partially mediates the deleterious effect of a psychiatric history on these same outcomes. Inclusion of PROMs in future cannabis trials is warranted.

The "natural" history of medically treated temporal lobe epilepsy: what can an evidence-based approach tell us?

We systematically reviewed the literature to describe the "natural" history of medically treated temporal lobe epilepsy (TLE). No population-based studies recruiting incident cases of TLE irrespective of age exist. Prospective, population-based studies were limited to those recruiting only childhood-onset TLE or those reporting TLE as a subgroup of cohorts of focal epilepsies. Few studies have been performed in the "MRI era" limiting information on natural history secondary to specific pathologies. Available data suggests that TLE is highly variable, with unpredictable transient remissions and low rates of seizure freedom (30 to 50%). Etiology and failure of first and second drug seem to be the most important predictors for treatment prognosis. The role of initial precipitating injuries remains speculative, as imaging information of related events is either missing or conflicting. Prospective cohorts of new-onset TLE with long-term followup using advanced MRI techniques, timely EEG recordings, and assessments of psychiatric comorbidities are needed.