Leveraging Natural Language Processing to Improve Electronic Health Record Suicide Risk Prediction for Veterans Health Administration Users.

Background: Suicide risk prediction models frequently rely on structured electronic health record (EHR) data, including patient demographics and health care usage variables. Unstructured EHR data, such as clinical notes, may improve predictive accuracy by allowing access to detailed information that does not exist in structured data fields. To assess comparative benefits of including unstructured data, we developed a large case-control dataset matched on a state-of-the-art structured EHR suicide risk algorithm, utilized natural language processing (NLP) to derive a clinical note predictive model, and evaluated to what extent this model provided predictive accuracy over and above existing predictive thresholds.Methods: We developed a matched case-control sample of Veterans Health Administration (VHA) patients in 2017 and 2018. Each case (all patients that died by suicide in that interval, n = 4,584) was matched with 5 controls (patients who remained alive during treatment year) who shared the same suicide risk percentile. All sample EHR notes were selected and abstracted using NLP methods. We applied machine-learning classification algorithms to NLP output to develop predictive models. We calculated area under the curve (AUC) and suicide risk concentration to evaluate predictive accuracy overall and for high-risk patients.Results: The best performing NLP-derived models provided 19% overall additional predictive accuracy (AUC = 0.69; 95% CI, 0.67, 0.72) and 6-fold additional risk concentration for patients at the highest risk tier (top 0.1%), relative to the structured EHR model.Conclusions: The NLP-supplemented predictive models provided considerable benefit when compared to conventional structured EHR models. Results support future structured and unstructured EHR risk model integrations.

Dynamic suicide topic modelling: Deriving population-specific, psychosocial and time-sensitive suicide risk variables from Electronic Health Record psychotherapy notes.

In the machine learning subfield of natural language processing, a topic model is a type of unsupervised method that is used to uncover abstract topics within a corpus of text. Dynamic topic modelling (DTM) is used for capturing change in these topics over time. The study deploys DTM on corpus of electronic health record psychotherapy notes. This retrospective study examines whether DTM helps distinguish closely matched patients that did and did not die by suicide. Cohort consists of United States Department of Veterans Affairs (VA) patients diagnosed with Posttraumatic Stress Disorder (PTSD) between 2004 and 2013. Each case (those who died by suicide during the year following diagnosis) was matched with five controls (those who remained alive) that shared psychotherapists and had similar suicide risk based on VA's suicide prediction algorithm. Cohort was restricted to patients who received psychotherapy for 9+ months after initial PTSD diagnoses (cases = 77; controls = 362). For cases, psychotherapy notes from diagnosis until death were examined. For controls, psychotherapy notes from diagnosis until matched case's death date were examined. A Python-based DTM algorithm was utilized. Derived topics identified population-specific themes, including PTSD, psychotherapy, medication, communication and relationships. Control topics changed significantly more over time than case topics. Topic differences highlighted engagement, expressivity and therapeutic alliance. This study strengthens groundwork for deriving population-specific, psychosocial and time-sensitive suicide risk variables.

Association between post-traumatic stress disorder severity and death by suicide in US military veterans: retrospective cohort study.

BACKGROUND: There is mixed evidence regarding the direction of a potential association between post-traumatic stress disorder (PTSD) and suicide mortality. AIMS: This is the first population-based study to account for both PTSD diagnosis and PTSD symptom severity simultaneously in the examination of suicide mortality. METHOD: Retrospective study that included all US Department of Veterans Affairs (VA) patients with a PTSD diagnosis and at least one symptom severity assessment using the PTSD Checklist (PCL) between 1 October 1999 and 31 December 2018 (n = 754 197). We performed multivariable proportional hazards regression models using exposure groups defined by level of PTSD symptom severity to estimate suicide mortality rates. For patients with multiple PCL scores, we performed additional models using exposure groups defined by level of change in PTSD symptom severity. We assessed suicide mortality using the VA/Department of Defense Mortality Data Repository. RESULTS: Any level of PTSD symptoms above the minimum threshold for symptomatic remission (i.e. PCL score >18) was associated with double the suicide mortality rate at 1 month after assessment. This relationship decreased over time but patients with moderate to high symptoms continued to have elevated suicide rates. Worsening PTSD symptoms were associated with a 25% higher long-term suicide mortality rate. Among patients with improved PTSD symptoms, those with symptomatic remission had a substantial and sustained reduction in the suicide rate compared with those without symptomatic remission (HR = 0.56; 95% CI 0.37-0.88). CONCLUSIONS: Ameliorating PTSD can reduce risk of suicide mortality, but patients must achieve symptomatic remission to attain this benefit.

Leveraging unstructured electronic medical record notes to derive population-specific suicide risk models.

Electronic medical record (EMR)-based suicide risk prediction methods typically rely on analysis of structured variables such as demographics, visit history, and prescription data. Leveraging unstructured EMR notes may improve predictive accuracy by allowing access to nuanced clinical information. We utilized natural language processing (NLP) to analyze a large EMR note corpus to develop a data-driven suicide risk prediction model. We developed a matched case-control sample of U.S. Department of Veterans Affairs (VA) patients in 2015 and 2016. We randomly matched each case (all patients that died by suicide in that interval, n = 5029) with five controls (patients that remained alive). We processed note corpus using NLP methods and applied machine-learning classification algorithms to output. We calculated area under the curve (AUC) and risk tiers to determine predictive accuracy. NLP-derived models demonstrated strong predictive accuracy. Patients that scored within top 10% of risk model accounted for up to 29% of suicide decedents. NLP-derived model compares positively to other leading prediction methods. Our approach is highly implementable, only requiring access to text data and open-source software. Additional studies should evaluate ensemble models incorporating NLP-derived information alongside more typical structured variables.

Improvements to PTSD quality metrics with natural language processing.

RATIONALE AIMS AND OBJECTIVES: As quality measurement becomes increasingly reliant on the availability of structured electronic medical record (EMR) data, clinicians are asked to perform documentation using tools that facilitate data capture. These tools may not be available, feasible, or acceptable in all clinical scenarios. Alternative methods of assessment, including natural language processing (NLP) of clinical notes, may improve the completeness of quality measurement in real-world practice. Our objective was to measure the quality of care for a set of evidence-based practices using structured EMR data alone, and then supplement those measures with additional data derived from NLP. METHOD: As a case example, we studied the quality of care for posttraumatic stress disorder (PTSD) in the United States Department of Veterans Affairs (VA) over a 20-year period. We measured two aspects of PTSD care, including delivery of evidence-based psychotherapy (EBP) and associated use of measurement-based care (MBC), using structured EMR data. We then recalculated these measures using additional data derived from NLP of clinical note text. RESULTS: There were 2 098 389 VA patients with a diagnosis of PTSD between 2000 and 2019, 72% (n = 1 515 345) of whom had not previously received EBP for PTSD and were treated after a 2015 mandate to document EBP using templates that generate structured EMR data. Using structured EMR data, we determined that 3.2% (n = 48 004) of those patients met our EBP for PTSD quality standard between 2015 and 2019, and 48.1% (n = 23 088) received associated MBC. With the addition of NLP-derived data, estimates increased to 4.1% (n = 62 789) and 58.0% (n = 36 435), respectively. CONCLUSION: Healthcare quality data can be significantly improved by supplementing structured EMR data with NLP-derived data. By using NLP, health systems may be able to fill the gaps in documentation when structured tools are not yet available or there are barriers to using them in clinical practice.

Patient and Clinical Factors Associated With Response to Medications for Posttraumatic Stress Disorder.

Objective: Fluoxetine, paroxetine, sertraline, topiramate, and venlafaxine have previously shown efficacy for posttraumatic stress disorder (PTSD) in randomized clinical trials. Two prior studies using Department of Veterans Affairs (VA) medical records data show these medications are also effective in routine practice. Using an expanded retrospective cohort, we assessed the possibility of differential patterns of response based on patient and clinical factors.Methods: We identified 6,839 VA outpatients with clinical diagnoses of PTSD between October 1999 and September 2019 who initiated one of the medications and met pre-specified criteria for treatment duration and dose, combined with baseline and endpoint PTSD checklist (PCL) measurements. We compared 12-week changes in PCL score within clinical subgroups defined by sex, race and ethnicity, and military exposures, as well as comorbidities. Comorbidities were identified using International Classification of Diseases diagnostic codes and grouped according to major diagnostic classifications in the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (eg, Psychotic Disorders, Depressive Disorders). We used a propensity score weighting approach to balance covariates among medication arms within each clinical subgroup. In our exploratory analyses using unweighted data for the overall cohort, we built penalized logistic regression models to identify covariates that predicted meaningful improvement.Results: There were no significant differences between medications in our weighted subgroup analyses. In unweighted exploratory analyses, higher baseline PCL scores and concurrent receipt of evidence-based psychotherapy predicted meaningful improvement, while high levels of disability predicted not realizing meaningful improvement.Conclusions: In the largest real-world study of medications for PTSD to date, we did not observe a pattern of differential response among clinical subgroups. All patients taking medications for PTSD, especially those with the highest levels of disability, should consider combined treatment with evidence-based psychotherapy.

Geographic Variation in Initiation of Evidence-based Psychotherapy Among Veterans With PTSD.

INTRODUCTION: The United States Department of Veterans Affairs (VA) has invested in implementation of evidence-based psychotherapy (EBP) for post-traumatic stress disorder (PTSD) for over a decade, resulting in slow but steady uptake of these treatments nationally. However, no prior research has investigated the geographic variation in initiation of EBP. Our objectives were to determine whether there is geographic variation in the initiation of EBP for PTSD in the VA and to identify patient and clinic factors associated with EBP initiation. MATERIALS AND METHODS: We identified VA patients with PTSD who had not received EBP as of January 2016 (N = 946,667) using retrospective electronic medical records data and determined whether they initiated EBP by December 2017. We illustrated geographic variation in EBP initiation using national and regional maps. Using multivariate logistic regression, we determined patient, regional, and nearest VA facility predictors of initiating treatment. This study was approved by the Veterans Institutional Review Board of Northern New England. RESULTS: Nationally, 4.8% (n = 45,895) initiated EBP from 2016 to 2017, and there was geographic variation, ranging from none to almost 30% at the 3-digit ZIP code level. The strongest patient predictors of EBP initiation were the negative predictor of being older than 65 years (OR = 0.47; 95% CI, 0.45-0.49) and the positive predictor of reporting military-related sexual trauma (OR = 1.96; 95% CI, 1.90-2.03). The strongest regional predictors of EBP initiation were the negative predictor of living in the Northeast (OR = 0.89; 95% CI, 0.86-0.92) and the positive predictor of living in the Midwest (OR = 1.47; 95% CI, 1.44-1.51). The only nearest VA facility predictor of EBP initiation was the positive predictor of whether the facility was a VA Medical Center with a specialized PTSD clinic (OR = 1.23; 95% CI, 1.20-1.26). CONCLUSION: Although less than 5% of VA patients with PTSD initiated EBP, there was regional variation. Patient factors, region of residence, and nearest VA facility characteristics were all associated with whether patients initiated EBP. Strengths of this study include the use of national longitudinal data, while weaknesses include the potential for misclassification of PTSD diagnoses as well as the potential for misidentification of EBP. Our work indicates geographic areas where access to EBP for PTSD may be poor and can help target work improving access. Future studies should also assess completion of EBP for PTSD and related symptomatic and functional outcomes across geographic areas.

Adverse behavioral and emotional outcomes from child abuse and witnessed violence.

This article examines mental health outcomes of children who have witnessed violence in their social environment and/or have been physically abused. Participants (n = 167) come from a longitudinal study on child maltreatment. Outcomes-including depression, anger, and anxiety--are measured by the Child Behavior Checklist and the Trauma Symptom Checklist for Children. The authors used adjusted multivariate analyses to test the statistical significance of associations. The majority of children were female (57%) and non-White (64%). One third had been physically victimized; 46% had witnessed moderate-high levels of violence. Results confirm that children are negatively affected by victimization and violence they witness in their homes and neighborhoods. Victimization was a significant predictor of child aggression and depression; witnessed violence was found to be a significant predictor of aggression, depression, anger, and anxiety. Implications will be discussed.