Comparing Outcomes of the Veterans Health Administration's Traumatic Brain Injury and Mental Health Screening Programs: Types and Frequency of Specialty Services Used.

The Veterans Health Administration (VHA) screens veterans who deployed in support of the wars in Afghanistan and Iraq for traumatic brain injury (TBI) and mental health (MH) disorders. Chronic symptoms after mild TBI overlap with MH symptoms, for which there are already established screens within the VHA. It is unclear whether the TBI screen facilitates treatment for appropriate specialty care over and beyond the MH screens. Our primary objective was to determine whether TBI screening is associated with different types (MH, Physical Medicine & Rehabilitation [PM&R], and Neurology) and frequency of specialty services compared with the MH screens. A retrospective cohort design examined veterans receiving VHA care who were screened for both TBI and MH disorders between Fiscal Year (FY) 2007 and FY 2018 (N = 241,136). We calculated service utilization counts in MH, PM&R, and Neurology in the six months after the screens. Zero-inflated negative binomial regression models of encounters (counts) were fit separately by specialty care type and for a total count of specialty services. We found that screening positive for TBI resulted in 2.38 times more specialty service encounters than screening negative for TBI. Compared with screening positive for MH only, screening positive for both MH and TBI resulted in 1.78 times more specialty service encounters and 1.33 times more MH encounters. The TBI screen appears to increase use of MH, PM&R, and Neurology services for veterans with post-deployment health concerns, even in those also identified as having a possible MH disorder.

Improving identification of fall-related injuries in ambulatory care using statistical text mining.

OBJECTIVES: We determined whether statistical text mining (STM) can identify fall-related injuries in electronic health record (EHR) documents and the impact on STM models of training on documents from a single or multiple facilities. METHODS: We obtained fiscal year 2007 records for Veterans Health Administration (VHA) ambulatory care clinics in the southeastern United States and Puerto Rico, resulting in a total of 26 010 documents for 1652 veterans treated for fall-related injury and 1341 matched controls. We used the results of an STM model to predict fall-related injuries at the visit and patient levels and compared them with a reference standard based on chart review. RESULTS: STM models based on training data from a single facility resulted in accuracy of 87.5% and 87.1%, F-measure of 87.0% and 90.9%, sensitivity of 92.1% and 94.1%, and specificity of 83.6% and 77.8% at the visit and patient levels, respectively. Results from training data from multiple facilities were almost identical. CONCLUSIONS: STM has the potential to improve identification of fall-related injuries in the VHA, providing a model for wider application in the evolving national EHR system.

Using information from the electronic health record to improve measurement of unemployment in service members and veterans with mTBI and post-deployment stress.

OBJECTIVE: The purpose of this pilot study is 1) to develop an annotation schema and a training set of annotated notes to support the future development of a natural language processing (NLP) system to automatically extract employment information, and 2) to determine if information about employment status, goals and work-related challenges reported by service members and Veterans with mild traumatic brain injury (mTBI) and post-deployment stress can be identified in the Electronic Health Record (EHR). DESIGN: Retrospective cohort study using data from selected progress notes stored in the EHR. SETTING: Post-deployment Rehabilitation and Evaluation Program (PREP), an in-patient rehabilitation program for Veterans with TBI at the James A. Haley Veterans' Hospital in Tampa, Florida. PARTICIPANTS: Service members and Veterans with TBI who participated in the PREP program (N = 60). MAIN OUTCOME MEASURES: Documentation of employment status, goals, and work-related challenges reported by service members and recorded in the EHR. RESULTS: Two hundred notes were examined and unique vocational information was found indicating a variety of self-reported employment challenges. Current employment status and future vocational goals along with information about cognitive, physical, and behavioral symptoms that may affect return-to-work were extracted from the EHR. The annotation schema developed for this study provides an excellent tool upon which NLP studies can be developed. CONCLUSIONS: Information related to employment status and vocational history is stored in text notes in the EHR system. Information stored in text does not lend itself to easy extraction or summarization for research and rehabilitation planning purposes. Development of NLP systems to automatically extract text-based employment information provides data that may improve the understanding and measurement of employment in this important cohort.

TagLine: Information Extraction for Semi-Structured Text in Medical Progress Notes.

Statistical text mining and natural language processing have been shown to be effective for extracting useful information from medical documents. However, neither technique is effective at extracting the information stored in semi-structure text elements. A prototype system (TagLine) was developed to extract information from the semi-structured text using machine learning and a rule based annotator. Features for the learning machine were suggested by prior work, and by examining text, and selecting attributes that help distinguish classes of text lines. Classes were derived empirically from text and guided by an ontology developed by the VHA's Consortium for Health Informatics Research (CHIR). Decision trees were evaluated for class predictions on 15,103 lines of text achieved an overall accuracy of 98.5 percent. The class labels applied to the lines were then used for annotating semi-structured text elements. TagLine achieved F-measure over 0.9 for each of the structures, which included tables, slots and fillers.

Finding falls in ambulatory care clinical documents using statistical text mining.

OBJECTIVE: To determine how well statistical text mining (STM) models can identify falls within clinical text associated with an ambulatory encounter. MATERIALS AND METHODS: 2241 patients were selected with a fall-related ICD-9-CM E-code or matched injury diagnosis code while being treated as an outpatient at one of four sites within the Veterans Health Administration. All clinical documents within a 48-h window of the recorded E-code or injury diagnosis code for each patient were obtained (n=26 010; 611 distinct document titles) and annotated for falls. Logistic regression, support vector machine, and cost-sensitive support vector machine (SVM-cost) models were trained on a stratified sample of 70% of documents from one location (dataset Atrain) and then applied to the remaining unseen documents (datasets Atest-D). RESULTS: All three STM models obtained area under the receiver operating characteristic curve (AUC) scores above 0.950 on the four test datasets (Atest-D). The SVM-cost model obtained the highest AUC scores, ranging from 0.953 to 0.978. The SVM-cost model also achieved F-measure values ranging from 0.745 to 0.853, sensitivity from 0.890 to 0.931, and specificity from 0.877 to 0.944. DISCUSSION: The STM models performed well across a large heterogeneous collection of document titles. In addition, the models also generalized across other sites, including a traditionally bilingual site that had distinctly different grammatical patterns. CONCLUSIONS: The results of this study suggest STM-based models have the potential to improve surveillance of falls. Furthermore, the encouraging evidence shown here that STM is a robust technique for mining clinical documents bodes well for other surveillance-related topics.

Using ensemble models to classify the sentiment expressed in suicide notes.

In 2007, suicide was the tenth leading cause of death in the U.S. Given the significance of this problem, suicide was the focus of the 2011 Informatics for Integrating Biology and the Bedside (i2b2) Natural Language Processing (NLP) shared task competition (track two). Specifically, the challenge concentrated on sentiment analysis, predicting the presence or absence of 15 emotions (labels) simultaneously in a collection of suicide notes spanning over 70 years. Our team explored multiple approaches combining regular expression-based rules, statistical text mining (STM), and an approach that applies weights to text while accounting for multiple labels. Our best submission used an ensemble of both rules and STM models to achieve a micro-averaged F(1) score of 0.5023, slightly above the mean from the 26 teams that competed (0.4875).

Using ontology network structure in text mining.

Statistical text mining treats documents as bags of words, with a focus on term frequencies within documents and across document collections. Unlike natural language processing (NLP) techniques that rely on an engineered vocabulary or a full-featured ontology, statistical approaches do not make use of domain-specific knowledge. The freedom from biases can be an advantage, but at the cost of ignoring potentially valuable knowledge. The approach proposed here investigates a hybrid strategy based on computing graph measures of term importance over an entire ontology and injecting the measures into the statistical text mining process. As a starting point, we adapt existing search engine algorithms such as PageRank and HITS to determine term importance within an ontology graph. The graph-theoretic approach is evaluated using a smoking data set from the i2b2 National Center for Biomedical Computing, cast as a simple binary classification task for categorizing smoking-related documents, demonstrating consistent improvements in accuracy.