Use of troponins in the classification of myocardial infarction from electronic health records. The Atherosclerosis Risk in Communities (ARIC) Study.

OBJECTIVE: Electronic health record (EHR) data are underutilized for abstracting classification criteria for heart disease. We compared extraction of EHR data on troponin I and T levels with human abstraction. METHODS: Using EHR for hospitalizations identified through the Atherosclerosis Risk in Communities (ARIC) Study in four US hospitals, we compared blood levels of troponins I and T extracted from EHR structured data elements with levels obtained through data abstraction by human abstractors to 3 decimal places. Observations were divided randomly 50/50 into training and validation sets. Bayesian multilevel logistic regression models were used to estimate agreement by hospital in first and maximum troponin levels, troponin assessment date, troponin upper limit of normal (ULN), and classification of troponin levels as normal (< ULN), equivocal (1-2× ULN), abnormal (>2× ULN), or missing. RESULTS: Estimated overall agreement in first measured troponin level in the validation data was 88.2% (95% credible interval: 65.0%-97.5%) and 95.5% (91.2-98.2%) for the maximum troponin level observed during hospitalization. The largest variation in probability of agreement was for first troponin measured, which ranged from 66.4% to 95.8% among hospitals. CONCLUSION: Extraction of maximum troponin values during a hospitalization from EHR structured data is feasible and accurate.

Emergency Medical Text Classifier: New system improves processing and classification of triage notes.

OBJECTIVE: Automated syndrome classification aims to aid near real-time syndromic surveillance to serve as an early warning system for disease outbreaks, using Emergency Department (ED) data. We present a system that improves the automatic classification of an ED record with triage note into one or more syndrome categories using the vector space model coupled with a 'learning' module that employs a pseudo-relevance feedback mechanism. MATERIALS AND METHODS: Terms from standard syndrome definitions are used to construct an initial reference dictionary for generating the syndrome and triage note vectors. Based on cosine similarity between the vectors, each record is classified into a syndrome category. We then take terms from the top-ranked records that belong to the syndrome of interest as feedback. These terms are added to the reference dictionary and the process is repeated to determine the final classification. The system was tested on two different datasets for each of three syndromes: Gastro-Intestinal (GI), Respiratory (Resp) and Fever-Rash (FR). Performance was measured in terms of sensitivity (Se) and specificity (Sp). RESULTS: The use of relevance feedback produced high values of sensitivity and specificity for all three syndromes in both test sets: GI: 90% and 71%, Resp: 97% and 73%, FR: 100% and 87%, respectively, in test set 1, and GI: 88% and 69%, Resp: 87% and 61%, FR: 97% and 71%, respectively, in test set 2. CONCLUSIONS: The new system for pre-processing and syndromic classification of ED records with triage notes achieved improvements in Se and Sp. Our results also demonstrate that the system can be tuned to achieve different levels of performance based on user requirements.

Implementation of Emergency Medical Text Classifier for syndromic surveillance.

Public health officials use syndromic surveillance systems to facilitate early detection and response to infectious disease outbreaks. Emergency department clinical notes are becoming more available for surveillance but present the challenge of accurately extracting concepts from these text data. The purpose of this study was to implement a new system, Emergency Medical Text Classifier (EMT-C), into daily production for syndromic surveillance and evaluate system performance and user satisfaction. The system was designed to meet user preferences for a syndromic classifier that maximized positive predictive value and minimized false positives in order to provide a manageable workload. EMT-C performed better than the baseline system on all metrics and users were slightly more satisfied with it. It is vital to obtain user input and test new systems in the production environment.

Integration of syndromic surveillance data into public health practice at state and local levels in North Carolina.

OBJECTIVES: We sought to describe the integration of syndromic surveillance data into daily surveillance practice at local health departments (LHDs) and make recommendations for the effective integration of syndromic and reportable disease data for public health use. METHODS: Structured interviews were conducted with local health directors and communicable disease nursing staff from a stratified random sample of LHDs from May through September 2009. Interviews captured information on direct access to the North Carolina syndromic surveillance system and on the use of syndromic surveillance information for outbreak management, program management, and the creation of reports. We analyzed syndromic surveillance system data to assess the number of signals resulting in a public health response. RESULTS: Syndromic surveillance data were used for outbreak investigation (19% of respondents) and program management and report writing (43% of respondents); a minority reported use of both syndromic and reportable disease data for these purposes (15% and 23%, respectively). Receiving data from frequent system users was associated with using data for these purposes (p=0.016 and p=0.033, respectively, for syndromic and reportable disease data). A small proportion of signals (<25%) resulted in a public health response. CONCLUSIONS: Use of syndromic surveillance data by North Carolina local public health authorities resulted in meaningful public health action, including both case investigation and program management. While useful, the syndromic surveillance data system was oriented toward sensitivity rather than efficiency. Successful incorporation of new surveillance data is likely to require systems that are oriented toward efficiency.

TN-TIES: A system for extracting temporal information from emergency department triage notes.

The triage note field of the Emergency Department (ED) patient record describes the reason for the patient's visit, including specific symptoms and incidents. Here we present the Triage Note Temporal Information Extraction System (TN-TIES), which systematically processes triage note text and outputs a human and machine readable interpretation of the timing of the events leading up to the ED visit. TN-TIES consists of chunking, classification, and interpretation processing stages. The results at each stage are promising. This system is a first step towards a complete interpretation and timeline presentation of all events that occurred before a patients visit to the ED, which could help clinicians, public health officials, and others understand and visualize the data.

Toward vocabulary control for chief complaint.

The chief complaint (CC) is the data element that documents the patient's reason for visiting the emergency department (ED). The need for a CC vocabulary has been acknowledged at national meetings and in multiple publications, but to our knowledge no groups have specifically focused on the requirements and development plans for a CC vocabulary. The national consensus meeting "Towards Vocabulary Control for Chief Complaint" was convened to identify the potential uses for ED CC and to develop the framework for CC vocabulary control. The 10-point consensus recommendations for action were 1) begin to develop a controlled vocabulary for CC, 2) obtain funding, 3) establish an infrastructure, 4) work with standards organizations, 5) address CC vocabulary characteristics for all user communities, 6) create a collection of CC for research, 7) identify the best candidate vocabulary for ED CCs, 8) conduct vocabulary validation studies, 9) establish beta test sites, and 10) plan publicity and marketing for the vocabulary.

Unified medical language system coverage of emergency-medicine chief complaints.

BACKGROUND: Emergency department (ED) chief-complaint (CC) data increasingly are important for clinical-care and secondary uses such as syndromic surveillance. There is no widely used ED CC vocabulary, but experts have suggested evaluation of existing health-care vocabularies for ED CC. OBJECTIVES: To evaluate the ED CC coverage in existing biomedical vocabularies from the Unified Medical Language System (UMLS). METHODS: The study sample included all CC entries for all visits to three EDs over one year. The authors used a special-purpose text processor to clean CC entries, which then were mapped to UMLS concepts. The UMLS match rates then were calculated and analyzed for matching concepts and nonmatching entries. RESULTS: A total of 203,509 ED visits was included. After cleaning with the text processor, 82% of the CCs matched a UMLS concept. The authors identified 5,617 unique UMLS concepts in the ED CC data, but many were used for only one or two visits. One thousand one hundred thirty-six CC concepts were used more than ten times and covered 99% of all the ED visits. The largest biomedical vocabulary in the UMLS is the Systematized Nomenclature of Medicine Clinical Terms (SNOMED CT), which included concepts for 79% of all ED CC entries. However, some common CCs were not found in SNOMED CT. CONCLUSIONS: The authors found that ED CC concepts are well covered by the UMLS and that the best source of vocabulary coverage is from SNOMED CT. There are some gaps in UMLS and SNOMED CT coverage of ED CCs. Future work on vocabulary control for ED CCs should build upon existing vocabularies.

Evaluation of emergency medical text processor, a system for cleaning chief complaint text data.

OBJECTIVES: Emergency Medical Text Processor (EMT-P) version 1, a natural language processing system that cleans emergency department text (e.g., chst pn, chest pai), was developed to maximize extraction of standard terms (e.g., chest pain). The authors compared the number of standard terms extracted from raw chief complaint (CC) data with that for CC data cleaned with EMT-P and evaluated the accuracy of EMT-P. METHODS: This cross-sectional observation study included CC text entries for all emergency department visits to three tertiary care centers in 2001. Terms were extracted from CC entries before and after cleaning with EMT-P. Descriptive statistics included number and percentage of all entries (tokens) and all unique entries (types) that matched a standard term from the Unified Medical Language System (UMLS). An expert panel rated the accuracy of the CC-UMLS term matches; inter-rater reliability was measured with kappa. RESULTS: The authors collected 203,509 CC entry tokens, of which 63,946 were unique entry types. For the raw data, 89,337 tokens (44%) and 5,081 types (8%) matched a standard term. After EMT-P cleaning, 168,050 tokens (83%) and 44,430 types (69%) matched a standard term. The expert panel reached consensus on 201 of the 222 CC-UMLS term matches reviewed (kappa=0.69-0.72). Ninety-six percent of the 201 matches were rated equivalent or related. Thirty-eight percent of the nonmatches were found to match UMLS concepts. CONCLUSIONS: EMT-P version 1 is relatively accurate, and cleaning with EMT-P improved the CC-UMLS term match rate over raw data. The authors identified areas for improvement in future EMT-P versions and issues to be resolved in developing a standard CC terminology.

Using nurses' natural language entries to build a concept-oriented terminology for patients' chief complaints in the emergency department.

Information about the chief complaint (CC), also known as the patient's reason for seeking emergency care, is critical for patient prioritization for treatment and determination of patient flow through the emergency department (ED). Triage nurses document the CC at the start of the ED visit, and the data are increasingly available in electronic form. Despite the clinical and operational significance of the CC to the ED, there is no standard CC terminology. We propose the construction of concept-oriented nursing terminologies from the actual language used by experts. We use text analysis to extract CC concepts from triage nurses' natural language entries. Our methodology for building the nursing terminology utilizes natural language processing techniques and the Unified Medical Language System.

Diagnosis clusters for emergency medicine.

OBJECTIVES: Aggregated emergency department (ED) data are useful for research, ED operations, and public health surveillance. Diagnosis data are widely available as The International Classification of Diseases, version, 9, Clinical Modification (ICD-9-CM) codes; however, there are over 24,000 ICD-9-CM code-descriptor pairs. Standardized groupings (clusters) of ICD-9-CM codes have been developed by other disciplines, including family medicine (FM), internal medicine (IM), inpatient care (Agency for Healthcare Research and Quality [AHRQ]), and vital statistics (NCHS). The purpose of this study was to evaluate the coverage of four existing ICD-9-CM cluster systems for emergency medicine. METHODS: In this descriptive study, four cluster systems were used to group ICD-9-CM final diagnosis data from a southeastern university tertiary referral center. Included were diagnoses for all ED visits in July 2000 and January 2001. In the comparative analysis, the authors determined the coverage in the four cluster systems, defined as the proportion of final diagnosis codes that were placed into clusters and the frequencies of diagnosis codes in each cluster. RESULTS: The final sample included 7,543 visits with 19,530 diagnoses. Coverage of the ICD-9-CM codes in the ED sample was: AHRQ, 99%; NCHS, 88%; FM, 71%; IM, 68%. Seventy-six percent of the AHRQ clusters were small, defined as grouping <1% of the diagnosis codes in the sample. CONCLUSIONS: The AHRQ system provided the best coverage of ED ICD-9-CM codes. However, most of the clusters were small and not significantly different from the raw data.

Emergency Department data for bioterrorism surveillance: electronic data availability, timeliness, sources and standards.

Emergency Department (ED) data are a key component of bioterrorism surveillance systems. Little research has been done to examine differences in ED data capture and entry across hospitals, regions and states. The purpose of this study was to describe the current state of ED data for use in bioterrorism surveillance in 2 regions of the country. We found that chief complaint (CC) data are available electronically in 54% of the North Carolina EDs surveyed, and in 100% of the Seattle area EDs. Over half of all EDs reported that CCs are recorded in free text form. Though all EDs have electronic diagnosis data, less than half report that diagnoses are coded within 24 hours of the ED visit.