Artificial Intelligence Assesses Clinicians' Adherence to Asthma Guidelines Using Electronic Health Records.

BACKGROUND: Clinicians' asthma guideline adherence in asthma care is suboptimal. The effort to improve adherence can be enhanced by assessing and monitoring clinicians' adherence to guidelines reflected in electronic health records (EHRs), which require costly manual chart review because many care elements cannot be identified by structured data. OBJECTIVE: This study was designed to demonstrate the feasibility of an artificial intelligence tool using natural language processing (NLP) leveraging the free text EHRs of pediatric patients to extract key components of the 2007 National Asthma Education and Prevention Program guidelines. METHODS: This is a retrospective cross-sectional study using a birth cohort with a diagnosis of asthma at Mayo Clinic between 2003 and 2016. We used 1,039 clinical notes with an asthma diagnosis from a random sample of 300 patients. Rule-based NLP algorithms were developed to identify asthma guideline-congruent elements by examining care description in EHR free text. RESULTS: Natural language processing algorithms demonstrated a sensitivity (0.82-1.0), specificity (0.95-1.0), positive predictive value (0.86-1.0), and negative predictive value (0.92-1.0) against manual chart review for asthma guideline-congruent elements. Assessing medication compliance and inhaler technique assessment were the most challenging elements to assess because of the complexity and wide variety of descriptions. CONCLUSIONS: Natural language processing technologies may enable the automated assessment of clinicians' documentation in EHRs regarding adherence to asthma guidelines and can be a useful population management and research tool to assess and monitor asthma care quality. Multisite studies with a larger sample size are needed to assess the generalizability of these NLP algorithms.

Identification of asthma control factor in clinical notes using a hybrid deep learning model.

BACKGROUND: There are significant variabilities in guideline-concordant documentation in asthma care. However, assessing clinician's documentation is not feasible using only structured data but requires labor-intensive chart review of electronic health records (EHRs). A certain guideline element in asthma control factors, such as review inhaler techniques, requires context understanding to correctly capture from EHR free text. METHODS: The study data consist of two sets: (1) manual chart reviewed data-1039 clinical notes of 300 patients with asthma diagnosis, and (2) weakly labeled data (distant supervision)-27,363 clinical notes from 800 patients with asthma diagnosis. A context-aware language model, Bidirectional Encoder Representations from Transformers (BERT) was developed to identify inhaler techniques in EHR free text. Both original BERT and clinical BioBERT (cBERT) were applied with a cost-sensitivity to deal with imbalanced data. The distant supervision using weak labels by rules was also incorporated to augment the training set and alleviate a costly manual labeling process in the development of a deep learning algorithm. A hybrid approach using post-hoc rules was also explored to fix BERT model errors. The performance of BERT with/without distant supervision, hybrid, and rule-based models were compared in precision, recall, F-score, and accuracy. RESULTS: The BERT models on the original data performed similar to a rule-based model in F1-score (0.837, 0.845, and 0.838 for rules, BERT, and cBERT, respectively). The BERT models with distant supervision produced higher performance (0.853 and 0.880 for BERT and cBERT, respectively) than without distant supervision and a rule-based model. The hybrid models performed best in F1-score of 0.877 and 0.904 over the distant supervision on BERT and cBERT. CONCLUSIONS: The proposed BERT models with distant supervision demonstrated its capability to identify inhaler techniques in EHR free text, and outperformed both the rule-based model and BERT models trained on the original data. With a distant supervision approach, we may alleviate costly manual chart review to generate the large training data required in most deep learning-based models. A hybrid model was able to fix BERT model errors and further improve the performance.

Automated Detection of Periprosthetic Joint Infections and Data Elements Using Natural Language Processing.

BACKGROUND: Periprosthetic joint infection (PJI) data elements are contained in both structured and unstructured documents in electronic health records and require manual data collection. The goal of this study is to develop a natural language processing (NLP) algorithm to replicate manual chart review for PJI data elements. METHODS: PJI was identified among all total joint arthroplasty (TJA) procedures performed at a single academic institution between 2000 and 2017. Data elements that comprise the Musculoskeletal Infection Society (MSIS) criteria were manually extracted and used as the gold standard for validation. A training sample of 1208 TJA surgeries (170 PJI cases) was randomly selected to develop the prototype NLP algorithms and an additional 1179 surgeries (150 PJI cases) were randomly selected as the test sample. The algorithms were applied to all consultation notes, operative notes, pathology reports, and microbiology reports to predict the correct status of PJI based on MSIS criteria. RESULTS: The algorithm, which identified patients with PJI based on MSIS criteria, achieved an f1-score (harmonic mean of precision and recall) of 0.911. Algorithm performance in extracting the presence of sinus tract, purulence, pathologic documentation of inflammation, and growth of cultured organisms from the involved TJA achieved f1-scores that ranged from 0.771 to 0.982, sensitivity that ranged from 0.730 to 1.000, and specificity that ranged from 0.947 to 1.000. CONCLUSION: NLP-enabled algorithms have the potential to automate data collection for PJI diagnostic elements, which could directly improve patient care and augment cohort surveillance and research efforts. Further validation is needed in other hospital settings. LEVEL OF EVIDENCE: Level III, Diagnostic.

Use of Natural Language Processing Algorithms to Identify Common Data Elements in Operative Notes for Knee Arthroplasty.

BACKGROUND: Natural language processing (NLP) methods have the capability to process clinical free text in electronic health records, decreasing the need for costly manual chart review, and improving data quality. We developed rule-based NLP algorithms to automatically extract surgery specific data elements from knee arthroplasty operative notes. METHODS: Within a cohort of 20,000 knee arthroplasty operative notes from 2000 to 2017 at a large tertiary institution, we randomly selected independent pairs of training and test sets to develop and evaluate NLP algorithms to detect five major data elements. The size of the training and test datasets were similar and ranged between 420 to 1592 surgeries. Expert rules using keywords in operative notes were used to implement NLP algorithms capturing: (1) category of surgery (total knee arthroplasty, unicompartmental knee arthroplasty, patellofemoral arthroplasty), (2) laterality of surgery, (3) constraint type, (4) presence of patellar resurfacing, and (5) implant model (catalog numbers). We used institutional registry data as our gold standard to evaluate the NLP algorithms. RESULTS: NLP algorithms to detect the category of surgery, laterality, constraint, and patellar resurfacing achieved 98.3%, 99.5%, 99.2%, and 99.4% accuracy on test datasets, respectively. The implant model algorithm achieved an F1-score (harmonic mean of precision and recall) of 99.9%. CONCLUSIONS: NLP algorithms are a promising alternative to costly manual chart review to automate the extraction of embedded information within knee arthroplasty operative notes. Further validation in other hospital settings will enhance widespread implementation and efficiency in data capture for research and clinical purposes. LEVEL OF EVIDENCE: Level III.

Deep Learning Identification of Asthma Inhaler Techniques in Clinical Notes.

There are significant variabilities in clinicians' guideline-concordant documentation in asthma care. However, assessing clinicians' documentation is not feasible using only structured data but requires labor intensive chart review of electronic health records. Although the national asthma guidelines are available it is still challenging to use them as a real-time tool for providing feedback on adhering documentation guidelines for asthma care improvement. A certain guideline element, such as teaching or reviewing inhaler techniques, is difficult to capture by handcrafted rules since it requires contextual understanding of clinical narratives. This study examined a deep learning based natural language model, Bidirectional Encoder Representations from Transformers (BERT) coupled with distant supervision to identify inhaler techniques from clinical narratives. The BERT model with distant supervision outperformed the rule-based approach and achieved performance gain compared with the BERT without distant supervision.

Mining news media for understanding public health concerns.

INTRODUCTION: News media play an important role in raising public awareness, framing public opinions, affecting policy formulation, and acknowledgment of public health issues. Traditional qualitative content analysis for news sentiments and focuses are time-consuming and may not efficiently convey sentiments nor the focuses of news media. METHODS: We used descriptive statistics and state-of-art text mining to conduct sentiment analysis and topic modeling, to efficiently analyze over 3 million Reuters news articles during 2007-2017 for identifying their coverage, sentiments, and focuses for public health issues. Based on the top keywords from public health scientific journals, we identified 10 major public health issues (i.e., "air pollution," "alcohol drinking," "asthma," "depression," "diet," "exercise," "obesity," "pregnancy," "sexual behavior," and "smoking"). RESULTS: The news coverage for seven public health issues, "Smoking," "Exercise," "Alcohol drinking," "Diet," "Obesity," "Depression," and "Asthma" decreased over time. The news coverage for "Sexual behavior," "Pregnancy," and "Air pollution" fluctuated during 2007-2017. The sentiments of the news articles for three of the public health issues, "exercise," "alcohol drinking," and "diet" were predominately positive and associated such as "energy." Sentiments for the remaining seven public health issues were mainly negative, linked to negative terms, e.g., diseases. The results of topic modeling reflected the media's focus on public health issues. CONCLUSIONS: Text mining methods may address the limitations of traditional qualitative approaches. Using big data to understand public health needs is a novel approach that could help clinical and translational science awards programs focus on community-engaged research efforts to address community priorities.