Design considerations for a hierarchical semantic compositional framework for medical natural language understanding.

Medical natural language processing (NLP) systems are a key enabling technology for transforming Big Data from clinical report repositories to information used to support disease models and validate intervention methods. However, current medical NLP systems fall considerably short when faced with the task of logically interpreting clinical text. In this paper, we describe a framework inspired by mechanisms of human cognition in an attempt to jump the NLP performance curve. The design centers on a hierarchical semantic compositional model (HSCM), which provides an internal substrate for guiding the interpretation process. The paper describes insights from four key cognitive aspects: semantic memory, semantic composition, semantic activation, and hierarchical predictive coding. We discuss the design of a generative semantic model and an associated semantic parser used to transform a free-text sentence into a logical representation of its meaning. The paper discusses supportive and antagonistic arguments for the key features of the architecture as a long-term foundational framework.

Detecting diabetic retinopathy through machine learning on electronic health record data from an urban, safety net healthcare system.

OBJECTIVE: Clinical guidelines recommend annual eye examinations to detect diabetic retinopathy (DR) in patients with diabetes. However, timely DR detection remains a problem in medically underserved and under-resourced settings in the United States. Machine learning that identifies patients with latent/undiagnosed DR could help to address this problem. MATERIALS AND METHODS: Using electronic health record data from 40 631 unique diabetic patients seen at Los Angeles County Department of Health Services healthcare facilities between January 1, 2015 and December 31, 2017, we compared ten machine learning environments, including five classifier models, for assessing the presence or absence of DR. We also used data from a distinct set of 9300 diabetic patients seen between January 1, 2018 and December 31, 2018 as an external validation set. RESULTS: Following feature subset selection, the classifier with the best AUC on the external validation set was a deep neural network using majority class undersampling, with an AUC of 0.8, the sensitivity of 72.17%, and specificity of 74.2%. DISCUSSION: A deep neural network produced the best AUCs and sensitivity results on the test set and external validation set. Models are intended to be used to screen guideline noncompliant diabetic patients in an urban safety-net setting. CONCLUSION: Machine learning on diabetic patients' routinely collected clinical data could help clinicians in safety-net settings to identify and target unscreened diabetic patients who potentially have undiagnosed DR.

Evaluating Casama: Contextualized semantic maps for summarization of lung cancer studies.

OBJECTIVE: It is crucial for clinicians to stay up to date on current literature in order to apply recent evidence to clinical decision making. Automatic summarization systems can help clinicians quickly view an aggregated summary of literature on a topic. Casama, a representation and summarization system based on "contextualized semantic maps," captures the findings of biomedical studies as well as the contexts associated with patient population and study design. This paper presents a user-oriented evaluation of Casama in comparison to a context-free representation, SemRep. MATERIALS AND METHODS: The effectiveness of the representation was evaluated by presenting users with manually annotated Casama and SemRep summaries of ten articles on driver mutations in cancer. Automatic annotations were evaluated on a collection of articles on EGFR mutation in lung cancer. Seven users completed a questionnaire rating the summarization quality for various topics and applications. RESULTS: Casama had higher median scores than SemRep for the majority of the topics (p≤ 0.00032), all of the applications (p≤ 0.00089), and in overall summarization quality (p≤ 1.5e-05). Casama's manual annotations outperformed Casama's automatic annotations (p = 0.00061). DISCUSSION: Casama performed particularly well in the representation of strength of evidence, which was highly rated both quantitatively and qualitatively. Users noted that Casama's less granular, more targeted representation improved usability compared to SemRep. CONCLUSION: This evaluation demonstrated the benefits of a contextualized representation for summarizing biomedical literature on cancer. Iteration on specific areas of Casama's representation, further development of its algorithms, and a clinically-oriented evaluation are warranted.

Medical Imaging Informatics.

Imaging is one of the most important sources of clinically observable evidence that provides broad coverage, can provide insight on low-level scale properties, is noninvasive, has few side effects, and can be performed frequently. Thus, imaging data provides a viable observable that can facilitate the instantiation of a theoretical understanding of a disease for a particular patient context by connecting imaging findings to other biologic parameters in the model (e.g., genetic, molecular, symptoms, and patient survival). These connections can help inform their possible states and/or provide further coherent evidence. The field of radiomics is particularly dedicated to this task and seeks to extract quantifiable measures wherever possible. Example properties of investigation include genotype characterization, histopathology parameters, metabolite concentrations, vascular proliferation, necrosis, cellularity, and oxygenation. Important issues within the field include: signal calibration, spatial calibration, preprocessing methods (e.g., noise suppression, motion correction, and field bias correction), segmentation of target anatomic/pathologic entities, extraction of computed features, and inferencing methods connecting imaging features to biological states.

Evaluating a Novel Summary Visualization for Clinical Trial Reports: A Usability Study.

Contributions of clinical trials are captured in published reports that are unstructured and often require extensive manual review to gain a deeper understanding of the study itself. Our goal is to increase comprehension and decrease the time necessary to understand these reports through the use of visualization tools. In this paper, we specify and evaluate the visualization of a previously developed representation as well as gain insight from user input for further development. The usability experiment consisted of a two-arm study with users either having or not having access to the visualization. A user questionnaire was used to measure time spent and accuracy in comprehension; intuitiveness and reproducibility of the visualization; and preferences. We found that having the visualization required on average 28.1% less time (25.8 min vs. 35.8 min, p=0.01) while maintaining similar accuracy (73.7% vs. 67.0%). Users were then asked to create their own visualizations, with their visualizations averaging 86.1% similar to the gold standard. All participants either preferred the visualization over the status quo or preferred both equally. These results demonstrate that novel visualizations for trial reports could provide time savings and achieve similar accuracy as reviewing the paper itself. Understanding the strength and quality of clinical trials can be alleviated with a visualization that makes content explicit.

Assessing Variability in Brain Tumor Segmentation to Improve Volumetric Accuracy and Characterization of Change.

Brain tumor analysis is moving towards volumetric assessment of magnetic resonance imaging (MRI), providing a more precise description of disease progression to better inform clinical decision-making and treatment planning. While a multitude of segmentation approaches exist, inherent variability in the results of these algorithms may incorrectly indicate changes in tumor volume. In this work, we present a systematic approach to characterize variability in tumor boundaries that utilizes equivalence tests as a means to determine whether a tumor volume has significantly changed over time. To demonstrate these concepts, 32 MRI studies from 8 patients were segmented using four different approaches (statistical classifier, region-based, edge-based, knowledge-based) to generate different regions of interest representing tumor extent. We showed that across all studies, the average Dice coefficient for the superset of the different methods was 0.754 (95% confidence interval 0.701-0.808) when compared to a reference standard. We illustrate how variability obtained by different segmentations can be used to identify significant changes in tumor volume between sequential time points. Our study demonstrates that variability is an inherent part of interpreting tumor segmentation results and should be considered as part of the interpretation process.

Toward patient-tailored summarization of lung cancer literature.

As the volume of biomedical literature increases, it can be challenging for clinicians to stay up-to-date. Graphical summarization systems help by condensing knowledge into networks of entities and relations. However, existing systems present relations out of context, ignoring key details such as study population. To better support precision medicine, summarization systems should include such information to contextualize and tailor results to individual patients. This paper introduces "contextualized semantic maps" for patient-tailored graphical summarization of published literature. These efforts are demonstrated in the domain of driver mutations in non-small cell lung cancer (NSCLC). A representation for relations and study population context in NSCLC was developed. An annotated gold standard for this representation was created from a set of 135 abstracts; F1-score annotator agreement was 0.78 for context and 0.68 for relations. Visualizing the contextualized relations demonstrated that context facilitates the discovery of key findings that are relevant to patient-oriented queries.

A formal representation for numerical data presented in published clinical trial reports.

Assessing the quality of and integrating clinical trial reports are necessary to practice evidence-based medicine. In particular, the numerical data is essential to understanding the strength and quality of the clinical trial study. In this paper, we present a formal representation for standardizing numerical data in published clinical trial reports, and our efforts towards developing computational tools to capture and visualize this representation. The approach includes two aspects: a process model used to precisely define experimental context behind the numerical value; and a spreadsheet, an intuitive and familiar tool used to organize numerical data. We demonstrated this representation using clinical trial reports on non-small cell lung cancer (NSCLC). We performed a preliminary evaluation to determine the usefulness of this formalism for identifying the characteristics, quality and significance of a clinical trial. Our initial results demonstrate that the representation is sufficiently expressive to capture reported numerical information in published papers.

Hierarchical semantic structures for medical NLP.

We present a framework for building a medical natural language processing (NLP) system capable of deep understanding of clinical text reports. The framework helps developers understand how various NLP-related efforts and knowledge sources can be integrated. The aspects considered include: 1) computational issues dealing with defining layers of intermediate semantic structures to reduce the dimensionality of the NLP problem; 2) algorithmic issues in which we survey the NLP literature and discuss state-of-the-art procedures used to map between various levels of the hierarchy; and 3) implementation issues to software developers with available resources. The objective of this poster is to educate readers to the various levels of semantic representation (e.g., word level concepts, ontological concepts, logical relations, logical frames, discourse structures, etc.). The poster presents an architecture for which diverse efforts and resources in medical NLP can be integrated in a principled way.

Imaging-based observational databases for clinical problem solving: the role of informatics.

Imaging has become a prevalent tool in the diagnosis and treatment of many diseases, providing a unique in vivo, multi-scale view of anatomic and physiologic processes. With the increased use of imaging and its progressive technical advances, the role of imaging informatics is now evolving--from one of managing images, to one of integrating the full scope of clinical information needed to contextualize and link observations across phenotypic and genotypic scales. Several challenges exist for imaging informatics, including the need for methods to transform clinical imaging studies and associated data into structured information that can be organized and analyzed. We examine some of these challenges in establishing imaging-based observational databases that can support the creation of comprehensive disease models. The development of these databases and ensuing models can aid in medical decision making and knowledge discovery and ultimately, transform the use of imaging to support individually-tailored patient care.

Imaging informatics for consumer health: towards a radiology patient portal.

OBJECTIVE: With the increased routine use of advanced imaging in clinical diagnosis and treatment, it has become imperative to provide patients with a means to view and understand their imaging studies. We illustrate the feasibility of a patient portal that automatically structures and integrates radiology reports with corresponding imaging studies according to several information orientations tailored for the layperson. METHODS: The imaging patient portal is composed of an image processing module for the creation of a timeline that illustrates the progression of disease, a natural language processing module to extract salient concepts from radiology reports (73% accuracy, F1 score of 0.67), and an interactive user interface navigable by an imaging findings list. The portal was developed as a Java-based web application and is demonstrated for patients with brain cancer. RESULTS AND DISCUSSION: The system was exhibited at an international radiology conference to solicit feedback from a diverse group of healthcare professionals. There was wide support for educating patients about their imaging studies, and an appreciation for the informatics tools used to simplify images and reports for consumer interpretation. Primary concerns included the possibility of patients misunderstanding their results, as well as worries regarding accidental improper disclosure of medical information. CONCLUSIONS: Radiologic imaging composes a significant amount of the evidence used to make diagnostic and treatment decisions, yet there are few tools for explaining this information to patients. The proposed radiology patient portal provides a framework for organizing radiologic results into several information orientations to support patient education.

Context-based electronic health record: toward patient specific healthcare.

Due to the increasingly data-intensive clinical environment, physicians now have unprecedented access to detailed clinical information from a multitude of sources. However, applying this information to guide medical decisions for a specific patient case remains challenging. One issue is related to presenting information to the practitioner: displaying a large (irrelevant) amount of information often leads to information overload. Next-generation interfaces for the electronic health record (EHR) should not only make patient data easily searchable and accessible, but also synthesize fragments of evidence documented in the entire record to understand the etiology of a disease and its clinical manifestation in individual patients. In this paper, we describe our efforts toward creating a context-based EHR, which employs biomedical ontologies and (graphical) disease models as sources of domain knowledge to identify relevant parts of the record to display. We hypothesize that knowledge (e.g., variables, relationships) from these sources can be used to standardize, annotate, and contextualize information from the patient record, improving access to relevant parts of the record and informing medical decision making. To achieve this goal, we describe a framework that aggregates and extracts findings and attributes from free-text clinical reports, maps findings to concepts in available knowledge sources, and generates a tailored presentation of the record based on the information needs of the user. We have implemented this framework in a system called Adaptive EHR, demonstrating its capabilities to present and synthesize information from neurooncology patients. This paper highlights the challenges and potential applications of leveraging disease models to improve the access, integration, and interpretation of clinical patient data.

Automated extraction of reported statistical analyses: towards a logical representation of clinical trial literature.

Randomized controlled trials are an important source of evidence for guiding clinical decisions when treating a patient. However, given the large number of studies and their variability in quality, determining how to summarize reported results and formalize them as part of practice guidelines continues to be a challenge. We have developed a set of information extraction and annotation tools to automate the identification of key information from papers related to the hypothesis, sample size, statistical test, confidence interval, significance level, and conclusions. We adapted the Automated Sequence Annotation Pipeline to map extracted phrases to relevant knowledge sources. We trained and tested our system on a corpus of 42 full-text articles related to chemotherapy of non-small cell lung cancer. On our test set of 7 papers, we obtained an overall precision of 86%, recall of 78%, and an F-score of 0.82 for classifying sentences. This work represents our efforts towards utilizing this information for quality assessment, meta-analysis, and modeling.

Improving the accuracy of therapy descriptions in clinical trials using a bottom-up approach.

Randomized clinical trial (RCT) reports commonly have complicated therapy descriptions that are written in free-text. Drug therapy is difficult to describe due to the dynamic nature of how protocols change and the many ways drugs can be administered. Details regarding protocol changes and drug administration must be explained clearly for reproducibility and reliability. A process model supplemented with concept ontologies can clarify the dynamics of how therapies change and make knowledge more explicit. We demonstrated the process to develop a representation model to reveal specific context concerning drug therapies within clinical trial report literature. A PubMed search was conducted to identify RCTs on non-small-cell lung cancer (NSCLC) pertaining to epithelial growth factor receptor (EGFR) mutations. Twenty-seven clinical trials were used to develop the model using a bottom-up approach. This representation describes drug dosage, administration details, and drug cycles within different experimental arms and control groups. We then presented preliminary evaluation of the clarity and understandability of the representation.

Natural language processing with dynamic classification improves P300 speller accuracy and bit rate.

The P300 speller is an example of a brain-computer interface that can restore functionality to victims of neuromuscular disorders. Although the most common application of this system has been communicating language, the properties and constraints of the linguistic domain have not to date been exploited when decoding brain signals that pertain to language. We hypothesized that combining the standard stepwise linear discriminant analysis with a Naive Bayes classifier and a trigram language model would increase the speed and accuracy of typing with the P300 speller. With integration of natural language processing, we observed significant improvements in accuracy and 40-60% increases in bit rate for all six subjects in a pilot study. This study suggests that integrating information about the linguistic domain can significantly improve signal classification.

A Case-based Retrieval System using Natural Language Processing and Population-based Visualization.

Electronic medical records capture large quantities of patient data generated as a result of routine care. Secondary use of this data for clinical research could provide new insights into the evolution of diseases and help assess the effectiveness of available interventions. Unfortunately, the unstructured nature of clinical data hinders a user's ability to understand this data: tools are needed to structure, model, and visualize the data to elucidate patterns in a patient population. We present a case-based retrieval framework that incorporates an extraction tool to identify concepts from clinical reports, a disease model to capture necessary context for interpreting extracted concepts, and a model-driven visualization to facilitate querying and interpretation of the results. We describe how the model is used to group, filter, and retrieve similar cases. We present an application of the framework that aids users in exploring a population of intracranial aneurysm patients.

Tools for improving the characterization and visualization of changes in neuro-oncology patients.

Capturing how a patient's medical problems change over time is important for understanding the progression of a disease, its effects, and response to treatment. We describe two prototype tools that are being developed as part of a data processing pipeline for standardizing, structuring, and visualizing problems and findings documented in clinical reports associated with neuro-oncology patients. Given a list of problems and findings identified using a natural language processing (NLP) system, we have created a mapping tool that assigns an observation of a problem to one of nine classes that describe change. The second tool utilizes iconic representations of the nine classes to generate a timeline interface, enabling users to pan, zoom, and filter the data. The result of this preliminary work is an automated approach for understanding and summarizing the evolution of a problem within the patient electronic medical record.

A Neuro-Oncology Workstation for Structuring, Modeling, and Visualizing Patient Records.

The patient medical record contains a wealth of information consisting of prior observations, interpretations, and interventions that need to be interpreted and applied towards decisions regarding current patient care. Given the time constraints and the large-often extraneous-amount of data available, clinicians are tasked with the challenge of performing a comprehensive review of how a disease progresses in individual patients. To facilitate this process, we demonstrate a neuro-oncology workstation that assists in structuring and visualizing medical data to promote an evidence-based approach for understanding a patient's record. The workstation consists of three components: 1) a structuring tool that incorporates natural language processing to assist with the extraction of problems, findings, and attributes for structuring observations, events, and inferences stated within medical reports; 2) a data modeling tool that provides a comprehensive and consistent representation of concepts for the disease-specific domain; and 3) a visual workbench for visualizing, navigating, and querying the structured data to enable retrieval of relevant portions of the patient record. We discuss this workstation in the context of reviewing cases of glioblastoma multiforme patients.

Problem-centric organization and visualization of patient imaging and clinical data.

A patient's electronic medical record contains a large amount of unstructured textual information. As patient records become increasingly dense owing to an aging population and increased occurrence of chronic diseases, a tool is needed to help organize and navigate patient data in a way that facilitates a clinician's ability to understand this information and that improves efficiency. A system has been developed for physicians that summarizes clinical information from a patient record. This system provides a gestalt view of the patient's record by organizing information about each disease along four dimensions (axes): time (eg, disease progression over time), space (eg, tumor in left frontal lobe), existence (eg, certainty of existence of a finding), and causality (eg, response to treatment). A display is generated from information provided by radiology reports and discharge summaries. Natural language processing is used to identify clinical abnormalities (problems, symptoms, findings) from these reports as well as associated properties and relationships. This information is presented in an integrated format that organizes extracted findings into a problem list, depicts the information on a timeline grid, and provides direct access to relevant reports and images. The goal of this system is to improve the structure of clinical information and its presentation to the physician, thereby simplifying the information retrieval and knowledge discovery necessary to bridge the gap between acquiring raw data and making an informed diagnosis.

A tool for improving the longitudinal imaging characterization for neuro-oncology cases.

We describe the development of a prototype tool for the construction of longitudinal cases studies that can be used for teaching files, construction of clinical databases, and for patient education. The test domain is neuro-oncology. The features of the tool include: 1) natural language processing tools to assist structuring report information; 2) integration of imaging data; 3) integration of drug information; 4) target data model that includes the dimensions of space, time, existence, and causality; 5) user interface that provides three levels of information including overview, filtered summarization, and details on demand. The results of this preliminary work include a full prototype for neuro-oncology patients that allow users an efficient means for scanning a patients imaging and support data.