Clinical evaluation of an interoperable clinical decision-support system for the detection of systemic inflammatory response syndrome in critically ill children.

BACKGROUND: Systemic inflammatory response syndrome (SIRS) is defined as a non-specific inflammatory process in the absence of infection. SIRS increases susceptibility for organ dysfunction, and frequently affects the clinical outcome of affected patients. We evaluated a knowledge-based, interoperable clinical decision-support system (CDSS) for SIRS detection on a pediatric intensive care unit (PICU). METHODS: The CDSS developed retrieves routine data, previously transformed into an interoperable format, by using model-based queries and guideline- and knowledge-based rules. We evaluated the CDSS in a prospective diagnostic study from 08/2018-03/2019. 168 patients from a pediatric intensive care unit of a tertiary university hospital, aged 0 to 18 years, were assessed for SIRS by the CDSS and by physicians during clinical routine. Sensitivity and specificity (when compared to the reference standard) with 95% Wald confidence intervals (CI) were estimated on the level of patients and patient-days. RESULTS: Sensitivity and specificity was 91.7% (95% CI 85.5-95.4%) and 54.1% (95% CI 45.4-62.5%) on patient level, and 97.5% (95% CI 95.1-98.7%) and 91.5% (95% CI 89.3-93.3%) on the level of patient-days. Physicians' SIRS recognition during clinical routine was considerably less accurate (sensitivity of 62.0% (95% CI 56.8-66.9%)/specificity of 83.3% (95% CI 80.4-85.9%)) when measurd on the level of patient-days. Evaluation revealed valuable insights for the general design of the CDSS as well as specific rule modifications. Despite a lower than expected specificity, diagnostic accuracy was higher than the one in daily routine ratings, thus, demonstrating high potentials of using our CDSS to help to detect SIRS in clinical routine. CONCLUSIONS: We successfully evaluated an interoperable CDSS for SIRS detection in PICU. Our study demonstrated the general feasibility and potentials of the implemented algorithms but also some limitations. In the next step, the CDSS will be optimized to overcome these limitations and will be evaluated in a multi-center study. TRIAL REGISTRATION: NCT03661450 (ClinicalTrials.gov); registered September 7, 2018.

Designing an openEHR-Based Pipeline for Extracting and Standardizing Unstructured Clinical Data Using Natural Language Processing.

BACKGROUND: Merging disparate and heterogeneous datasets from clinical routine in a standardized and semantically enriched format to enable a multiple use of data also means incorporating unstructured data such as medical free texts. Although the extraction of structured data from texts, known as natural language processing (NLP), has been researched at least for the English language extensively, it is not enough to get a structured output in any format. NLP techniques need to be used together with clinical information standards such as openEHR to be able to reuse and exchange still unstructured data sensibly. OBJECTIVES: The aim of the study is to automatically extract crucial information from medical free texts and to transform this unstructured clinical data into a standardized and structured representation by designing and implementing an exemplary pipeline for the processing of pediatric medical histories. METHODS: We constructed a pipeline that allows reusing medical free texts such as pediatric medical histories in a structured and standardized way by (1) selecting and modeling appropriate openEHR archetypes as standard clinical information models, (2) defining a German dictionary with crucial text markers serving as expert knowledge base for a NLP pipeline, and (3) creating mapping rules between the NLP output and the archetypes. The approach was evaluated in a first pilot study by using 50 manually annotated medical histories from the pediatric intensive care unit of the Hannover Medical School. RESULTS: We successfully reused 24 existing international archetypes to represent the most crucial elements of unstructured pediatric medical histories in a standardized form. The self-developed NLP pipeline was constructed by defining 3.055 text marker entries, 132 text events, 66 regular expressions, and a text corpus consisting of 776 entries for automatic correction of spelling mistakes. A total of 123 mapping rules were implemented to transform the extracted snippets to an openEHR-based representation to be able to store them together with other structured data in an existing openEHR-based data repository. In the first evaluation, the NLP pipeline yielded 97% precision and 94% recall. CONCLUSION: The use of NLP and openEHR archetypes was demonstrated as a viable approach for extracting and representing important information from pediatric medical histories in a structured and semantically enriched format. We designed a promising approach with potential to be generalized, and implemented a prototype that is extensible and reusable for other use cases concerning German medical free texts. In a long term, this will harness unstructured clinical data for further research purposes such as the design of clinical decision support systems. Together with structured data already integrated in openEHR-based representations, we aim at developing an interoperable openEHR-based application that is capable of automatically assessing a patient's risk status based on the patient's medical history at time of admission.

Clinical Decision-Support Systems for Detection of Systemic Inflammatory Response Syndrome, Sepsis, and Septic Shock in Critically Ill Patients: A Systematic Review.

BACKGROUND: The design of computerized systems able to support automated detection of threatening conditions in critically ill patients such as systemic inflammatory response syndrome (SIRS) and sepsis has been fostered recently. The increase of research work in this area is due to both the growing digitalization in health care and the increased appreciation of the importance of early sepsis detection and intervention. To be able to understand the variety of systems and their characteristics as well as performances, a systematic literature review is required. Existing reviews on this topic follow a rather restrictive searching methodology or they are outdated. As much progress has been made during the last 5 years, an updated review is needed to be able to keep track of current developments in this area of research. OBJECTIVES: To provide an overview about current approaches for the design of clinical decision-support systems (CDSS) in the context of SIRS, sepsis, and septic shock, and to categorize and compare existing approaches. METHODS: A systematic literature review was performed in accordance with the preferred reporting items for systematic reviews and meta-analyses (PRISMA) statement. Searches for eligible articles were conducted on five electronic bibliographic databases, including PubMed/MEDLINE, IEEE Xplore, Embase, Scopus, and ScienceDirect. Initial results were screened independently by two reviewers based on clearly defined eligibility criteria. A backward as well as an updated search enriched the initial results. Data were extracted from included articles and presented in a standardized way. Articles were classified into predefined categories according to characteristics extracted previously. The classification was performed according to the following categories: clinical setting including patient population and mono- or multicentric study, support type of the system such as prediction or detection, systems characteristics such as knowledge- or data-driven algorithms used, evaluation of methodology, and results including ground truth definition, sensitivity, and specificity. All results were assessed qualitatively by two reviewers. RESULTS: The search resulted in 2,373 articles out of which 55 results were identified as eligible. Over 80% of the articles describe monocentric studies. More than 50% include adult patients, and only four articles explicitly report the inclusion of pediatric patients. Patient recruitment often is very selective, which can be observed from highly varying inclusion and exclusion criteria. The task of disease detection is covered in 62% of the articles; prediction of upcoming conditions in 33%. Sepsis is covered in 67% of the articles, SIRS as sole entity in only 4%, whereas 27% focus on severe sepsis and/or septic shock. The most common combinations of categories "algorithm used" and "support type" are knowledge-based detection of sepsis and data-driven prediction of sepsis. In evaluations, manual chart review (38%) and diagnosis coding (29%) represent the most frequently used ground truth definitions; most studies present a sample size between 10,001 and 100,000 cases (31%) and performances highly differ with only five articles presenting sensitivities and specificities above 90%; four of them using knowledge-based rather than machine learning algorithms. The presentations of holistic CDSS approaches, including technical implementation details, system interfaces, and data and interoperability aspects enabling the use of CDSS in routine settings are missing in nearly all articles. CONCLUSIONS: The review demonstrated the high variety of research in this context successfully. A clear trend is observable toward the use of data-driven algorithms, and a lack of research could be identified in covering the pediatric population as well as acknowledging SIRS as an independent and threatening condition. The quality as well as the significance of the presented evaluations for assessing the performances of the algorithms in clinical routine settings are often not meeting the current standard of scientific work. Our future interest will be concentrated on these realistic settings by implementing and evaluating SIRS detection approaches as well as considering factors to make the CDSS useable in clinical routine from both technical and medical perspectives.

CADDIE2-evaluation of a clinical decision-support system for early detection of systemic inflammatory response syndrome in paediatric intensive care: study protocol for a diagnostic study.

INTRODUCTION: Systemic inflammatory response syndrome (SIRS) is one of the most critical indicators determining the clinical outcome of paediatric intensive care patients. Clinical decision support systems (CDSS) can be designed to support clinicians in detection and treatment. However, the use of such systems is highly discussed as they are often associated with accuracy problems and 'alert fatigue'. We designed a CDSS for detection of paediatric SIRS and hypothesise that a high diagnostic accuracy together with an adequate alerting will accelerate the use. Our study will (1) determine the diagnostic accuracy of the CDSS compared with gold standard decisions created by two blinded, experienced paediatricians, and (2) compare the system's diagnostic accuracy with that of routine clinical care decisions compared with the same gold standard. METHODS AND ANALYSIS: CADDIE2 is a prospective diagnostic accuracy study taking place at the Department of Pediatric Cardiology and Intensive Care Medicine at the Hannover Medical School; it represents the second step towards our vision of cross-institutional and data-driven decision-support for intensive care environments (CADDIE). The study comprises (1) recruitment of up to 300 patients (start date 1 August 2018), (2) creation of gold standard decisions (start date 1 May 2019), (3) routine SIRS assessments by physicians (starts with recruitment), (4) SIRS assessments by a CDSS (start date 1 May 2019), and (5) statistical analysis with a modified approach for determining sensitivity and specificity and comparing the accuracy results of the different diagnostic approaches (planned start date 1 July 2019). ETHICS AND DISSEMINATION: Ethics approval was obtained at the study centre (Ethics Committee of Hannover Medical School). Results of the main study will be communicated via publication in a peer-reviewed journal. TRIAL REGISTRATION NUMBER: ClinicalTrials.gov NCT03661450; Pre-results.