Classifiers of Medical Eponymy in Scientific Texts.

Many concepts in the medical literature are named after persons. Frequent ambiguities and spelling varieties, however, complicate the automatic recognition of such eponyms with natural language processing (NLP) tools. Recently developed methods include word vectors and transformer models that incorporate context information into the downstream layers of a neural network architecture. To evaluate these models for classifying medical eponymy, we label eponyms and counterexamples mentioned in a convenience sample of 1,079 Pubmed abstracts, and fit logistic regression models to the vectors from the first (vocabulary) and last (contextualized) layers of a SciBERT language model. According to the area under sensitivity-specificity curves, models based on contextualized vectors achieved a median performance of 98.0% in held-out phrases. This outperformed models based on vocabulary vectors (95.7%) by a median of 2.3 percentage points. When processing unlabeled inputs, such classifiers appeared to generalize to eponyms that did not appear among any annotations. These findings attest to the effectiveness of developing domain-specific NLP functions based on pre-trained language models, and underline the utility of context information for classifying potential eponyms.

Analysis of the Representation of Frequent Clinical Attributes in the Unified Medical Language System.

Mapping clinical attributes from hospital information systems to standardized terminologies may allow their scientific reuse for multicenter studies. The Unified Medical Language System (UMLS) defines synonyms in different terminologies, which could be valuable for achieving semantic interoperability between different sites. Here we aim to explore the potential relevance of UMLS concepts and associated semantic relations for widely used clinical terminologies in a German university hospital. To semi-automatically examine a sample of the 200 most frequent codes from Erlangen University Hospital for three relevant terminologies, we implemented a script that queries their UMLS representation and associated mappings via a programming interface. We found that 94% of frequent diagnostic codes were available in UMLS, and that most of these codes could be mapped to other terminologies such as SNOMED CT. We observed that all examined laboratory codes were represented in UMLS, and that various translations to other languages were available for these concepts. The classification that is most widely used in German hospital for documenting clinical procedures was not originally represented in UMLS, but external mappings to SNOMED CT allowed identifying UMLS entries for 90.5% of frequent codes. Future research could extend this investigation to other code sets and terminologies, or study the potential utility of available mappings for specific applications.

Evaluation of Domain-Specific Word Vectors for Biomedical Word Sense Disambiguation.

Among medical applications of natural language processing (NLP), word sense disambiguation (WSD) estimates alternative meanings from text around homonyms. Recently developed NLP methods include word vectors that combine easy computability with nuanced semantic representations. Here we explore the utility of simple linear WSD classifiers based on aggregating word vectors from a modern biomedical NLP library in homonym contexts. We evaluated eight WSD tasks that consider literature abstracts as textual contexts. Discriminative performance was measured in held-out annotations as the median area under sensitivity-specificity curves (AUC) across tasks and 200 bootstrap repetitions. We find that classifiers trained on domain-specific vectors outperformed those from a general language model by 4.0 percentage points, and that a preprocessing step of filtering stopwords and punctuation marks enhanced discrimination by another 0.7 points. The best models achieved a median AUC of 0.992 (interquartile range 0.975 - 0.998). These improvements suggest that more advanced WSD methods might also benefit from leveraging domain-specific vectors derived from large biomedical corpora.

Design of an Interactive Web Application for Teaching Uncertainty Interpretations of Clinical Tests.

BACKGROUND: Assessing the uncertainty of diagnostic findings is essential for advising patients. Previous research has demonstrated the difficulty of computing the expected correctness of positive or negative results, although clinical decision support (CDS) tools promise to facilitate adequate interpretations. OBJECTIVES: To teach the potential utility of CDS tools to medical students, we designed an interactive software module that computes and visualizes relevant probabilities from typical inputs. METHODS: We reviewed the literature on recommended graphical approaches and decided to support contingency tables, plain table formats, tree diagrams, and icon arrays. RESULTS: We implemented these functions in a single-page web application, which was configured to complement our local learning management system where students also access interpretation tasks. CONCLUSION: Our technical choices promoted a rapid implementation. We intend to explore the utility of the tool during some upcoming courses. Future developments could also model a more complex clinical reality where the likelihood of alternative diagnoses is estimated from sets of clinical investigations.

Mapping the Entire Record-An Alternative Approach to Data Access from Medical Logic Modules.

OBJECTIVES: This study aimed to describe an alternative approach for accessing electronic medical records (EMRs) from clinical decision support (CDS) functions based on Arden Syntax Medical Logic Modules, which can be paraphrased as "map the entire record." METHODS: Based on an experimental Arden Syntax processor, we implemented a method to transform patient data from a commercial patient data management system (PDMS) to tree-structured documents termed CDS EMRs. They are encoded in a specific XML format that can be directly transformed to Arden Syntax data types by a mapper natively integrated into the processor. The internal structure of a CDS EMR reflects the tabbed view of an EMR in the graphical user interface of the PDMS. RESULTS: The study resulted in an architecture that provides CDS EMRs in the form of a network service. The approach enables uniform data access from all Medical Logic Modules and requires no mapping parameters except a case number. Measurements within a CDS EMR can be addressed with straightforward path expressions. The approach is in routine use at a German university hospital for more than 2 years. CONCLUSION: This practical approach facilitates the use of CDS functions in the clinical routine at our local hospital. It is transferrable to standard-compliant Arden Syntax processors with moderate effort. Its comprehensibility can also facilitate teaching and development. Moreover, it may lower the entry barrier for the application of the Arden Syntax standard and could therefore promote its dissemination.

Correction: KETOS: Clinical decision support and machine learning as a service - A training and deployment platform based on Docker, OMOP-CDM, and FHIR Web Services.

[This corrects the article DOI: 10.1371/journal.pone.0223010.].

A method for the graphical modeling of relative temporal constraints.

Searching for patient cohorts in electronic patient data often requires the definition of temporal constraints between the selection criteria. However, beyond a certain degree of temporal complexity, the non-graphical, form-based approaches implemented in current translational research platforms may be limited when modeling such constraints. In our opinion, there is a need for an easily accessible and implementable, fully graphical method for creating temporal queries. We aim to respond to this challenge with a new graphical notation. Based on Allen's time interval algebra, it allows for modeling temporal queries by arranging simple horizontal bars depicting symbolic time intervals. To make our approach applicable to complex temporal patterns, we apply two extensions: with duration intervals, we enable the inference about relative temporal distances between patient events, and with time interval modifiers, we support counting and excluding patient events, as well as constraining numeric values. We describe how to generate database queries from this notation. We provide a prototypical implementation, consisting of a temporal query modeling frontend and an experimental backend that connects to an i2b2 system. We evaluate our modeling approach on the MIMIC-III database to demonstrate that it can be used for modeling typical temporal phenotyping queries.

KETOS: Clinical decision support and machine learning as a service - A training and deployment platform based on Docker, OMOP-CDM, and FHIR Web Services.

BACKGROUND AND OBJECTIVE: To take full advantage of decision support, machine learning, and patient-level prediction models, it is important that models are not only created, but also deployed in a clinical setting. The KETOS platform demonstrated in this work implements a tool for researchers allowing them to perform statistical analyses and deploy resulting models in a secure environment. METHODS: The proposed system uses Docker virtualization to provide researchers with reproducible data analysis and development environments, accessible via Jupyter Notebook, to perform statistical analysis and develop, train and deploy models based on standardized input data. The platform is built in a modular fashion and interfaces with web services using the Health Level 7 (HL7) Fast Healthcare Interoperability Resources (FHIR) standard to access patient data. In our prototypical implementation we use an OMOP common data model (OMOP-CDM) database. The architecture supports the entire research lifecycle from creating a data analysis environment, retrieving data, and training to final deployment in a hospital setting. RESULTS: We evaluated the platform by establishing and deploying an analysis and end user application for hemoglobin reference intervals within the University Hospital Erlangen. To demonstrate the potential of the system to deploy arbitrary models, we loaded a colorectal cancer dataset into an OMOP database and built machine learning models to predict patient outcomes and made them available via a web service. We demonstrated both the integration with FHIR as well as an example end user application. Finally, we integrated the platform with the open source DataSHIELD architecture to allow for distributed privacy preserving data analysis and training across networks of hospitals. CONCLUSION: The KETOS platform takes a novel approach to data analysis, training and deploying decision support models in a hospital or healthcare setting. It does so in a secure and privacy-preserving manner, combining the flexibility of Docker virtualization with the advantages of standardized vocabularies, a widely applied database schema (OMOP-CDM), and a standardized way to exchange medical data (FHIR).

Feasibility analysis of conducting observational studies with the electronic health record.

BACKGROUND: The secondary use of electronic health records (EHRs) promises to facilitate medical research. We reviewed general data requirements in observational studies and analyzed the feasibility of conducting observational studies with structured EHR data, in particular diagnosis and procedure codes. METHODS: After reviewing published observational studies from the University Hospital of Erlangen for general data requirements, we identified three different study populations for the feasibility analysis with eligibility criteria from three exemplary observational studies. For each study population, we evaluated the availability of relevant patient characteristics in our EHR, including outcome and exposure variables. To assess data quality, we computed distributions of relevant patient characteristics from the available structured EHR data and compared them to those of the original studies. We implemented computed phenotypes for patient characteristics where necessary. In random samples, we evaluated how well structured patient characteristics agreed with a gold standard from manually interpreted free texts. We categorized our findings using the four data quality dimensions "completeness", "correctness", "currency" and "granularity". RESULTS: Reviewing general data requirements, we found that some investigators supplement routine data with questionnaires, interviews and follow-up examinations. We included 847 subjects in the feasibility analysis (Study 1 n = 411, Study 2 n = 423, Study 3 n = 13). All eligibility criteria from two studies were available in structured data, while one study required computed phenotypes in eligibility criteria. In one study, we found that all necessary patient characteristics were documented at least once in either structured or unstructured data. In another study, all exposure and outcome variables were available in structured data, while in the other one unstructured data had to be consulted. The comparison of patient characteristics distributions, as computed from structured data, with those from the original study yielded similar distributions as well as indications of underreporting. We observed violations in all four data quality dimensions. CONCLUSIONS: While we found relevant patient characteristics available in structured EHR data, data quality problems may entail that it remains a case-by-case decision whether diagnosis and procedure codes are sufficient to underpin observational studies. Free-text data or subsequently supplementary study data may be important to complement a comprehensive patient history.

User-Centered Development of an Online Platform for Drug Dosing Recommendations in Pediatrics.

BACKGROUND: Drug therapy in pediatric patients is a complex process. Children are subject to continuous growth and variation in drug-metabolizing enzyme activity, requiring continuous adaption of dosages. In Germany, currently no publicly available database exists that provides evidence-based information on drug dosages in pediatrics. For local drug dosing support, a prototype database has been developed within the Children's Hospital, Erlangen. A user-centered development process was initiated to establish an online platform for evidence-based dosing recommendations, as well as pharmacological and pharmaceutical drug information in pediatrics. OBJECTIVES: The objectives of the study were to survey the demand for such a platform and to assess the usability of the different versions of the developed system. METHODS: The developed prototype was evaluated in a pluralistic walkthrough with prospective end users. After a redesign, the second prototype of the online platform underwent an online usability testing based on a tailored questionnaire and the System Usability Scale (SUS) (n = 12). RESULTS: Eleven of 12 participants expressed a demand for an online platform for pediatric dosing recommendations. The majority of the participants requested the integration of extended features, such as drug-drug interaction alerts, or information on adverse effects, pharmacokinetics, and pharmacodynamics. Particularly noteworthy is the demand for an online calculator; 5 of a total of 15 participants explicitly requested a calculator for dosages (based on age, weight, body surface) and glomerular filtration rate. The usability of the second prototype was rated "good to excellent" with a median SUS of 81.25. CONCLUSION: Local domain experts demand an online platform for pediatric dosing recommendations. The application of the user-centered design approach enabled the development of a prototype suitable for practical use. Multiple additional required functionalities have been identified, whereby the importance of an online calculator for patient-individual dosing recommendations was particularly emphasized.

Extractive summarization of clinical trial descriptions.

PURPOSE: Text summarization of clinical trial descriptions has the potential to reduce the time required to familiarize oneself with the subject of studies by condensing long-form detailed descriptions to concise, meaning-preserving synopses. This work describes the process and quality of automatically generated summaries of clinical trial descriptions using extractive text summarization methods. METHODS: We generated a novel dataset from the detailed descriptions and brief summaries of trials registered on clinicaltrials.gov. We executed several text summarization algorithms on the detailed descriptions in this corpus and calculated the standard ROUGE metrics using the brief summaries included in the record as a reference. To investigate the correlation of these metrics with human sentiments, four reviewers assessed the content-completeness of the generated summaries and the helpfulness of both the generated and reference summaries via a Likert scale questionnaire. RESULTS: The filtering stages of the dataset generation process reduce the 277,228 trials registered on clinicaltrials.gov to 101,016 records usable for the summarization task. On average, the summaries in this corpus are 25% the length of the detailed descriptions. Of the evaluated text summarization methods, the TextRank algorithm exhibits the overall best performance with a ROUGE-1 F1 score of 0.3531, ROUGE-2 F1 score of 0.1723, and ROUGE-L F1 score of 0.3003. These scores correlate with the assessment of the helpfulness and content similarity by the human reviewers. Inter-rater agreement for the helpfulness and content similarity was slight and fair respectively (Fleiss' kappa of 0.12 and 0.22). CONCLUSIONS: Extractive summarization is a viable tool for generating meaning-preserving synopses of detailed clinical trial descriptions. Further, the human evaluation has shown that the ROUGE-L F1 score is useful for rating the general quality of generated summaries of clinical trial descriptions in an automated way.

Next-generation reference intervals for pediatric hematology.

Background Interpreting hematology analytes in children is challenging due to the extensive changes in hematopoiesis that accompany physiological development and lead to pronounced sex- and age-specific dynamics. Continuous percentile charts from birth to adulthood allow accurate consideration of these dynamics. However, the ethical and practical challenges unique to pediatric reference intervals have restricted the creation of such percentile charts, and limitations in current approaches to laboratory test result displays restrict their use when guiding clinical decisions. Methods We employed an improved data-driven approach to create percentile charts from laboratory data collected during patient care in 10 German centers (9,576,910 samples from 358,292 patients, 412,905-1,278,987 samples per analyte). We demonstrate visualization of hematology test results using percentile charts and z-scores (www.pedref.org/hematology) and assess the potential of percentiles and z-scores to support diagnosis of different hematological diseases. Results We created percentile charts for hemoglobin, hematocrit, red cell indices, red cell count, red cell distribution width, white cell count and platelet count in girls and boys from birth to 18 years of age. Comparison of pediatricians evaluating complex clinical scenarios using percentile charts versus conventional/tabular representations shows that percentile charts can enhance physician assessment in selected example cases. Age-specific percentiles and z-scores, compared with absolute test results, improve the identification of children with blood count abnormalities and the discrimination between different hematological diseases. Conclusions The provided reference intervals enable precise assessment of pediatric hematology test results. Representation of test results using percentiles and z-scores facilitates their interpretation and demonstrates the potential of digital approaches to improve clinical decision-making.

Evaluation of a Patient-Facing Digital Prototype for Perioperative Risk Assessment.

Preparations for anesthesiological management of patients build on preoperative patient self-reports concerning risk factors and comorbidities. In this setting, electronic documentation could facilitate innovative computerized functions, although patient-facing digital questionnaires require appropriate tools that patients can access effectively. To explore the feasibility of an electronic application for preoperative data acquisition directly from patients, a digital, tablet-based prototypical application has been developed within a user-centered design process in order to replace a previously used paper-based anamnesis sheet for perioperative risk evaluation. The implemented prototype has been extensively tested and iteratively improved to progressively provide an easy-to-use data entry function. To assess the suitability of this tool for everyday data acquisition by patients and physicians and to identify usability problems, the stepwise development process was accompanied by a heuristic evaluation as well as a think-aloud evaluation, while another 56 participating patients completed a feedback sheet according to ISO 9241/10. The latter method detected additional usability problems that occurred during the use of the application, which contributed to iterative improvements of the prototype. Throughout the development process, 81 issues were identified and largely resolved. After these revisions of the prototype, the number of problems found per tester decreased from 4.75 to 0.96, while the overall rating increased to 6.14 out of 7 points (SD = 1.2). These improvements demonstrate the value and efficiency of such a user-centered design process and illustrate that a user-friendly patient-facing digital data entry can replace preoperative paper questionnaires for anesthesiological management.

An Extension of the Arden Syntax to Facilitate Clinical Document Generation.

While clinical information systems usually store patient records in database tables, human interpretations as well as information transfer between institutions often require that clinical data can be represented as documents. To automate document generation from patient data in conjunction with the rich computational facilities of clinical decision support, we propose a template-based extension of the Arden Syntax, and discuss the benefits and limitations observed during a pilot application for patient recruitment. While the original Arden Syntax supports string concatenation as well as the substitution of unnamed placeholders, we integrated an additional method based on embedding expressions into strings. A dedicated parser identifies the expressions and automatically substitutes them at runtime, which can for example be harnessed to display the most recent value from a time series. The resulting mechanism supports the generation of extensive clinical documents without the need to apply specific operators. To evaluate the proposed extension, we implemented an Arden module that identifies an intensive care patient cohort that conforms to the eligibility criteria of a clinical trial and outputs a concise patient overview in different document formats. While string interpolation in the original Arden standard has been tailored to clinical event monitoring, we interpret that our accessible approach usefully extends Arden's data-to-text capabilities. Future research might target the development of an interactive template editor that would hide the complexity of formatting directives and conditional expressions behind a graphical user interface, and explore how computer-linguistic formalisms might facilitate advanced features such as automatic inflections of verbs and nouns.

A detailed analysis of the Arden Syntax expression grammar.

OBJECTIVE: The Arden Syntax for Medical Logic Systems is a standard for encoding and sharing medical knowledge in the form of Medical Logic Modules. To improve accessibility for clinicians, the originators of the standard deliberately designed Arden Syntax expressions to resemble natural language, and parentheses around operands are not generally required. For certain patterns of nested expressions, however, the use of parentheses is mandatory, otherwise they are not accepted by an Arden Syntax environment. In this study, we refer to such patterns as anomalies. The purpose of this paper is to investigate the extent and the circumstances of such anomalies, and to outline a solution based on an alternative grammar encoding approach. METHODS: To analyze the distribution of anomalies in nested expressions, we developed two custom-made complementary utilities. The first utility, termed parser, checks a single expression pattern against the specification-compliant grammar for syntactic correctness. The second utility, termed composer, automatically creates an extensive amount of expression patterns by permuting and nesting operators without the use of parentheses, and stores these together with the expected syntactic correctness. By means of these utilities we conducted a comprehensive analysis of anomalies by comparing the expected correctness with the actual correctness. Any detected anomalies are stored into a set of files, grouped by the respective top-level operator, for a subsequent analysis. RESULTS: The composer utility nested 165 unary, binary, or ternary operators of Arden Syntax version 2.8 to a depth of two, resulting in a set of 76,533 expression patterns, of which 18,978 (24.8%) have been identified as anomalies. An automated assessment of their practical relevance for medical knowledge encoding is infeasible. Manual screening of selected samples indicated that only a small proportion of the detected anomalies would be relevant. The cause of the anomalies lies in the encoding of the grammar. A change of the basic encoding approach with some additional customizations eliminates the anomalies. A working expression parser is included in the supplementary material. CONCLUSION: Arden Syntax expressions are affected by anomalies. Since only a small proportion of them have practical relevance and they cannot cause false calculations or clinical decisions, their practical impact is likely limited. However, they may be potential points of confusion for knowledge engineers. An alternative expression grammar, based on a different encoding approach, would not only eliminate the anomalies, but could considerably facilitate both maintenance and further development of the standard.

Predicting Clinical Outcomes in Colorectal Cancer Using Machine Learning.

Using gene markers and other patient features to predict clinical outcomes plays a vital role in enhancing clinical decision making and improving prognostic accuracy. This work uses a large set of colorectal cancer patient data to train predictive models using machine learning methods such as random forest, general linear model, and neural network for clinically relevant outcomes including disease free survival, survival, radio-chemotherapy response (RCT-R) and relapse. The most successful predictive models were created for dichotomous outcomes like relapse and RCT-R with accuracies of 0.71 and 0.70 on blinded test data respectively. The best prediction models regarding overall survival and disease-free survival had C-Index scores of 0.86 and 0.76 respectively. These models could be used in the future to aid a decision for or against chemotherapy and improve survival prognosis. We propose that future work should focus on creating reusable frameworks and infrastructure for training and delivering predictive models to physicians, so that they could be readily applied to other diseases in practice and be continuously developed integrating new data.

Effects of staff training and electronic event monitoring on long-term adherence to lung-protective ventilation recommendations.

PURPOSE: To investigate long-term effects of staff training and electronic clinical decision support (CDS) on adherence to lung-protective ventilation recommendations. MATERIALS AND METHODS: In 2012, group instructions and workshops at two surgical intensive care units (ICUs) started, focusing on standardized protocols for mechanical ventilation and volutrauma prevention. Subsequently implemented CDS functions continuously monitor ventilation parameters, and from 2015 triggered graphical notifications when tidal volume (VT) violated individual thresholds. To estimate the effects of these educational and technical interventions, we retrospectively analyzed nine years of VT records from routine care. As outcome measures, we calculated relative frequencies of settings that conform to recommendations, case-specific mean excess VT, and total ICU survival. RESULTS: Assessing 571,478 VT records from 10,241 ICU cases indicated that adherence during pressure-controlled ventilation improved significantly after both interventions; the share of conforming VT records increased from 61.6% to 83.0% and then 86.0%. Despite increasing case severity, ICU survival remained nearly constant over time. CONCLUSIONS: Staff training effectively improves adherence to lung-protective ventilation strategies. The observed CDS effect seemed less pronounced, although it can easily be adapted to new recommendations. Both interventions, which futures studies could deploy in combination, promise to improve the precision of mechanical ventilation.

Using Arden Syntax for the creation of a multi-patient surveillance dashboard.

OBJECTIVE: Most practically deployed Arden-Syntax-based clinical decision support (CDS) modules process data from individual patients. The specification of Arden Syntax, however, would in principle also support multi-patient CDS. The patient data management system (PDMS) at our local intensive care units does not natively support patient overviews from customizable CDS routines, but local physicians indicated a demand for multi-patient tabular overviews of important clinical parameters such as key laboratory measurements. As our PDMS installation provides Arden Syntax support, we set out to explore the capability of Arden Syntax for multi-patient CDS by implementing a prototypical dashboard for visualizing laboratory findings from patient sets. METHODS AND MATERIAL: Our implementation leveraged the object data type, supported by later versions of Arden, which turned out to be serviceable for representing complex input data from several patients. For our prototype, we designed a modularized architecture that separates the definition of technical operations, in particular the control of the patient context, from the actual clinical knowledge. Individual Medical Logic Modules (MLMs) for processing single patient attributes could then be developed according to well-tried Arden Syntax conventions. RESULTS: We successfully implemented a working dashboard prototype entirely in Arden Syntax. The architecture consists of a controller MLM to handle the patient context, a presenter MLM to generate a dashboard view, and a set of traditional MLMs containing the clinical decision logic. Our prototype could be integrated into the graphical user interface of the local PDMS. We observed that with realistic input data the average execution time of about 200ms for generating dashboard views attained applicable performance. CONCLUSION: Our study demonstrated the general feasibility of creating multi-patient CDS routines in Arden Syntax. We believe that our prototypical dashboard also suggests that such implementations can be relatively easy, and may simultaneously hold promise for sharing dashboards between institutions and reusing elementary components for additional dashboards.

Using Arden Syntax Medical Logic Modules to reduce overutilization of laboratory tests for detection of bacterial infections-Success or failure?

OBJECTIVE: Bacterial infections frequently cause prolonged intensive care unit (ICU) stays. Repeated measurements of the procalcitonin (PCT) biomarker are typically used for early detection and follow up of bacterial infections and sepsis, but those PCT measurements are costly. To avoid overutilization, we developed and evaluated a clinical decision support system (CDSS) in Arden Syntax which computes necessary and preventable PCT orders. METHODS: The CDSS implements a rule set based on the latest PCT value, the time period since this measurement, and the PCT trend scenario. We assessed the CDSS effects on the daily rate of ordered PCT tests within a prospective study having two ON and two OFF phases in a surgical ICU. In addition, we performed interviews with the participating physicians to investigate their experience with the CDSS advice. RESULTS: Prior to the deployment of the CDSS, 22% of the performed PCT tests were potentially preventable according to the rule set. During the first ON phase the daily rate of ordered PCT tests per patient decreased significantly from 0.807 to 0.662. In subsequent OFF, ON and OFF phases, however, PCT utilization reached again daily rates of 0.733, 0.803, and 0.792, respectively. The interviews demonstrated that the physicians were aware of the problem of PCT overutilization, which they primarily attributed to acute time constraints. The responders assumed that the majority of preventable measurements are indiscriminately ordered for patients during longer ICU stays. CONCLUSION: We observed an 18% reduction of PCT tests within the first four weeks of CDSS support in the investigated ICU. This reduction may have been influenced by raised awareness of the overutilization problem; the extent of this influence cannot be determined in our study design. No reduction of PCT tests could be observed during the second ON phase. The physician interviews indicated that time critical ICU situations can prevent extensive reflection about the necessity of individual tests. In order to achieve an enduring effect on PCT utilization, we will have to proceed to electronic order entry.

Evaluation of an Interactive Visualization Tool for the Interpretation of Pediatric Laboratory Test Results.

The physiological age-related development of pediatric laboratory results interferes with pathological derangements, which can complicate the interpretation of test results. Recently proposed continuous reference intervals (RIs) promise to be beneficial, although their clinical use may depend on graphical presentations. To estimate the clinical utility of continuous RIs, we developed and evaluated an interactive visualization tool, and examined the differentiation of hemoglobinopathies that is attainable based on the underlying innovative RI model. The implemented web application allows users to easily enter laboratory test results, and displays various visualizations in conjunction with the corresponding RIs, such as charts and personalized Z-scores. To evaluate the usability of the visualization tool, we conducted concurrent think-aloud sessions with four physicians, who were prompted to solve a set of typical interpretation tasks, and acquired additional information through a questionnaire including the System Usability Scale (SUS). We used 85 de-identified clinical cases for an exemplified assessment of how well model-based interpretations of blood count parameters reproduced previously diagnosed hemoglobinopathies. Usability tests as well as questionnaire responses indicated that the developed tool was well received by the physicians. Results from the think-aloud evaluation revealed only minor problems and the tool reached an average SUS score of 86.9, suggesting good usability. Hemoglobinopathy discrimination depended on the considered subtype, although the overall performance of the novel method rivaled the one of the conventional approach. The interactive visualization of innovative continuous reference intervals demonstrated promising results, which justifies further testing on the path towards clinical routine.