Implementation of a Clinical Decision Support System for Children With Minor Blunt Head Trauma Who Are at Nonnegligible Risk for Traumatic Brain Injuries.

STUDY OBJECTIVE: To determine the effect of providing risk estimates of clinically important traumatic brain injuries and management recommendations on emergency department (ED) outcomes for children with isolated intermediate Pediatric Emergency Care Applied Research Network clinically important traumatic brain injury risk factors. METHODS: This was a secondary analysis of a nonrandomized clinical trial with concurrent controls, conducted at 5 pediatric and 8 general EDs between November 2011 and June 2014, enrolling patients younger than 18 years who had minor blunt head trauma. After a baseline period, intervention sites received electronic clinical decision support providing patient-level clinically important traumatic brain injury risk estimates and management recommendations. The following primary outcomes in patients with one intermediate Pediatric Emergency Care Applied Research Network risk factor were compared before and after clinical decision support: proportion of ED computed tomography (CT) scans, adjusted for age, time trend, and site; and prevalence of clinically important traumatic brain injuries. RESULTS: The risk of clinically important traumatic brain injuries was known for 3,859 children with isolated findings (1,711 at intervention sites before clinical decision support, 1,702 at intervention sites after clinical decision support, and 446 at control sites). In this group, pooled CT proportion decreased from 24.2% to 21.6% after clinical decision support (odds ratio 0.86; 95% confidence interval 0.73 to 1.01). Decreases in CT use were noted across intervention EDs, but not in controls. The pooled adjusted odds ratio for CT use after clinical decision support was 0.73 (95% confidence interval 0.60 to 0.88). Among the entire cohort, clinically important traumatic brain injury was diagnosed at the index ED visit for 37 of 37 (100%) patients before clinical decision support and 32 of 33 patients (97.0%) after clinical decision support. CONCLUSION: Providing specific risks of clinically important traumatic brain injury through electronic clinical decision support was associated with a modest and safe decrease in ED CT use for children at nonnegligible risk of clinically important traumatic brain injuries.

Use of Traumatic Brain Injury Prediction Rules With Clinical Decision Support.

OBJECTIVES: We determined whether implementing the Pediatric Emergency Care Applied Research Network (PECARN) traumatic brain injury (TBI) prediction rules and providing risks of clinically important TBIs (ciTBIs) with computerized clinical decision support (CDS) reduces computed tomography (CT) use for children with minor head trauma. METHODS: Nonrandomized trial with concurrent controls at 5 pediatric emergency departments (PEDs) and 8 general EDs (GEDs) between November 2011 and June 2014. Patients were <18 years old with minor blunt head trauma. Intervention sites received CDS with CT recommendations and risks of ciTBI, both for patients at very low risk of ciTBI (no Pediatric Emergency Care Applied Research Network rule factors) and those not at very low risk. The primary outcome was the rate of CT, analyzed by site, controlling for time trend. RESULTS: We analyzed 16 635 intervention and 2394 control patients. Adjusted for time trends, CT rates decreased significantly (P < .05) but modestly (2.3%-3.7%) at 2 of 4 intervention PEDs for children at very low risk. The other 2 PEDs had small (0.8%-1.5%) nonsignificant decreases. CT rates did not decrease consistently at the intervention GEDs, with low baseline CT rates (2.1%-4.0%) in those at very low risk. The control PED had little change in CT use in similar children (from 1.6% to 2.9%); the control GED showed a decrease in the CT rate (from 7.1% to 2.6%). For all children with minor head trauma, intervention sites had small decreases in CT rates (1.7%-6.2%). CONCLUSIONS: The implementation of TBI prediction rules and provision of risks of ciTBIs by using CDS was associated with modest, safe, but variable decreases in CT use. However, some secular trends were also noted.

Clinical Decision Support for a Multicenter Trial of Pediatric Head Trauma: Development, Implementation, and Lessons Learned.

INTRODUCTION: For children who present to emergency departments (EDs) due to blunt head trauma, ED clinicians must decide who requires computed tomography (CT) scanning to evaluate for traumatic brain injury (TBI). The Pediatric Emergency Care Applied Research Network (PECARN) derived and validated two age-based prediction rules to identify children at very low risk of clinically-important traumatic brain injuries (ciTBIs) who do not typically require CT scans. In this case report, we describe the strategy used to implement the PECARN TBI prediction rules via electronic health record (EHR) clinical decision support (CDS) as the intervention in a multicenter clinical trial. METHODS: Thirteen EDs participated in this trial. The 10 sites receiving the CDS intervention used the Epic(®) EHR. All sites implementing EHR-based CDS built the rules by using the vendor's CDS engine. Based on a sociotechnical analysis, we designed the CDS so that recommendations could be displayed immediately after any provider entered prediction rule data. One central site developed and tested the intervention package to be exported to other sites. The intervention package included a clinical trial alert, an electronic data collection form, the CDS rules and the format for recommendations. RESULTS: The original PECARN head trauma prediction rules were derived from physician documentation while this pragmatic trial led each site to customize their workflows and allow multiple different providers to complete the head trauma assessments. These differences in workflows led to varying completion rates across sites as well as differences in the types of providers completing the electronic data form. Site variation in internal change management processes made it challenging to maintain the same rigor across all sites. This led to downstream effects when data reports were developed. CONCLUSIONS: The process of a centralized build and export of a CDS system in one commercial EHR system successfully supported a multicenter clinical trial.

Use of a remote clinical decision support service for a multicenter trial to implement prediction rules for children with minor blunt head trauma.

OBJECTIVE: To evaluate the architecture, integration requirements, and execution characteristics of a remote clinical decision support (CDS) service used in a multicenter clinical trial. The trial tested the efficacy of implementing brain injury prediction rules for children with minor blunt head trauma. MATERIALS AND METHODS: We integrated the Epic(®) electronic health record (EHR) with the Enterprise Clinical Rules Service (ECRS), a web-based CDS service, at two emergency departments. Patterns of CDS review included either a delayed, near-real-time review, where the physician viewed CDS recommendations generated by the nursing assessment, or a real-time review, where the physician viewed recommendations generated by their own documentation. A backstopping, vendor-based CDS triggered with zero delay when no recommendation was available in the EHR from the web-service. We assessed the execution characteristics of the integrated system and the source of the generated recommendations viewed by physicians. RESULTS: The ECRS mean execution time was 0.74 ±0.72 s. Overall execution time was substantially different at the two sites, with mean total transaction times of 19.67 and 3.99 s. Of 1930 analyzed transactions from the two sites, 60% (310/521) of all physician documentation-initiated recommendations and 99% (1390/1409) of all nurse documentation-initiated recommendations originated from the remote web service. DISCUSSION: The remote CDS system was the source of recommendations in more than half of the real-time cases and virtually all the near-real-time cases. Comparisons are limited by allowable variation in user workflow and resolution of the EHR clock. CONCLUSION: With maturation and adoption of standards for CDS services, remote CDS shows promise to decrease time-to-trial for multicenter evaluations of candidate decision support interventions.

Lessons learned from implementing service-oriented clinical decision support at four sites: A qualitative study.

OBJECTIVE: To identify challenges, lessons learned and best practices for service-oriented clinical decision support, based on the results of the Clinical Decision Support Consortium, a multi-site study which developed, implemented and evaluated clinical decision support services in a diverse range of electronic health records. METHODS: Ethnographic investigation using the rapid assessment process, a procedure for agile qualitative data collection and analysis, including clinical observation, system demonstrations and analysis and 91 interviews. RESULTS: We identified challenges and lessons learned in eight dimensions: (1) hardware and software computing infrastructure, (2) clinical content, (3) human-computer interface, (4) people, (5) workflow and communication, (6) internal organizational policies, procedures, environment and culture, (7) external rules, regulations, and pressures and (8) system measurement and monitoring. Key challenges included performance issues (particularly related to data retrieval), differences in terminologies used across sites, workflow variability and the need for a legal framework. DISCUSSION: Based on the challenges and lessons learned, we identified eight best practices for developers and implementers of service-oriented clinical decision support: (1) optimize performance, or make asynchronous calls, (2) be liberal in what you accept (particularly for terminology), (3) foster clinical transparency, (4) develop a legal framework, (5) support a flexible front-end, (6) dedicate human resources, (7) support peer-to-peer communication, (8) improve standards. CONCLUSION: The Clinical Decision Support Consortium successfully developed a clinical decision support service and implemented it in four different electronic health records and four diverse clinical sites; however, the process was arduous. The lessons identified by the Consortium may be useful for other developers and implementers of clinical decision support services.

A highly scalable, interoperable clinical decision support service.

OBJECTIVE: To create a clinical decision support (CDS) system that is shareable across healthcare delivery systems and settings over large geographic regions. MATERIALS AND METHODS: The enterprise clinical rules service (ECRS) realizes nine design principles through a series of enterprise java beans and leverages off-the-shelf rules management systems in order to provide consistent, maintainable, and scalable decision support in a variety of settings. RESULTS: The ECRS is deployed at Partners HealthCare System (PHS) and is in use for a series of trials by members of the CDS consortium, including internally developed systems at PHS, the Regenstrief Institute, and vendor-based systems deployed at locations in Oregon and New Jersey. Performance measures indicate that the ECRS provides sub-second response time when measured apart from services required to retrieve data and assemble the continuity of care document used as input. DISCUSSION: We consider related work, design decisions, comparisons with emerging national standards, and discuss uses and limitations of the ECRS. CONCLUSIONS: ECRS design, implementation, and use in CDS consortium trials indicate that it provides the flexibility and modularity needed for broad use and performs adequately. Future work will investigate additional CDS patterns, alternative methods of data passing, and further optimizations in ECRS performance.

A pilot study of distributed knowledge management and clinical decision support in the cloud.

OBJECTIVE: Implement and perform pilot testing of web-based clinical decision support services using a novel framework for creating and managing clinical knowledge in a distributed fashion using the cloud. The pilot sought to (1) develop and test connectivity to an external clinical decision support (CDS) service, (2) assess the exchange of data to and knowledge from the external CDS service, and (3) capture lessons to guide expansion to more practice sites and users. MATERIALS AND METHODS: The Clinical Decision Support Consortium created a repository of shared CDS knowledge for managing hypertension, diabetes, and coronary artery disease in a community cloud hosted by Partners HealthCare. A limited data set for primary care patients at a separate health system was securely transmitted to a CDS rules engine hosted in the cloud. Preventive care reminders triggered by the limited data set were returned for display to clinician end users for review and display. During a pilot study, we (1) monitored connectivity and system performance, (2) studied the exchange of data and decision support reminders between the two health systems, and (3) captured lessons. RESULTS: During the six month pilot study, there were 1339 patient encounters in which information was successfully exchanged. Preventive care reminders were displayed during 57% of patient visits, most often reminding physicians to monitor blood pressure for hypertensive patients (29%) and order eye exams for patients with diabetes (28%). Lessons learned were grouped into five themes: performance, governance, semantic interoperability, ongoing adjustments, and usability. DISCUSSION: Remote, asynchronous cloud-based decision support performed reasonably well, although issues concerning governance, semantic interoperability, and usability remain key challenges for successful adoption and use of cloud-based CDS that will require collaboration between biomedical informatics and computer science disciplines. CONCLUSION: Decision support in the cloud is feasible and may be a reasonable path toward achieving better support of clinical decision-making across the widest range of health care providers.

Patient education program for venous thromboembolism prevention in hospitalized patients.

PURPOSE: Up to 15% of clinician-ordered doses of injectable pharmacological prophylaxis to prevent venous thromboembolism are not administered. Patient refusal accounts for nearly 50% of these omitted doses. We conducted a prospective cohort study to determine whether a patient education program would improve medication adherence to clinician-ordered injectable prophylactic anticoagulation. METHODS: We identified 528 hospitalized patients ordered to receive injectable pharmacological venous thromboembolism prophylaxis. We evaluated the impact of pharmacist-led patient education sessions on medication adherence (defined as the ratio of doses administered to doses scheduled) compared with our historical cohort. RESULTS: Individualized patient education sessions were conducted within 24 hours of the initial order for prophylactic anticoagulation in 99% of patients. Adherence to clinician-ordered pharmacological venous thromboembolism prophylaxis was higher after the patient education program than in our historical cohort (94.4% vs 89.9%, P <.0001). The proportion of patients receiving 100% of scheduled doses of injectable pharmacological venous thromboembolism prophylaxis was higher after our novel patient education program than in our historical cohort (73.7% vs 62.4%, P=.001). Patient refusal as a reason for omitted doses was less frequent after the patient education program (29.3% vs 43.7%, P=.001). CONCLUSION: Pharmacist-led individualized patient education sessions were associated with higher medication adherence to clinician-ordered injectable pharmacological venous thromboembolism prophylaxis and a reduction in patient refusal as a reason for omitted doses. A randomized controlled trial to evaluate the impact of a patient education program on medication adherence to pharmacological venous thromboembolism prophylaxis is warranted.

Clinical information system services and capabilities desired for scalable, standards-based, service-oriented decision support: consensus assessment of the Health Level 7 clinical decision support Work Group.

A standards-based, service-oriented architecture for clinical decision support (CDS) has the potential to significantly enhance CDS scalability and robustness. To enable such a CDS architecture, the Health Level 7 CDS Work Group reviewed the literature, hosted multi-stakeholder discussions, and consulted domain experts to identify and prioritize the services and capabilities required from clinical information systems (CISs) to enable service-oriented CDS. In addition, relevant available standards were identified. Through this process, ten CIS services and eight CIS capabilities were identified as being important for enabling scalable, service-oriented CDS. In particular, through a survey of 46 domain experts, five services and capabilities were identified as being especially critical: 1) the use of standard information models and terminologies; 2) the ability to leverage a Decision Support Service (DSS); 3) support for a clinical data query service; 4) support for an event subscription and notification service; and 5) support for a user communication service.

Using a service oriented architecture approach to clinical decision support: performance results from two CDS Consortium demonstrations.

The Clinical Decision Support Consortium has completed two demonstration trials involving a web service for the execution of clinical decision support (CDS) rules in one or more electronic health record (EHR) systems. The initial trial ran in a local EHR at Partners HealthCare. A second EHR site, associated with Wishard Memorial Hospital, Indianapolis, IN, was added in the second trial. Data were gathered during each 6 month period and analyzed to assess performance, reliability, and response time in the form of means and standard deviations for all technical components of the service, including assembling and preparation of input data. The mean service call time for each period was just over 2 seconds. In this paper we report on the findings and analysis to date while describing the areas for further analysis and optimization as we continue to expand our use of a Services Oriented Architecture approach for CDS across multiple institutions.

A multi-layered framework for disseminating knowledge for computer-based decision support.

BACKGROUND: There are several challenges in encoding guideline knowledge in a form that is portable to different clinical sites, including the heterogeneity of clinical decision support (CDS) tools, of patient data representations, and of workflows. METHODS: We have developed a multi-layered knowledge representation framework for structuring guideline recommendations for implementation in a variety of CDS contexts. In this framework, guideline recommendations are increasingly structured through four layers, successively transforming a narrative text recommendation into input for a CDS system. We have used this framework to implement rules for a CDS service based on three guidelines. We also conducted a preliminary evaluation, where we asked CDS experts at four institutions to rate the implementability of six recommendations from the three guidelines. CONCLUSION: The experience in using the framework and the preliminary evaluation indicate that this approach has promise in creating structured knowledge, to implement in CDS systems, that is usable across organizations.

Factors influencing alert acceptance: a novel approach for predicting the success of clinical decision support.

BACKGROUND: Clinical decision support systems can prevent knowledge-based prescription errors and improve patient outcomes. The clinical effectiveness of these systems, however, is substantially limited by poor user acceptance of presented warnings. To enhance alert acceptance it may be useful to quantify the impact of potential modulators of acceptance. METHODS: We built a logistic regression model to predict alert acceptance of drug-drug interaction (DDI) alerts in three different settings. Ten variables from the clinical and human factors literature were evaluated as potential modulators of provider alert acceptance. ORs were calculated for the impact of knowledge quality, alert display, textual information, prioritization, setting, patient age, dose-dependent toxicity, alert frequency, alert level, and required acknowledgment on acceptance of the DDI alert. RESULTS: 50,788 DDI alerts were analyzed. Providers accepted only 1.4% of non-interruptive alerts. For interruptive alerts, user acceptance positively correlated with frequency of the alert (OR 1.30, 95% CI 1.23 to 1.38), quality of display (4.75, 3.87 to 5.84), and alert level (1.74, 1.63 to 1.86). Alert acceptance was higher in inpatients (2.63, 2.32 to 2.97) and for drugs with dose-dependent toxicity (1.13, 1.07 to 1.21). The textual information influenced the mode of reaction and providers were more likely to modify the prescription if the message contained detailed advice on how to manage the DDI. CONCLUSION: We evaluated potential modulators of alert acceptance by assessing content and human factors issues, and quantified the impact of a number of specific factors which influence alert acceptance. This information may help improve clinical decision support systems design.

Coded entry versus free-text and alert overrides: what you get depends on how you ask.

PURPOSE: A key trade-off in computerized clinical documentation exists between collecting coded data versus free-text. Coded data are more readily computer-readable and easier to reuse in different contexts. However, clinical information often exceeds the scope of commonly available terminologies, and coding may be resisted by providers. Alert override reasons are one domain for which agreed-upon terminologies are rarely used. Few data are available on how the collection of information affects the responses of providers. METHODS: We took advantage of a natural experiment and compared coded and uncoded reasons for drug-drug interaction (DDI) alert overrides entered in two inpatient prescribing systems with an identical DDI database but with one system offering coded reasons and the other free-text entry. We only included alerts which were issued in both sites and which physicians had to acknowledge. RESULTS: Over a one-year study period, 15,636 alerts were issued. The reasons for override entered in the coded approach matched the free-text site in only 46%. When using free-text, physicians provided many reasons not among the coded options, and often reported that they considered the alert inappropriate, including their rationale regarding this. However, the information entered as free-text included many typing and spelling errors, and the same concept was often represented in different ways, e.g. 209 different ways in which "will monitor as recommended" was noted. CONCLUSIONS: The reasons for alert override vary substantially according to the data entry type, which implies that data entry choice may lead to substantial distortion of the underlying data.

Creating shareable decision support services: an interdisciplinary challenge.

Creating shareable decision support services is a complex task requiring effort from multiple interdisciplinary role players with a wide variety of experience and expertise. The CDS Consortium research project has developed such a service, defining a multi-layer representation of knowledge and building upon an architectural service design created at Partners Health Care, and is demonstrating its use in both a local and an external institutional setting. The process was iterative, and we encountered unexpected requirements based on decisions made at various points. We report in this paper on challenges we faced while pursuing this research: knowledge representation and modeling, data interchange and standards adoption, the process of getting agreement on content, logistics of integrating into a system that already has multiple CDS interventions, legal issues around privacy and access, inter-team communication and organization.

Physician alerts to prevent symptomatic venous thromboembolism in hospitalized patients.

BACKGROUND: Venous thromboembolism (VTE) prophylaxis remains underused among hospitalized patients. We designed and carried out a large, multicenter, randomized controlled trial to test the hypothesis that an alert from a hospital staff member to the attending physician will reduce the rate of symptomatic VTE among high-risk patients not receiving prophylaxis. METHODS AND RESULTS: We enrolled patients using a validated point score system to detect hospitalized patients at high risk for symptomatic VTE who were not receiving prophylaxis. We randomized 2493 patients (82% on Medical Services) from 25 study sites to the intervention group (n=1238), in which the responsible physician was alerted by another hospital staff member, or the control group (n=1255), in which no alert was issued. The primary end point was symptomatic, objectively confirmed VTE within 90 days. Patients whose physicians were alerted were more than twice as likely to receive VTE prophylaxis as control subjects (46.0% versus 20.6%; P<0.0001). The symptomatic VTE rate was lower in the intervention group (2.7% versus 3.4%; hazard ratio, 0.79; 95% CI, 0.50 to 1.25), but the difference did not achieve statistical significance. The rate of major bleeding at 30 days in the alert group was similar to that in the control group (2.1% versus 2.3%; P=0.68). CONCLUSIONS: A strategy of direct notification of the physician by a staff member increases prophylaxis use and leads to a reduction in the rate of symptomatic VTE in hospitalized patients. However, VTE prophylaxis continues to be underused even after physician notification, especially among Medical Service patients.

Tiering drug-drug interaction alerts by severity increases compliance rates.

OBJECTIVE: Few data exist measuring the effect of differentiating drug-drug interaction (DDI) alerts in computerized provider order entry systems (CPOE) by level of severity ("tiering"). We sought to determine if rates of provider compliance with DDI alerts in the inpatient setting differed when a tiered presentation was implemented. DESIGN: We performed a retrospective analysis of alert log data on hospitalized patients at two academic medical centers during the period from 2/1/2004 through 2/1/2005. Both inpatient CPOE systems used the same DDI checking service, but one displayed alerts differentially by severity level (tiered presentation, including hard stops for the most severe alerts) while the other did not. Participants were adult inpatients who generated a DDI alert, and providers who wrote the orders. Alerts were presented during the order entry process, providing the clinician with the opportunity to change the patient's medication orders to avoid the interaction. MEASUREMENTS: Rate of compliance to alerts at a tiered site compared to a non-tiered site. RESULTS: We reviewed 71,350 alerts, of which 39,474 occurred at the non-tiered site and 31,876 at the tiered site. Compliance with DDI alerts was significantly higher at the site with tiered DDI alerts compared to the non-tiered site (29% vs. 10%, p < 0.001). At the tiered site, 100% of the most severe alerts were accepted, vs. only 34% at the non-tiered site; moderately severe alerts were also more likely to be accepted at the tiered site (29% vs. 10%). CONCLUSION: Tiered alerting by severity was associated with higher compliance rates of DDI alerts in the inpatient setting, and lack of tiering was associated with a high override rate of more severe alerts.