Use of order sets in inpatient computerized provider order entry systems: a comparative analysis of usage patterns at seven sites.

BACKGROUND: Many computerized provider order entry (CPOE) systems include the ability to create electronic order sets: collections of clinically related orders grouped by purpose. Order sets promise to make CPOE systems more efficient, improve care quality and increase adherence to evidence-based guidelines. However, the development and implementation of order sets can be expensive and time-consuming and limited literature exists about their utilization. METHODS: Based on analysis of order set usage logs from a diverse purposive sample of seven sites with commercially and internally developed inpatient CPOE systems, we developed an original order set classification system. Order sets were categorized across seven non-mutually exclusive axes: admission/discharge/transfer (ADT), perioperative, condition-specific, task-specific, service-specific, convenience, and personal. In addition, 731 unique subtypes were identified within five axes: four in ADT (S=4), three in perioperative, 144 in condition-specific, 513 in task-specific, and 67 in service-specific. RESULTS: Order sets (n=1914) were used a total of 676,142 times at the participating sites during a one-year period. ADT and perioperative order sets accounted for 27.6% and 24.2% of usage respectively. Peripartum/labor, chest pain/acute coronary syndrome/myocardial infarction and diabetes order sets accounted for 51.6% of condition-specific usage. Insulin, angiography/angioplasty and arthroplasty order sets accounted for 19.4% of task-specific usage. Emergency/trauma, obstetrics/gynecology/labor delivery and anesthesia accounted for 32.4% of service-specific usage. Overall, the top 20% of order sets accounted for 90.1% of all usage. Additional salient patterns are identified and described. CONCLUSION: We observed recurrent patterns in order set usage across multiple sites as well as meaningful variations between sites. Vendors and institutional developers should identify high-value order set types through concrete data analysis in order to optimize the resources devoted to development and implementation.

Improving completeness of electronic problem lists through clinical decision support: a randomized, controlled trial.

BACKGROUND: Accurate clinical problem lists are critical for patient care, clinical decision support, population reporting, quality improvement, and research. However, problem lists are often incomplete or out of date. OBJECTIVE: To determine whether a clinical alerting system, which uses inference rules to notify providers of undocumented problems, improves problem list documentation. STUDY DESIGN AND METHODS: Inference rules for 17 conditions were constructed and an electronic health record-based intervention was evaluated to improve problem documentation. A cluster randomized trial was conducted of 11 participating clinics affiliated with a large academic medical center, totaling 28 primary care clinical areas, with 14 receiving the intervention and 14 as controls. The intervention was a clinical alert directed to the provider that suggested adding a problem to the electronic problem list based on inference rules. The primary outcome measure was acceptance of the alert. The number of study problems added in each arm as a pre-specified secondary outcome was also assessed. Data were collected during 6-month pre-intervention (11/2009-5/2010) and intervention (5/2010-11/2010) periods. RESULTS: 17,043 alerts were presented, of which 41.1% were accepted. In the intervention arm, providers documented significantly more study problems (adjusted OR=3.4, p<0.001), with an absolute difference of 6277 additional problems. In the intervention group, 70.4% of all study problems were added via the problem list alerts. Significant increases in problem notation were observed for 13 of 17 conditions. CONCLUSION: Problem inference alerts significantly increase notation of important patient problems in primary care, which in turn has the potential to facilitate quality improvement. TRIAL REGISTRATION: ClinicalTrials.gov: NCT01105923.

Randomized controlled trial of health maintenance reminders provided directly to patients through an electronic PHR.

BACKGROUND: Provider and patient reminders can be effective in increasing rates of preventive screenings and vaccinations. However, the effect of patient-directed electronic reminders is understudied. OBJECTIVE: To determine whether providing reminders directly to patients via an electronic Personal Health Record (PHR) improved adherence to care recommendations. DESIGN: We conducted a cluster randomized trial without blinding from 2005 to 2007 at 11 primary care practices in the Partners HealthCare system. PARTICIPANTS: A total of 21,533 patients with access to a PHR were invited to the study, and 3,979 (18.5%) consented to enroll. INTERVENTIONS: Patients in the intervention arm received health maintenance (HM) reminders via a secure PHR "eJournal," which allowed them to review and update HM and family history information. Patients in the active control arm received access to an eJournal that allowed them to input and review information related to medications, allergies and diabetes management. MAIN MEASURES: The primary outcome measure was adherence to guideline-based care recommendations. KEY RESULTS: Intention-to-treat analysis showed that patients in the intervention arm were significantly more likely to receive mammography (48.6% vs 29.5%, p = 0.006) and influenza vaccinations (22.0% vs 14.0%, p = 0.018). No significant improvement was observed in rates of other screenings. Although Pap smear completion rates were higher in the intervention arm (41.0% vs 10.4%, p < 0.001), this finding was no longer significant after excluding women's health clinics. Additional on-treatment analysis showed significant increases in mammography (p = 0.019) and influenza vaccination (p = 0.015) for intervention arm patients who opened an eJournal compared to control arm patients, but no differences for any measure among patients who did not open an eJournal. CONCLUSIONS: Providing patients with HM reminders via a PHR may be effective in improving some elements of preventive care.

A method and knowledge base for automated inference of patient problems from structured data in an electronic medical record.

BACKGROUND: Accurate knowledge of a patient's medical problems is critical for clinical decision making, quality measurement, research, billing and clinical decision support. Common structured sources of problem information include the patient problem list and billing data; however, these sources are often inaccurate or incomplete. OBJECTIVE: To develop and validate methods of automatically inferring patient problems from clinical and billing data, and to provide a knowledge base for inferring problems. STUDY DESIGN AND METHODS: We identified 17 target conditions and designed and validated a set of rules for identifying patient problems based on medications, laboratory results, billing codes, and vital signs. A panel of physicians provided input on a preliminary set of rules. Based on this input, we tested candidate rules on a sample of 100,000 patient records to assess their performance compared to gold standard manual chart review. The physician panel selected a final rule for each condition, which was validated on an independent sample of 100,000 records to assess its accuracy. RESULTS: Seventeen rules were developed for inferring patient problems. Analysis using a validation set of 100,000 randomly selected patients showed high sensitivity (range: 62.8-100.0%) and positive predictive value (range: 79.8-99.6%) for most rules. Overall, the inference rules performed better than using either the problem list or billing data alone. CONCLUSION: We developed and validated a set of rules for inferring patient problems. These rules have a variety of applications, including clinical decision support, care improvement, augmentation of the problem list, and identification of patients for research cohorts.

Governance for clinical decision support: case studies and recommended practices from leading institutions.

OBJECTIVE: Clinical decision support (CDS) is a powerful tool for improving healthcare quality and ensuring patient safety; however, effective implementation of CDS requires effective clinical and technical governance structures. The authors sought to determine the range and variety of these governance structures and identify a set of recommended practices through observational study. DESIGN: Three site visits were conducted at institutions across the USA to learn about CDS capabilities and processes from clinical, technical, and organizational perspectives. Based on the results of these visits, written questionnaires were sent to the three institutions visited and two additional sites. Together, these five organizations encompass a variety of academic and community hospitals as well as small and large ambulatory practices. These organizations use both commercially available and internally developed clinical information systems. MEASUREMENTS: Characteristics of clinical information systems and CDS systems used at each site as well as governance structures and content management approaches were identified through extensive field interviews and follow-up surveys. RESULTS: Six recommended practices were identified in the area of governance, and four were identified in the area of content management. Key similarities and differences between the organizations studied were also highlighted. CONCLUSION: Each of the five sites studied contributed to the recommended practices presented in this paper for CDS governance. Since these strategies appear to be useful at a diverse range of institutions, they should be considered by any future implementers of decision support.

Complementary methods of system usability evaluation: surveys and observations during software design and development cycles.

Poor usability of clinical information systems delays their adoption by clinicians and limits potential improvements to the efficiency and safety of care. Recurring usability evaluations are therefore, integral to the system design process. We compared four methods employed during the development of outpatient clinical documentation software: clinician email response, online survey, observations and interviews. Results suggest that no single method identifies all or most problems. Rather, each approach is optimal for evaluations at a different stage of design and characterizes different usability aspect. Email responses elicited from clinicians and surveys report mostly technical, biomedical, terminology and control problems and are most effective when a working prototype has been completed. Observations of clinical work and interviews inform conceptual and workflow-related problems and are best performed early in the cycle. Appropriate use of these methods consistently during development may significantly improve system usability and contribute to higher adoption rates among clinicians and to improved quality of care.

Order sets in computerized physician order entry systems: an analysis of seven sites.

Most computerized physician order entry (CPOE) systems have built-in support for order sets (collections of orders grouped by a clinical purpose). Evidence and experience suggest that order sets are important tools for ordering efficiency and decision support and may influence ordering. Developing and maintaining order sets is costly, so hospitals often must prioritize which order sets can be created. We analyzed order set utilization at seven diverse sites with CPOE. The number of order sets per site ranged from 81 to 535, and the number of order set uses per discharge ranged from 0.48 to 9.89. We also compared the top ten order sets at each site, and found many commonalities, such as generic and condition-specific admission order sets, surgical sets and clinical pathways. We also found that, at each site, utilization of order sets was skewed, with a small number of order sets comprising the bulk of utilization. These findings may be useful for order sets developers, particularly in settings where resources are constrained and the most important order sets must be developed first.

Clinical decision support capabilities of commercially-available clinical information systems.

BACKGROUND: The most effective decision support systems are integrated with clinical information systems, such as inpatient and outpatient electronic health records (EHRs) and computerized provider order entry (CPOE) systems. Purpose The goal of this project was to describe and quantify the results of a study of decision support capabilities in Certification Commission for Health Information Technology (CCHIT) certified electronic health record systems. METHODS: The authors conducted a series of interviews with representatives of nine commercially available clinical information systems, evaluating their capabilities against 42 different clinical decision support features. RESULTS: Six of the nine reviewed systems offered all the applicable event-driven, action-oriented, real-time clinical decision support triggers required for initiating clinical decision support interventions. Five of the nine systems could access all the patient-specific data items identified as necessary. Six of the nine systems supported all the intervention types identified as necessary to allow clinical information systems to tailor their interventions based on the severity of the clinical situation and the user's workflow. Only one system supported all the offered choices identified as key to allowing physicians to take action directly from within the alert. Discussion The principal finding relates to system-by-system variability. The best system in our analysis had only a single missing feature (from 42 total) while the worst had eighteen.This dramatic variability in CDS capability among commercially available systems was unexpected and is a cause for concern. CONCLUSIONS: These findings have implications for four distinct constituencies: purchasers of clinical information systems, developers of clinical decision support, vendors of clinical information systems and certification bodies.

Fall TIPS: strategies to promote adoption and use of a fall prevention toolkit.

Patient falls are serious problems in hospitals. Risk factors for falls are well understood and nurses routinely assess for fall risk on all hospitalized patients. However, the link from nursing assessment of fall risk, to identification and communication of tailored interventions to prevent falls is yet to be established. The Fall TIPS (Tailoring Interventions for Patient Safety) Toolkit was developed to leverage existing practices and workflows and to employ information technology to improve fall prevention practices. The purpose of this paper is to describe the Fall TIPS Toolkit and to report on strategies used to drive adoption of the Toolkit in four acute care hospitals. Using the IHI "Framework for Spread" as a conceptual model, the research team describes the "spread" of the Fall TIPS Toolkit as means to integrate effective fall prevention practices into the workflow of interdisciplinary caregivers, patients and family members.