Use of order sets in inpatient computerized provider order entry systems: a comparative analysis of usage patterns at seven sites.

BACKGROUND: Many computerized provider order entry (CPOE) systems include the ability to create electronic order sets: collections of clinically related orders grouped by purpose. Order sets promise to make CPOE systems more efficient, improve care quality and increase adherence to evidence-based guidelines. However, the development and implementation of order sets can be expensive and time-consuming and limited literature exists about their utilization. METHODS: Based on analysis of order set usage logs from a diverse purposive sample of seven sites with commercially and internally developed inpatient CPOE systems, we developed an original order set classification system. Order sets were categorized across seven non-mutually exclusive axes: admission/discharge/transfer (ADT), perioperative, condition-specific, task-specific, service-specific, convenience, and personal. In addition, 731 unique subtypes were identified within five axes: four in ADT (S=4), three in perioperative, 144 in condition-specific, 513 in task-specific, and 67 in service-specific. RESULTS: Order sets (n=1914) were used a total of 676,142 times at the participating sites during a one-year period. ADT and perioperative order sets accounted for 27.6% and 24.2% of usage respectively. Peripartum/labor, chest pain/acute coronary syndrome/myocardial infarction and diabetes order sets accounted for 51.6% of condition-specific usage. Insulin, angiography/angioplasty and arthroplasty order sets accounted for 19.4% of task-specific usage. Emergency/trauma, obstetrics/gynecology/labor delivery and anesthesia accounted for 32.4% of service-specific usage. Overall, the top 20% of order sets accounted for 90.1% of all usage. Additional salient patterns are identified and described. CONCLUSION: We observed recurrent patterns in order set usage across multiple sites as well as meaningful variations between sites. Vendors and institutional developers should identify high-value order set types through concrete data analysis in order to optimize the resources devoted to development and implementation.

Improving completeness of electronic problem lists through clinical decision support: a randomized, controlled trial.

BACKGROUND: Accurate clinical problem lists are critical for patient care, clinical decision support, population reporting, quality improvement, and research. However, problem lists are often incomplete or out of date. OBJECTIVE: To determine whether a clinical alerting system, which uses inference rules to notify providers of undocumented problems, improves problem list documentation. STUDY DESIGN AND METHODS: Inference rules for 17 conditions were constructed and an electronic health record-based intervention was evaluated to improve problem documentation. A cluster randomized trial was conducted of 11 participating clinics affiliated with a large academic medical center, totaling 28 primary care clinical areas, with 14 receiving the intervention and 14 as controls. The intervention was a clinical alert directed to the provider that suggested adding a problem to the electronic problem list based on inference rules. The primary outcome measure was acceptance of the alert. The number of study problems added in each arm as a pre-specified secondary outcome was also assessed. Data were collected during 6-month pre-intervention (11/2009-5/2010) and intervention (5/2010-11/2010) periods. RESULTS: 17,043 alerts were presented, of which 41.1% were accepted. In the intervention arm, providers documented significantly more study problems (adjusted OR=3.4, p<0.001), with an absolute difference of 6277 additional problems. In the intervention group, 70.4% of all study problems were added via the problem list alerts. Significant increases in problem notation were observed for 13 of 17 conditions. CONCLUSION: Problem inference alerts significantly increase notation of important patient problems in primary care, which in turn has the potential to facilitate quality improvement. TRIAL REGISTRATION: ClinicalTrials.gov: NCT01105923.

Clinician attitudes toward and use of electronic problem lists: a thematic analysis.

BACKGROUND: The clinical problem list is an important tool for clinical decision making, quality measurement and clinical decision support; however, problem lists are often incomplete and provider attitudes towards the problem list are poorly understood. METHODS: An ethnographic study of healthcare providers conducted from April 2009 to January 2010 was carried out among academic and community outpatient medical practices in the Greater Boston area across a wide range of medical and surgical specialties. Attitudes towards the problem list were then analyzed using grounded theory methods. RESULTS: Attitudes were variable, and dimensions of variations fit into nine themes: workflow, ownership and responsibility, relevance, uses, content, presentation, accuracy, alternatives, support/education and one cross-cutting theme of culture. CONCLUSIONS: Significant variation was observed in clinician attitudes towards and use of the electronic patient problem list. Clearer guidance and best practices for problem list utilization are needed.

A method and knowledge base for automated inference of patient problems from structured data in an electronic medical record.

BACKGROUND: Accurate knowledge of a patient's medical problems is critical for clinical decision making, quality measurement, research, billing and clinical decision support. Common structured sources of problem information include the patient problem list and billing data; however, these sources are often inaccurate or incomplete. OBJECTIVE: To develop and validate methods of automatically inferring patient problems from clinical and billing data, and to provide a knowledge base for inferring problems. STUDY DESIGN AND METHODS: We identified 17 target conditions and designed and validated a set of rules for identifying patient problems based on medications, laboratory results, billing codes, and vital signs. A panel of physicians provided input on a preliminary set of rules. Based on this input, we tested candidate rules on a sample of 100,000 patient records to assess their performance compared to gold standard manual chart review. The physician panel selected a final rule for each condition, which was validated on an independent sample of 100,000 records to assess its accuracy. RESULTS: Seventeen rules were developed for inferring patient problems. Analysis using a validation set of 100,000 randomly selected patients showed high sensitivity (range: 62.8-100.0%) and positive predictive value (range: 79.8-99.6%) for most rules. Overall, the inference rules performed better than using either the problem list or billing data alone. CONCLUSION: We developed and validated a set of rules for inferring patient problems. These rules have a variety of applications, including clinical decision support, care improvement, augmentation of the problem list, and identification of patients for research cohorts.

Summarization of clinical information: a conceptual model.

BACKGROUND: To provide high-quality and safe care, clinicians must be able to optimally collect, distill, and interpret patient information. Despite advances in text summarization, only limited research exists on clinical summarization, the complex and heterogeneous process of gathering, organizing and presenting patient data in various forms. OBJECTIVE: To develop a conceptual model for describing and understanding clinical summarization in both computer-independent and computer-supported clinical tasks. DESIGN: Based on extensive literature review and clinical input, we developed a conceptual model of clinical summarization to lay the foundation for future research on clinician workflow and automated summarization using electronic health records (EHRs). RESULTS: Our model identifies five distinct stages of clinical summarization: (1) Aggregation, (2) Organization, (3) Reduction and/or Transformation, (4) Interpretation and (5) Synthesis (AORTIS). The AORTIS model describes the creation of complex, task-specific clinical summaries and provides a framework for clinical workflow analysis and directed research on test results review, clinical documentation and medical decision-making. We describe a hypothetical case study to illustrate the application of this model in the primary care setting. CONCLUSION: Both practicing physicians and clinical informaticians need a structured method of developing, studying and evaluating clinical summaries in support of a wide range of clinical tasks. Our proposed model of clinical summarization provides a potential pathway to advance knowledge in this area and highlights directions for further research.