A Statistical Approach for the Learning Curve of Physicians in Utilization of Electronic Order Sets.

BACKGROUND: Understanding a physician's behavior toward learning order sets is important as it is a key information to design order sets with optimized contents. OBJECTIVE: The objective of this article is to test a hypothesis: for a physician using a new order set repeatedly, the utilization rate of order set contents has a pattern of either increase or decrease. METHODS: To test the hypothesis, we retrieved empirical data of order set usage in local hospitals that adopted a new computerized physician order entry (CPOE) system and enterprise wide standard order sets. We extracted 4-year data including 63,583 orders made by 600 physicians in the inpatient setting and analyzed patterns of the learning curve at several aggregation levels. RESULT: The analysis results demonstrated that content modification rates over time were relatively flat except for a few localized patterns. CONCLUSION: Based on our finding, we reject our initial hypothesis.

Exploring Different Approaches in Measuring EHR-based Adherence to Best Practice - A Case Study with Order Sets and Associated Outcomes.

In connection with a recent enterprise-wide rollout of a new electronic health record, Intermountain Healthcare is investing significant effort in building a central library of best-practice order sets. These order sets represent best practice guidelines for specific clinical scenarios and are deployed with the intent of standardizing care, reducing variation, and consistently delivering good clinical outcomes to the populations we serve. The importance of measuring their use and the level to which caregivers adhere to these standards becomes an important factor in understanding and characterizing the impact that they deliver. Notwithstanding the importance of these metrics, well- defined methods for measuring adherence to a given clinical guideline as delivered through an order set are not fully characterized in the medical literature. In this paper, we describe initial efforts at measuring compliance to a defined 'best practice' standard by means of content utilization analysis, a calculated adherence model, and relevant clinical key performance indicators. The degree to which specified clinical outcomes vary across these measurement models are compared for a group of order sets tied to treating coronary artery bypass graft patients and heart failure patients. While the patterns derived from this analysis show some uncertainty, more granular methods that look at line-item, or 'order level' detail reveal more significant differences in the corresponding set of outcomes than higher-level adherence surrogates.

Extracting Actionable Recommendations for Modifying Enterprise Order Set Templates from CPOE Utilization Patterns.

As part of an enterprise-wide rollout of a new EHR, Intermountain Healthcare is investing significant effort in building a central library of best-practice order sets. As part of this effort, we have built analytics tools that can capture and determine actionable opportunities for change to order set templates, as reflected by aggregate user data. In order to determine the acceptability of this system and set meaningful thresholds for actual use, we extracted recommendations for additions, removals, and change in initial order selection status for a series of thirteen order sets. We asked local clinical experts to review the changes and classify them as acceptable or not. In total, the system identified 362 potential changes in the order set templates and 186 were deemed acceptable. While further enhancement will co sharpen the efficacy of the intervention, we expect that this type of utility will provide useful insight for content owners.

Experience with Integrating Diagnostic Decision Support Software with Electronic Health Records: Benefits versus Risks of Information Sharing.

INTRODUCTION: Reducing misdiagnosis has long been a goal of medical informatics. Current thinking has focused on achieving this goal by integrating diagnostic decision support into electronic health records. METHODS: A diagnostic decision support system already in clinical use was integrated into electronic health record systems at two large health systems, after clinician input on desired capabilities. The decision support provided three outputs: editable text for use in a clinical note, a summary including the suggested differential diagnosis with a graphical representation of probability, and a list of pertinent positive and pertinent negative findings (with onsets). RESULTS: Structured interviews showed widespread agreement that the tool was useful and that the integration improved workflow. There was disagreement among various specialties over the risks versus benefits of documenting intermediate diagnostic thinking. Benefits were most valued by specialists involved in diagnostic testing, who were able to use the additional clinical context for richer interpretation of test results. Risks were most cited by physicians making clinical diagnoses, who expressed concern that a process that generated diagnostic possibilities exposed them to legal liability. DISCUSSION AND CONCLUSION: Reconciling the preferences of the various groups could include saving only the finding list as a patient-wide resource, saving intermediate diagnostic thinking only temporarily, or adoption of professional guidelines to clarify the role of decision support in diagnosis.

Visualization of Order Set Creation and Usage Patterns in Early Implementation Phases of an Electronic Health Record.

Robust order set catalogs are considered to be a vital part of a computerized physician order entry (CPOE) implementation. Tools and processes for building, localizing, and maintaining these content sets in a centralized repository are important in facilitating the knowledge management lifecycle. Collectively, these order sets represent a significant investment of effort and expertise in capturing and distributing best clinical practice throughout an enterprise. In order to address an important gap of understanding how order sets are both created and used in practice in a current EHR installation, we have developed tools to analyze how order sets are used and customized in clinical practice. In this paper, we present the capabilities of these tools. We further characterize early development patterns in our enterprise order set catalog in early phases of a system-wide vendor EHR rollout. We present data that show how personalized order sets (favorites) are authored and then used in clinical practice. We anticipate that this type of utility will provide useful insight and feedback for those tasked with content governance and maintenance in CPOE systems.

Profiling Fast Healthcare Interoperability Resources (FHIR) of Family Health History based on the Clinical Element Models.

In this study we developed a Fast Healthcare Interoperability Resources (FHIR) profile to support exchanging a full pedigree based family health history (FHH) information across multiple systems and applications used by clinicians, patients, and researchers. We used previously developed clinical element models (CEMs) that are capable of representing the FHH information, and derived essential data elements including attributes, constraints, and value sets. We analyzed gaps between the FHH CEM elements and existing FHIR resources. Based on the analysis, we developed a profile that consists of 1) FHIR resources for essential FHH data elements, 2) extensions for additional elements that were not covered by the resources, and 3) a structured definition to integrate patient and family member information in a FHIR message. We implemented the profile using an open-source based FHIR framework and validated it using patient-entered FHH data that was captured through a locally developed FHH tool.

Infobutton usage in Patient Portal MyHealth.

Infobuttons have proven to be an important element in modern electronic health records (EHR), providing educational materials to both providers and patients. However, the usage of infobuttons in personalized health records (PHR) is only lightly documented in the literature. Patient-facing infobuttons pose a new challenge because patients have different questions and educational levels than professional users in EHRs. In this paper, we present usage data for patient-facing infobuttons that have recently been integrated in Intermountain Healthcare's patient portal MyHealth. We summarize use patterns by usage classified in modules, electronic resources (eResource), and infobutton sessions. Based on the analysis, we propose further enhancements to the current implementation of infobuttons in MyHealth.

Analysis of empty responses from electronic resources in infobutton managers.

Infobuttons provide context-aware educational materials to both providers and patients and are becoming an important element in modern electronic health records (EHR) and patient health records (PHR). However, the content from different electronic resources (e-resource) as responses from infobutton manager has not been fully analyzed and evaluated. In this paper, we propose a method for automatically analyzing responses from infobutton manager. A tool is implemented to retrieve and analyze responses from infobutton manager. To test the tool, we extracted and sampled common and uncommon concepts from EHR usage data in Intermountain Healthcare's enterprise data warehouse. From the output of the tool, we evaluate infobutton performance by multiple categories, including against the most and less common used concepts, grouped by different modules in patient portal, by different e-resources, and by type of access (standardized Health Level Seven (HL7) vs not). Based on the results of our evaluation, we provide suggestions for further enhancements of infobuttons to the current implementation, including suggesting accessing priorities of e-resources and encouraging the use of the HL7 standard.

Enabling locally-developed content for access through the infobutton by means of automated concept annotation.

Infobuttons have proven to be an increasingly important resource in providing a standardized approach to integrating useful educational materials at the point of care in electronic health records (EHRs). They provide a simple, uniform pathway for both patients and providers to receive pertinent education materials in a quick fashion from within EHRs and Personalized Health Records (PHRs). In recent years, the international standards organization Health Level Seven has balloted and approved a standards-based pathway for requesting and receiving data for infobuttons, simplifying some of the barriers for their adoption in electronic medical records and amongst content providers. Local content, developed by the hosting organization themselves, still needs to be indexed and annotated with appropriate metadata and terminologies in order to be fully accessible via the infobutton. In this manuscript we present an approach for automating the annotation of internally-developed patient education sheets with standardized terminologies and compare and contrast the approach with manual approaches used previously. We anticipate that a combination of system-generated and human reviewed annotations will provide the most comprehensive and effective indexing strategy, thereby allowing best access to internally-created content via the infobutton.

Disseminating context-specific access to online knowledge resources within electronic health record systems.

Clinicians' patient care information needs are frequent and largely unmet. Online knowledge resources are available that can help clinicians meet these information needs. Yet, significant barriers limit the use of these resources within the clinical workflow. Infobuttons are clinical decision support tools that use the clinical context (e.g., institution, user, patient) within electronic health record (EHR) systems to anticipate clinicians' questions and provide automated links to relevant information in knowledge resources. This paper describes OpenInfobutton (www.openinfobutton.org): a standards-based, open source Web service that was designed to disseminate infobutton capabilities in multiple EHR systems and healthcare organizations. OpenInfobutton has been successfully integrated with 38 knowledge resources at 5 large healthcare organizations in the United States. We describe the OpenInfobutton architecture, knowledge resource integration, and experiences at five large healthcare organizations.

Analysis of family health history based risk assessment algorithms: classification and data requirements.

Family Health History (FHH) is a valuable and potentially low-cost tool for risk assessment and diagnosis in patient-centered healthcare. In this study, we identified and analyzed existing FHH-based risk assessment algorithms (RAAs) for cardio-vascular disease (CVD) and colorectal cancer (CRC) to guide implementers of electronic health record (EHR) systems regarding the data requirements for computing risk using these algorithms. We found a core set of data elements that are required by most RAAs. While some of these data are available in EHR systems, the patients can be empowered to contribute the remainder.

Practical choices for infobutton customization: experience from four sites.

Context-aware links between electronic health records (EHRs) and online knowledge resources, commonly called "infobuttons" are being used increasingly as part of EHR "meaningful use" requirements. While an HL7 standard exists for specifying how the links should be constructed, there is no guidance on what links to construct. Collectively, the authors manage four infobutton systems that serve 16 institutions. The purpose of this paper is to publish our experience with linking various resources and specifying particular criteria that can be used by infobutton managers to select resources that are most relevant for a given situation. This experience can be used directly by those wishing to customize their own EHRs, for example by using the OpenInfobutton infobutton manager and its configuration tool, the Librarian Infobutton Tailoring Environment.

Combining infobuttons and semantic web rules for identifying patterns and delivering highly-personalized education materials.

Infobuttons have been established to be an effective resource for addressing information needs at the point of care, as evidenced by recent research and their inclusion in government-based electronic health record incentive programs in the United States. Yet their utility has been limited to wide success for only a specific set of domains (lab data, medication orders, and problem lists) and only for discrete, singular concepts that are already documented in the electronic medical record. In this manuscript, we present an effort to broaden their utility by connecting a semantic web-based phenotyping engine with an infobutton framework in order to identify and address broader issues in patient data, derived from multiple data sources. We have tested these patterns by defining and testing semantic definitions of pre-diabetes and metabolic syndrome. We intend to carry forward relevant information to the infobutton framework to present timely, relevant education resources to patients and providers.

Evolution in clinical knowledge management strategy at Intermountain Healthcare.

In this manuscript, we present an overview of the clinical knowledge management strategy at Intermountain Healthcare in support of our electronic medical record systems. Intermountain first initiated efforts in developing a centralized enterprise knowledge repository in 2001. Applications developed, areas of emphasis served, and key areas of focus are presented. We also detail historical and current areas of emphasis, in response to business needs.

Development and early usage patterns of a consumer-facing family health history tool.

Personalized medicine will require detailed clinical patient profiles, and a particular focus on capturing data that is useful in forecasting risk. A detailed family health history is considered a critical component of these profiles, insomuch that it has been coined as 'the best genetic test available'. Despite this, tools aimed at capturing this information for use in electronic health records have been characterized as inadequate. In this manuscript we detail the creation of a patient-facing family health history tool known as OurFamilyHealth, whose long-term emphasis is to facilitate risk assessment and clinical decision support. We present the rationale for such a tool, describe its development and release as a component of Intermountain Healthcare's patient portal, and detail early usage statistics surrounding the application. Data derived from the tool since its release are also compared against family history charting patterns in Intermountain's electronic health records, revealing differences in data availability.

Deriving consumer-facing disease concepts for family health histories using multi-source sampling.

The family health history has long been recognized as an effective way of understanding individuals' susceptibility to familial disease; yet electronic tools to support the capture and use of these data have been characterized as inadequate. As part of an ongoing effort to build patient-facing tools for entering detailed family health histories, we have compiled a set of concepts specific to familial disease using multi-source sampling. These concepts were abstracted by analyzing family health history data patterns in our enterprise data warehouse, collection patterns of consumer personal health records, analyses from the local state health department, a healthcare data dictionary, and concepts derived from genetic-oriented consumer education materials. Collectively, these sources yielded a set of more than 500 unique disease concepts, represented by more than 2500 synonyms for supporting patients in entering coded family health histories. We expect that these concepts will be useful in providing meaningful data and education resources for patients and providers alike.

A large-scale knowledge management method based on the analysis of the use of online knowledge resources.

UNLABELLED: Online health knowledge resources can be integrated into electronic health record systems using decision support tools known as "infobuttons." In this study we describe a knowledge management method based on the analysis of knowledge resource use via infobuttons in multiple institutions. METHODS: We conducted a two-phase analysis of laboratory test infobutton sessions at three healthcare institutions accessing two knowledge resources. The primary study measure was session coverage, i.e. the rate of infobutton sessions in which resources retrieved relevant content. RESULTS: In Phase One, resources covered 78.5% of the study sessions. In addition, a subset of 38 noncovered tests that most frequently raised questions was identified. In Phase Two, content development guided by the outcomes of Phase One resulted in a 4% average coverage increase. CONCLUSION: The described method is a valuable approach to large-scale knowledge management in rapidly changing domains.

Metadata extraction routines for improving infobutton performance.

Infobuttons have been proven as an effective means for providing quick, context-specific links to pertinent information resources at the point of care. Current infobutton manager implementations, however, lack the ability to exchange metadata, are limited to a relatively small set of information providers, and are targeted primarily for a clinician audience. As part of a local effort to implement infobuttons for patient use via a tethered personal health record, we present a series of metadata extraction routines. These routines were constructed to extract key pieces of information from health information providers on the Internet, including content coverage, language availability, and readability scores. The extraction routines were tested using thirty different disease conditions against eight different providers. The routines yielded 183 potential infobutton targets and associated metadata for each. The capabilities of the extraction routines will be expanded to cover new types of metadata in the future.

Glycemic control through computerized subcutaneous insulin calculators.

Sliding scale insulin (SSI) is a reactive therapy and does not maintain euglycemia in hospitalized patients. A combination of basal insulin, prandial insulin, and correction factor dosing provides a more consistent insulin state encouraging improved glycemic control. Intermountain Healthcare developed an application consisting of four calculators incorporating scheduled (basal and prandial) and correction-factor insulin. The result is a patient-specific order set with calculated insulin orders, a correction factor dosing table, and a carbohydrate dosing table. This paper describes the SQIC, access, usage monitoring, and challenges and solutions to the computerized decision support application created to replace SSI.

Ideal features for a patient-entered family history and risk assessment tool.

Family history (FH) is an important risk factor for many diseases and its assessment can be a powerful tool for identifying and stratifying patients at risk. As part of an initiative to improve FH collection and decision support at Intermountain Healthcare we conducted a literature review on FH data collection and reviewed a number of current Web-based tools. An ideal list of features was produced to help implement goals of the initiative.