New Unintended Adverse Consequences of Electronic Health Records.

Although the health information technology industry has made considerable progress in the design, development, implementation, and use of electronic health records (EHRs), the lofty expectations of the early pioneers have not been met. In 2006, the Provider Order Entry Team at Oregon Health & Science University described a set of unintended adverse consequences (UACs), or unpredictable, emergent problems associated with computer-based provider order entry implementation, use, and maintenance. Many of these originally identified UACs have not been completely addressed or alleviated, some have evolved over time, and some new ones have emerged as EHRs became more widely available. The rapid increase in the adoption of EHRs, coupled with the changes in the types and attitudes of clinical users, has led to several new UACs, specifically: complete clinical information unavailable at the point of care; lack of innovations to improve system usability leading to frustrating user experiences; inadvertent disclosure of large amounts of patient-specific information; increased focus on computer-based quality measurement negatively affecting clinical workflows and patient-provider interactions; information overload from marginally useful computer-generated data; and a decline in the development and use of internally-developed EHRs. While each of these new UACs poses significant challenges to EHR developers and users alike, they also offer many opportunities. The challenge for clinical informatics researchers is to continue to refine our current systems while exploring new methods of overcoming these challenges and developing innovations to improve EHR interoperability, usability, security, functionality, clinical quality measurement, and information summarization and display.

Clinical Decision Support: a 25 Year Retrospective and a 25 Year Vision.

OBJECTIVE: The objective of this review is to summarize the state of the art of clinical decision support (CDS) circa 1990, review progress in the 25 year interval from that time, and provide a vision of what CDS might look like 25 years hence, or circa 2040. METHOD: Informal review of the medical literature with iterative review and discussion among the authors to arrive at six axes (data, knowledge, inference, architecture and technology, implementation and integration, and users) to frame the review and discussion of selected barriers and facilitators to the effective use of CDS. RESULT: In each of the six axes, significant progress has been made. Key advances in structuring and encoding standardized data with an increased availability of data, development of knowledge bases for CDS, and improvement of capabilities to share knowledge artifacts, explosion of methods analyzing and inferring from clinical data, evolution of information technologies and architectures to facilitate the broad application of CDS, improvement of methods to implement CDS and integrate CDS into the clinical workflow, and increasing sophistication of the end-user, all have played a role in improving the effective use of CDS in healthcare delivery. CONCLUSION: CDS has evolved dramatically over the past 25 years and will likely evolve just as dramatically or more so over the next 25 years. Increasingly, the clinical encounter between a clinician and a patient will be supported by a wide variety of cognitive aides to support diagnosis, treatment, care-coordination, surveillance and prevention, and health maintenance or wellness.

Validation of a Crowdsourcing Methodology for Developing a Knowledge Base of Related Problem-Medication Pairs.

BACKGROUND: Clinical knowledge bases of problem-medication pairs are necessary for many informatics solutions that improve patient safety, such as clinical summarization. However, developing these knowledge bases can be challenging. OBJECTIVE: We sought to validate a previously developed crowdsourcing approach for generating a knowledge base of problem-medication pairs in a large, non-university health care system with a widely used, commercially available electronic health record. METHODS: We first retrieved medications and problems entered in the electronic health record by clinicians during routine care during a six month study period. Following the previously published approach, we calculated the link frequency and link ratio for each pair then identified a threshold cutoff for estimated problem-medication pair appropriateness through clinician review; problem-medication pairs meeting the threshold were included in the resulting knowledge base. We selected 50 medications and their gold standard indications to compare the resulting knowledge base to the pilot knowledge base developed previously and determine its recall and precision. RESULTS: The resulting knowledge base contained 26,912 pairs, had a recall of 62.3% and a precision of 87.5%, and outperformed the pilot knowledge base containing 11,167 pairs from the previous study, which had a recall of 46.9% and a precision of 83.3%. CONCLUSIONS: We validated the crowdsourcing approach for generating a knowledge base of problem-medication pairs in a large non-university health care system with a widely used, commercially available electronic health record, indicating that the approach may be generalizable across healthcare settings and clinical systems. Further research is necessary to better evaluate the knowledge, to compare crowdsourcing with other approaches, and to evaluate if incorporating the knowledge into electronic health records improves patient outcomes.

Death, taxes and advance directives.

Suboptimal care at the end-of-life can be due to lack of access or knowledge of patient wishes. Ambiguity is often the result of non-standardized formats. Borrowing digital technology from other industries and using existing health information infrastructure can greatly improve the completion, storage, and distribution of advance directives. We believe several simple, low-cost adaptations to regional and federal programs can raise the standard of end-of-life care.

State of the art in clinical informatics: evidence and examples.

OBJECTIVE: The field of clinical informatics has expanded substantially in the six decades since its inception. Early research focused on simple demonstrations that health information technology (HIT) such as electronic health records (EHRs), computerized provider order entry (CPOE), and clinical decision support (CDS) systems were feasible and potentially beneficial in clinical practice. METHODS: In this review, we present recent evidence on clinical informatics in the United States covering three themes: 1) clinical informatics systems and interventions for providers, including EHRs, CPOE, CDS, and health information exchange; 2) consumer health informatics systems, including personal health records and web-based and mobile HIT; and 3) methods and governance for clinical informatics, including EHR usability; data mining, text mining, natural language processing, privacy, and security. RESULTS: Substantial progress has been made in demonstrating that various clinical informatics methodologies and applications improve the structure, process, and outcomes of various facets of the healthcare system. CONCLUSION: Over the coming years, much more will be expected from the field. As we move past the "early adopters" in Rogers' diffusion of innovations' curve through the "early majority" and into the "late majority," there will be a crucial need for new research methodologies and clinical applications that have been rigorously demonstrated to work (i.e., to improve health outcomes) in multiple settings with different types of patients and clinicians.

Hospital and health information systems - Current perspectives. Contribution of the IMIA Health Information Systems Working Group.

OBJECTIVE: To celebrate over 30 years of health information systems' (HIS) evolution by bringing together pioneers in the field, members of the next generation of leaders, and government officials from several developing nations in Africa to discuss the past, present, and future of HISs. METHODS: Participants gathered in Le Franschhoek, South Africa for a 2 1/2 day working conference consisting of scientific presentations followed by several concurrent breakout sessions. A small writing group prepared draft statements representing their positions on various topics of discussion which were circulated and revised by the entire group. RESULTS: Many new tools, techniques and technologies were described and discussed in great detail. Interestingly, all of the key themes identified in the first HIS meeting held over 30 years ago are still of vital importance today: Patient Centered design, Clinical User Support, Real-time Education, Human-computer Factors and Measuring Clinical User Performance, Meaningful use. CONCLUSIONS: As we continue to work to develop next-generation HISs, we must remember the lessons of the past as we strive to develop the solutions for tomorrow.

Rapid assessment of clinical information systems in the healthcare setting: an efficient method for time-pressed evaluation.

OBJECTIVE: Recent legislation in the United States provides strong incentives for implementation of electronic health records (EHRs). The ensuing transformation in U.S. health care will increase demand for new methods to evaluate clinical informatics interventions. Timeline constraints and a rapidly changing environment will make traditional evaluation techniques burdensome. This paper describes an anthropological approach that provides a fast and flexible way to evaluate clinical information systems. METHODS: Adapting mixed-method evaluation approaches from anthropology, we describe a rapid assessment process (RAP) for assessing clinical informatics interventions in health care that we developed and used during seven site visits to diverse community hospitals and primary care settings in the U.S. SETTING: Our multidisciplinary team used RAP to evaluate factors that either encouraged people to use clinical decision support (CDS) systems or interfered with use of these systems in settings ranging from large urban hospitals to single-practitioner, private family practices in small towns. RESULTS: Critical elements of the method include: 1) developing a fieldwork guide; 2) carefully selecting observation sites and participants; 3) thoroughly preparing for site visits; 4) partnering with local collaborators; 5) collecting robust data by using multiple researchers and methods; and 6) analyzing and reporting data in a structured manner helpful to the organizations being evaluated. CONCLUSIONS: RAP, iteratively developed over the course of visits to seven clinical sites across the U.S., has succeeded in allowing a multidisciplinary team of informatics researchers to plan, gather and analyze data, and report results in a maximally efficient manner.

Current challenges and opportunities for better integration of human factors research with development of clinical information systems.

OBJECTIVES: Clinical information system (CIS) developers and implementers have begun to look to other scientific disciplines for new methods, tools, and techniques to help them better understand clinicians and their organizational structures, clinical work environments, capabilities of clinical information and communications technology, and the way these structures and processes interact. The goal of this article is to help CIS researchers, developers, implementers, and evaluators better understand the methods, tools, techniques, and literature of the field of human factors. METHODS: We developed a framework that explains how six key human factors topics relate to the design, implementation, and evaluation of CISs. RESULTS: Using this framework we discuss the following six topics: 1) informatics and patient safety; 2) user interface design and evaluation; 3) workflow and task analysis; 4) clinical decision making and decision support; 5) distributed cognition; and 6) mental workload and situation awareness. CONCLUSIONS: Integrating the methods, tools, and lessons learned from each of these six areas of human factors research early in CIS design and incorporating them iteratively during development can improve user performance, user satisfaction, and integration into clinical workflow. Ultimately, this approach will improve clinical information systems and healthcare delivery.

Grand challenges in dental informatics.

When truly significant scientific challenges are overcome, it profoundly changes the daily activities, as well as the future research activities, of everyone involved in the related field. By identifying and describing the grand challenges facing a scientific field, we can help funding agencies identify and prioritize projects for support, stimulate and encourage new investigators to work on these intellectual and technological challenges, and help define the field itself. In this article, we present an informatics-oriented, future-patient-care scenario, then describe a series of applications and the related informatics grand challenges facing the dental field today. New techniques and technologies to help us overcome these challenges would facilitate the development of truly monumental applications, such as a comprehensive electronic oral health record, an automated dental treatment planning system for all diagnoses, or a system to profile patient risk for chronic oral diseases.

The emerging science of very early detection of disease outbreaks.

A surge of development of new public health surveillance systems designed to provide more timely detection of outbreaks suggests that public health has a new requirement: extreme timeliness of detection. The authors review previous work relevant to measuring timeliness and to defining timeliness requirements. Using signal detection theory and decision theory, the authors identify strategies to improve timeliness of detection and position ongoing system development within that framework.

A survey of patient-provider e-mail communication: what do patients think?

Communication between patients and providers forms the backbone of the patient-provider relationship. Often such communication is strained due to time and space limitations on the part of both patients and providers. Many healthcare organizations are developing secure e-mail communication facilities to allow patients to exchange e-mail messages with their providers. Providers are worried that opening such lines of communication will inundate them with vast quantities of e-mail from their patients. Patients are worried that their messages will be intercepted and read by unauthorized people. In an attempt to determine how a group of internet-active, e-mail-ready patients currently use, or potentially view, the ability to exchange e-mail messages with their health care providers, we distributed a survey via e-mail to over 9500 patients. After determining each patient's e-mail activity level (based on the number of messages sent each day), we asked questions such as: "Have you ever sent e-mail to your provider?" "What issues or concerns have prevented you from sending e-mail messages to your provider?" "If your provider were to tell you that someone in his/her office may screen, read or perhaps reply to your message before he/she sees it, to what extent would you be concerned about this?" and "How would you rate your overall satisfaction with the use of e-mail to communicate with your provider?" Results from the survey indicate that nearly 85% of the patients surveyed send at least one e-mail message per day, but that very few (i.e. 6%) of the patients have actually sent an e-mail message to their provider. Interestingly, over half of the patients indicated that they would like to send their providers e-mail, but that they do not know their provider's e-mail address.

Subjective assessment of usefulness and appropriate presentation mode of alerts and reminders in the outpatient setting.

There is very little known about the limits of alerting in the setting of the outpatient Electronic Medical Record (EMR). We are interested in how users value and prefer such alerts. One hundred Kaiser Permanente primary care clinicians were sent a four-page questionnaire. It contained questions related to the usability and usefulness of different approaches to presenting reminder and alert information. The survey also contained questions about the desirability of six categories of alerts. Forty-three of 100 questionnaires were returned. Users generally preferred an active, more intrusive interaction model for "alerts" and a passive, less intrusive model for order messages and other types of reminders and notifications. Drug related alerts were more highly rated than health maintenance or disease state reminders. Users indicated that more alerts would make the system "more useful" but "less easy to use".

Techniques for identifying the applicability of new information management technologies in the clinical setting: an example focusing on handheld computers.

This article describes techniques and strategies used to judge the potential applicability of new information management technologies in the clinical setting and to develop specific design recommendations for new features and services. We focus on a project carried out to identify the potential uses of handheld computers (i.e., the Palm Pilot or a small WinCE-based device) in the ambulatory practice setting. We found that the potential for a robust handheld computing device to positively affect the outpatient ambulatory clinical setting is enormous, and that the information derived from the exploratory research project is useful in creating specific design recommendations for further development.

Communication breakdown in the outpatient referral process.

OBJECTIVE: To evaluate primary care and specialist physicians' satisfaction with interphysician communication and to identify the major problems in the current referral process. DESIGN: Surveys were mailed to providers to determine satisfaction with the referral process; then patient-specific surveys were e-mailed to this group to obtain real-time referral information. SETTING: Academic tertiary care medical center. PARTICIPANTS: Attending-level primary care physicians (PCPs) and specialists. MEASUREMENTS AND MAIN RESULTS: The response rate for mail surveys for PCPs was 57% and for specialists was 51%. In the mail survey, 63% of PCPs and 35% of specialists were dissatisfied with the current referral process. Respondents felt that major problems with the current referral system were lack of timeliness of information and inadequate referral letter content. Information considered important by recipient groups was often not included in letters that were sent. The response rate for the referral specific e-mail surveys was 56% for PCPs and 53% for specialists. In this e-mail survey, 68% of specialists reported that they received no information from the PCP prior to specific referral visits, and 38% of these said that this information would have been helpful. In addition, four weeks after specific referral visits, 25% of PCPs had still not received any information from specialists. CONCLUSIONS: Substantial problems were present in the referral process. The major issues were physician dissatisfaction, lack of timeliness, and inadequate content of interphysician communication. Information obtained from the general survey and referral-specific survey was congruent. Efforts to improve the referral system could improve both physician satisfaction and quality of patient care.

Evaluating physician satisfaction regarding user interactions with an electronic medical record system.

A limiting factor in realizing the full potential of electronic medical records (EMR) is physician reluctance to use these applications. There have been very few formal usability studies of experienced physician users of EMRs in routine clinical use. We distributed the Questionnaire for User Interaction Satisfaction (QUIS) to 75 primary care physicians who routinely use the Brigham and Women's Integrated Computing System (BICS). BICS scored highest in the area of screen design and lowest in the area of system capability. Overall user satisfaction was most highly correlated with screen design and layout, and surprisingly not with system response time. Human-computer interaction studies can help focus our design efforts as we strive to increase clinician usage of information technology.

A computer-based outpatient clinical referral system.

The process of generating a clinical referral for a patient, and the resulting transfer of information from the primary care physician to the specialist and back again, are key components in the struggle to deliver less costly and more effective clinical care. We have created a computer-based, outpatient clinical referral application that facilitates: (1) identifying an appropriate specialist; (2) collecting the clinical, demographic, and financial data required to generate a referral; and (3) transferring the information between the specialist and the primary care physician (PCP). This article describes the development of the application itself and several of the knowledge bases that were created to facilitate this process. Preliminary results indicate that the new computer-based referral process is faster to use than conventional methods.

Use of fuzzy set theory to extend Dhawan's journal selection model: ranking the biomedical informatics serials.

OBJECTIVE: Experts disagree on the parameters to use to identify the "best" serials within a scientific field. The author set out to develop an extension to Dhawan's journal selection model for ranking serials in any scientific field. METHODS: Comparison of three different instantiations of Dhawan's model were used to rank thirty-four biomedical informatics serials. RESULTS: The first instantiation of Dhawan's model identified seven serials and divided them into two groups. The second instantiation of Dhawan's model identified twelve serials and separated them into two groups. Using fuzzy set theory the new extended model produced a rank ordered list of the top twelve biomedical informatics serials. CONCLUSIONS: Use of fuzzy set theory to assign set membership and combine data in Dhawan's journal selection model allows one to: (1) eliminate the need to determine arbitrary cutoff points for inclusion of serials within each of Dhawan's evaluation criteria categories, (2) combine data from disparate sources, and (3) obtain a rank-ordered list of the biomedical informatics serials rather than simply identifying a set of the "top" serials. Such a ranked list provides librarians and researchers alike with the information necessary to help them make their biomedical informatics serial selection decisions based on objective, quantifiable data.

A graphical user interaction model for integrating complex clinical applications: a pilot study.

We have developed and implemented a multi-faceted, graphical user interaction model for an advanced clinical information system. This paper describes a classification scheme for applications used by clinicians in their daily work, discusses the way clinicians interact with these applications, and the issues that arise during these user interactions. Through its emphasis on support for application interoperation, the graphical user interface that implements the model presents a single, consistent, context to the user, and thereby helps maintain patient safety and ensure ease of use.

Design and development of a computer-based clinical referral system for use within a physician hospital organization.

The process of creating a clinical referral for a patient and the transfer of information from the primary care physician to the specialist and back again is a key component in the struggle to deliver less costly and more effective clinical care. We have created a computer-based clinical referral application which facilitates 1) identifying an appropriate specialist; 2) collecting the clinical, demographic, and financial data required to generate a referral; and 3) transferring the information between the specialist and the primary care physician. Preliminary results indicate that the new computer-based process is faster.

Components of the optimal ambulatory care computing environment.

We present here a framework of core components of an ambulatory care computing environment, based on clinical and functional needs and workflow scenarios. We have established this framework through the use of two study devices: a vision of the clinical office of the future, and a survey of possible computer applications, both designed to help clinicians and practice directors communicate their information needs to systems designers. Clinicians prioritize applications based on strategic and practice goals: support for clinical users' workflow, improved quality of care, reduced cost of care, and the ability to measure performance and status. By reorganizing the needed functionality from a clinical viewpoint into a technical viewpoint, we are able to identify core information components for systems design. Based on this analysis, information needs in the ambulatory environment can be divided into five primary functions: patient data retrieval, documentation, communication, knowledge resources, and aggregate reporting. Three other fundamental processes--knowledge-based interventions, information integration, and confidentiality--run through all of these front-line functions. Component applications and data structures built with this framework in mind will afford a maximum combination of functionality and flexibility to handle future changes in the clinical environment.