Application of electronic trigger tools to identify targets for improving diagnostic safety.

Progress in reducing diagnostic errors remains slow partly due to poorly defined methods to identify errors, high-risk situations, and adverse events. Electronic trigger (e-trigger) tools, which mine vast amounts of patient data to identify signals indicative of a likely error or adverse event, offer a promising method to efficiently identify errors. The increasing amounts of longitudinal electronic data and maturing data warehousing techniques and infrastructure offer an unprecedented opportunity to implement new types of e-trigger tools that use algorithms to identify risks and events related to the diagnostic process. We present a knowledge discovery framework, the Safer Dx Trigger Tools Framework, that enables health systems to develop and implement e-trigger tools to identify and measure diagnostic errors using comprehensive electronic health record (EHR) data. Safer Dx e-trigger tools detect potential diagnostic events, allowing health systems to monitor event rates, study contributory factors and identify targets for improving diagnostic safety. In addition to promoting organisational learning, some e-triggers can monitor data prospectively and help identify patients at high-risk for a future adverse event, enabling clinicians, patients or safety personnel to take preventive actions proactively. Successful application of electronic algorithms requires health systems to invest in clinical informaticists, information technology professionals, patient safety professionals and clinicians, all of who work closely together to overcome development and implementation challenges. We outline key future research, including advances in natural language processing and machine learning, needed to improve effectiveness of e-triggers. Integrating diagnostic safety e-triggers in institutional patient safety strategies can accelerate progress in reducing preventable harm from diagnostic errors.

Exploring new avenues to assess the sharp end of patient safety: an analysis of nationally aggregated peer review data.

BACKGROUND: Many healthcare organisations (HCOs) use peer review to evaluate clinical performance, but it is unclear whether these data provide useful insights for assessing the sharp end of patient safety. OBJECTIVE: To describe outcomes of peer review within the Department of Veterans Affairs (VA) healthcare system and identify opportunities to leverage peer review data for measurement and improvement of safety. DESIGN: We partnered with the VA's Risk Management Program Office to perform descriptive analyses of aggregated peer review data collected from 135 VA facilities between October 2011 and September 2012. We determined the frequency of screening factors used to initiate peer review and processes contributing to substandard care. We also evaluated peer review data for diagnosis-related performance concerns, an emerging area of interest in the patient safety field. RESULTS: During the study period, 23 287 cases were peer reviewed; 15 739 (68%) were sent to local peer review committees for final outcome determination after an initial review and 2320 cases were ultimately designated as substandard care (mean 17 cases/facility). In 20% of cases, the screening source was unspecified. The most common process contributing to substandard care was 'timing and appropriateness of treatment'. Approximately 16% of committee reviewed cases had diagnosis-related performance concerns, which were estimated to occur in approximately 0.5% of total hospital admissions. CONCLUSIONS: Peer review may be a useful tool for HCOs to assess their sharp end clinical performance, particularly safety events related to diagnostic and treatment errors. To address these emerging and largely preventable events, HCOs could consider revamping their existing peer review programmes to enable self-assessment, feedback and improvement.

An analysis of electronic health record-related patient safety concerns.

OBJECTIVE: A recent Institute of Medicine report called for attention to safety issues related to electronic health records (EHRs). We analyzed EHR-related safety concerns reported within a large, integrated healthcare system. METHODS: The Informatics Patient Safety Office of the Veterans Health Administration (VA) maintains a non-punitive, voluntary reporting system to collect and investigate EHR-related safety concerns (ie, adverse events, potential events, and near misses). We analyzed completed investigations using an eight-dimension sociotechnical conceptual model that accounted for both technical and non-technical dimensions of safety. Using the framework analysis approach to qualitative data, we identified emergent and recurring safety concerns common to multiple reports. RESULTS: We extracted 100 consecutive, unique, closed investigations between August 2009 and May 2013 from 344 reported incidents. Seventy-four involved unsafe technology and 25 involved unsafe use of technology. A majority (70%) involved two or more model dimensions. Most often, non-technical dimensions such as workflow, policies, and personnel interacted in a complex fashion with technical dimensions such as software/hardware, content, and user interface to produce safety concerns. Most (94%) safety concerns related to either unmet data-display needs in the EHR (ie, displayed information available to the end user failed to reduce uncertainty or led to increased potential for patient harm), software upgrades or modifications, data transmission between components of the EHR, or 'hidden dependencies' within the EHR. DISCUSSION: EHR-related safety concerns involving both unsafe technology and unsafe use of technology persist long after 'go-live' and despite the sophisticated EHR infrastructure represented in our data source. Currently, few healthcare institutions have reporting and analysis capabilities similar to the VA. CONCLUSIONS: Because EHR-related safety concerns have complex sociotechnical origins, institutions with long-standing as well as recent EHR implementations should build a robust infrastructure to monitor and learn from them.

Exploring the sociotechnical intersection of patient safety and electronic health record implementation.

OBJECTIVE: The intersection of electronic health records (EHR) and patient safety is complex. To examine the applicability of two previously developed conceptual models comprehensively to understand safety implications of EHR implementation in the English National Health Service (NHS). METHODS: We conducted a secondary analysis of interview data from a 30-month longitudinal, prospective, case study-based evaluation of EHR implementation in 12 NHS hospitals. We used a framework analysis approach to apply conceptual models developed by Sittig and Singh to understand better EHR implementation and use: an eight-dimension sociotechnical model and a three-phase patient safety model (safe technology, safe use of technology, and use of technology to improve safety). RESULTS: The intersection of patient safety and EHR implementation and use was characterized by risks involving technology (hardware and software, clinical content, and human-computer interfaces), the interaction of technology with non-technological factors, and improper or unsafe use of technology. Our data support that patient safety improvement activities as well as patient safety hazards change as an organization evolves from concerns about safe EHR functionality, ensuring safe and appropriate EHR use, to using the EHR itself to provide ongoing surveillance and monitoring of patient safety. DISCUSSION: We demonstrate the face validity of two models for understanding the sociotechnical aspects of safe EHR implementation and the complex interactions of technology within a healthcare system evolving from paper to integrated EHR. CONCLUSIONS: Using sociotechnical models, including those presented in this paper, may be beneficial to help stakeholders understand, synthesize, and anticipate risks at the intersection of patient safety and health information technology.

Physicians' diagnostic accuracy, confidence, and resource requests: a vignette study.

IMPORTANCE: Little is known about the relationship between physicians' diagnostic accuracy and their confidence in that accuracy. OBJECTIVE: To evaluate how physicians' diagnostic calibration, defined as the relationship between diagnostic accuracy and confidence in that accuracy, changes with evolution of the diagnostic process and with increasing diagnostic difficulty of clinical case vignettes. DESIGN, SETTING, AND PARTICIPANTS: We recruited general internists from an online physician community and asked them to diagnose 4 previously validated case vignettes of variable difficulty (2 easier; 2 more difficult). Cases were presented in a web-based format and divided into 4 sequential phases simulating diagnosis evolution: history, physical examination, general diagnostic testing data, and definitive diagnostic testing. After each phase, physicians recorded 1 to 3 differential diagnoses and corresponding judgments of confidence. Before being presented with definitive diagnostic data, physicians were asked to identify additional resources they would require to diagnose each case (ie, additional tests, second opinions, curbside consultations, referrals, and reference materials). MAIN OUTCOMES AND MEASURES: Diagnostic accuracy (scored as 0 or 1), confidence in diagnostic accuracy (on a scale of 0-10), diagnostic calibration, and whether additional resources were requested (no or yes). RESULTS: A total of 118 physicians with broad geographical representation within the United States correctly diagnosed 55.3% of easier and 5.8% of more difficult cases (P < .001). Despite a large difference in diagnostic accuracy between easier and more difficult cases, the difference in confidence was relatively small (7.2 vs 6.4 out of 10, for easier and more difficult cases, respectively) (P < .001) and likely clinically insignificant. Overall, diagnostic calibration was worse for more difficult cases (P < .001) and characterized by overconfidence in accuracy. Higher confidence was related to decreased requests for additional diagnostic tests (P = .01); higher case difficulty was related to more requests for additional reference materials (P = .01). CONCLUSIONS AND RELEVANCE: Our study suggests that physicians' level of confidence may be relatively insensitive to both diagnostic accuracy and case difficulty. This mismatch might prevent physicians from reexamining difficult cases where their diagnosis may be incorrect.

Compliance with guidelines to prevent surgical site infections: As simple as 1-2-3?

BACKGROUND: the purpose of this study was to assess predictive factors and compliance with surgical site infection (SSI) prevention guidelines at 2 county hospitals. DESIGN: chart review and analysis of laparotomy patients undergoing colorectal, hysterectomy, or abdominal vascular procedures over two 6-month periods 1 year apart and evaluation of safety climate using the Safety Attitudes Questionnaire (SAQ). RESULTS: overall compliance with all antibiotic prophylaxis guidelines was 62% (n = 442). Gynecologic surgery was an independent predictor of compliance with antibiotic prophylaxis guidelines in elective cases, and nonemergency status was an independent predictor when all cases were considered. Postoperative normothermia was predicted by hospital, procedure length, initial intraoperative temperature, and service. The SAQ had a 91% response rate. Contrary to expected, safety domain scores and agreement with statements on collaboration and teamwork were not predictive of compliance. CONCLUSION: interventions to improve poor compliance with infection prevention guidelines must be multifaceted, hospital- and service-specific, and resilient during emergencies. Good safety and teamwork climate are not sufficient.

Controversies in appendicitis.

BACKGROUND: Appendectomy for appendicitis is a common operation that is performed by most general surgeons. Despite the large number of appendectomies, numerous management controversies continue. METHODS: Review of the pertinent English language literature. RESULTS: Controversy persists regarding the use of imaging studies for diagnosis, laparoscopic vs. open surgical methods, nonoperative management of uncomplicated appendicitis, delayed management of the patient with phlegmon from severe appendicitis, the necessity for interval appendectomy, and the merits of deferral of off-hour appendectomy until the following morning. This review identifies the published reports that support the pros and cons of each controversy. CONCLUSIONS: Class I data generally are lacking to support the resolution of these controversies. Additional well-controlled clinical trials are necessary to obtain definitive answers.