Partnership as a Pathway to Diagnostic Excellence: The Challenges and Successes of Implementing the Safer Dx Learning Lab.

BACKGROUND: Learning health system (LHS) approaches could potentially help health care organizations (HCOs) identify and address diagnostic errors. However, few such programs exist, and their implementation is poorly understood. METHODS: The authors conducted a qualitative evaluation of the Safer Dx Learning Lab, a partnership between a health system and a research team, to identify and learn from diagnostic errors and improve diagnostic safety at an organizational level. The research team conducted virtual interviews to solicit participant feedback regarding experiences with the lab, focusing specifically on implementation and sustainment issues. RESULTS: Interviews of 25 members associated with the lab identified the following successes: learning and professional growth, improved workflow related to streamlining the process of reporting error cases, and a psychologically safe culture for identifying and reporting diagnostic errors. However, multiple barriers also emerged: competing priorities between clinical responsibilities and research, time-management issues related to a lack of protected time, and inadequate guidance to disseminate findings. Lessons learned included understanding the importance of obtaining buy-in from leadership and interested stakeholders, creating a psychologically safe environment for reporting cases, and the need for more protected time for clinicians to review and learn from cases. CONCLUSION: Findings suggest that a learning health systems approach using partnerships between researchers and a health system affected organizational culture by prioritizing learning from diagnostic errors and encouraging clinicians to be more open to reporting. The study findings can help organizations overcome barriers to engage clinicians and inform future implementation and sustainment of similar initiatives.

Diagnostic error in mental health: a review.

Diagnostic errors are associated with patient harm and suboptimal outcomes. Despite national scientific efforts to advance definition, measurement and interventions for diagnostic error, diagnosis in mental health is not well represented in this ongoing work. We aimed to summarise the current state of research on diagnostic errors in mental health and identify opportunities to align future research with the emerging science of diagnostic safety. We review conceptual considerations for defining and measuring diagnostic error, the application of these concepts to mental health settings, and the methods and subject matter focus of recent studies of diagnostic error in mental health. We found that diagnostic error is well understood to be a problem in mental healthcare. Although few studies used clear definitions or frameworks for understanding diagnostic error in mental health, several studies of missed, wrong, delayed and disparate diagnosis of common mental disorders have identified various avenues for future research and development. Nevertheless, a lack of clear consensus on how to conceptualise, define and measure errors in diagnosis will pose a barrier to advancement. Further research should focus on identifying preventable missed opportunities in the diagnosis of mental disorders, which may uncover generalisable opportunities for improvement.

Patient generated research priorities to improve diagnostic safety: A systematic prioritization exercise.

BACKGROUND: Most people experience a diagnostic error at least once in their lifetime. Patients' experiences with their diagnosis could provide important insights when setting research priorities to reduce diagnostic error. OBJECTIVE: Our objective was to engage patients in research agenda setting for improving diagnosis. PATIENT INVOLVEMENT: Patients were involved in generating, discussing, prioritizing, and ranking of research questions for diagnostic error reduction. METHODS: We used the prioritization methodology based on the Child Health and Nutrition Research Initiative (CHNRI). We first solicited research questions important for diagnostic error reduction from a large group of patients. Thirty questions were initially prioritized at an in-person meeting with 8 patients who were supported by 4 researchers. The resulting list was further prioritized by patients who scored questions on five predefined criteria. We then applied previously determined weights to these prioritization criteria to adjust the final prioritization score for each question, resulting in 10 highest priority research questions. RESULTS: Forty-one patients submitted 171 research questions. After prioritization, the highest priority topics included better care coordination across the diagnostic continuum and improving care transitions, improved identification and measurement of diagnostic errors and attention for implicit bias towards patients who are vulnerable to diagnostic errors. DISCUSSION: We systematically identified the top-10 patient generated research priorities for diagnostic error reduction using transparent and objective methods. Patients prioritized different research questions than researchers and therefore complemented an agenda previously generated by researchers. PRACTICAL VALUE: Research priorities identified by patients can be used by funders and researchers to conduct future research focused on reducing diagnostic errors. FUNDING: This project was funded by the Gordon and Betty Moore Foundation.

Electronic Co-design (ECO-design) Workshop for Increasing Clinician Participation in the Design of Health Services Interventions: Participatory Design Approach.

BACKGROUND: Participation from clinician stakeholders can improve the design and implementation of health care interventions. Participatory design methods, especially co-design methods, comprise stakeholder-led design activities that are time-consuming. Competing work demands and increasing workloads make clinicians' commitments to typical participatory methods even harder. The COVID-19 pandemic further exacerbated barriers to clinician participation in such interventions. OBJECTIVE: The aim of this study was to explore a web-based participatory design approach to conduct economical, electronic co-design (ECO-design) workshops with primary care clinicians. METHODS: We adapted traditional in-person co-design workshops to web-based delivery and adapted co-design workshop series to fit within a single 1-hour session. We applied the ECO-design workshop approach to codevelop feedback interventions regarding abnormal test result follow-up in primary care. We conducted ECO-design workshops with primary care clinicians at a medical center in Southern Texas, using videoconferencing software. Each workshop focused on one of three types of feedback interventions: conversation guide, email template, and dashboard prototype. We paired electronic materials and software features to facilitate participant interactions, prototyping, and data collection. The workshop protocol included four main activities: problem identification, solution generation, prototyping, and debriefing. Two facilitators were assigned to each workshop and one researcher resolved technical problems. After the workshops, our research team met to debrief and evaluate workshops. RESULTS: A total of 28 primary care clinicians participated in our ECO-design workshops. We completed 4 parallel workshops, each with 5-10 participants. We conducted traditional analyses and generated a clinician persona (ie, representative description) and user interface prototypes. We also formulated recommendations for future ECO-design workshop recruitment, technology, facilitation, and data collection. Overall, our adapted workshops successfully enabled primary care clinicians to participate without increasing their workload, even during a pandemic. CONCLUSIONS: ECO-design workshops are viable, economical alternatives to traditional approaches. This approach fills a need for efficient methods to involve busy clinicians in the design of health care interventions.

Creating a Learning Health System for Improving Diagnostic Safety: Pragmatic Insights from US Health Care Organizations.

OBJECTIVE: To identify challenges and pragmatic strategies for improving diagnostic safety at an organizational level using concepts from learning health systems METHODS: We interviewed 32 safety leaders across the USA on how their organizations approach diagnostic safety. Participants were recruited through email and represented geographically diverse academic and non-academic settings. The interview included questions on culture of reporting and learning from diagnostic errors; data gathering and analysis activities; diagnostic training and educational activities; and engagement of clinical leadership, staff, patients, and families in diagnostic safety activities. We conducted an inductive content analysis of interview transcripts and two reviewers coded all data. RESULTS: Of 32 participants, 12 reported having a specific program to address diagnostic errors. Multiple barriers to implement diagnostic safety activities emerged: serious concerns about psychological safety associated with diagnostic error; lack of infrastructure for measurement, monitoring, and improvement activities related to diagnosis; lack of leadership investment, which was often diverted to competing priorities related to publicly reported measures or other incentives; and lack of dedicated teams to work on diagnostic safety. Participants provided several strategies to overcome barriers including adapting trigger tools to identify safety events, engaging patients in diagnostic safety, and appointing dedicated diagnostic safety champions. CONCLUSIONS: Several foundational building blocks related to learning health systems could inform organizational efforts to reduce diagnostic error. Promoting an organizational culture specific to diagnostic safety, using science and informatics to improve measurement and analysis, leadership incentives to build institutional capacity to address diagnostic errors, and patient engagement in diagnostic safety activities can enable progress.

Defining Diagnostic Error: A Scoping Review to Assess the Impact of the National Academies' Report Improving Diagnosis in Health Care.

BACKGROUND: Standards for accurate and timely diagnosis are ill-defined. In 2015, the National Academies of Science, Engineering, and Medicine (NASEM) committee published a landmark report, Improving Diagnosis in Health Care , and proposed a new definition of diagnostic error, "the failure to ( a ) establish an accurate and timely explanation of the patient's health problem(s) or ( b ) communicate that explanation to the patient." OBJECTIVE: This study aimed to explore how researchers operationalize the NASEM's definition of diagnostic error with relevance to accuracy, timeliness, and/or communication in peer-reviewed published literature. METHODS: Using the Arskey and O'Malley's framework framework, we identified published literature from October 2015 to February 2021 using Medline and Google Scholar. We also conducted subject matter expert interviews with researchers. RESULTS: Of 34 studies identified, 16 were analyzed and abstracted to determine how diagnostic error was operationalized and measured. Studies were grouped by theme: epidemiology, patient focus, measurement/surveillance, and clinician focus. Nine studies indicated using the NASEM definition. Of those, 5 studies also operationalized with existing definitions proposed before the NASEM report. Four studies operationalized the components of the NASEM definition and did not cite existing definitions. Three studies operationalized error using existing definitions only. Subject matter experts indicated that the NASEM definition functions as foundation for researchers to conceptualize diagnostic error. CONCLUSIONS: The NASEM report produced a common understanding of diagnostic error that includes accuracy, timeliness, and communication. In recent peer-reviewed literature, most researchers continue to use pre-NASEM report definitions to operationalize accuracy and timeliness. The report catalyzed the use of patient-centered concepts in the definition, resulting in emerging studies focused on examining errors related to communicating diagnosis to patients.

Inviting patients to identify diagnostic concerns through structured evaluation of their online visit notes.

BACKGROUND: The 21st Century Cures Act mandates patients' access to their electronic health record (EHR) notes. To our knowledge, no previous work has systematically invited patients to proactively report diagnostic concerns while documenting and tracking their diagnostic experiences through EHR-based clinician note review. OBJECTIVE: To test if patients can identify concerns about their diagnosis through structured evaluation of their online visit notes. METHODS: In a large integrated health system, patients aged 18-85 years actively using the patient portal and seen between October 2019 and February 2020 were invited to respond to an online questionnaire if an EHR algorithm detected any recent unexpected return visit following an initial primary care consultation ("at-risk" visit). We developed and tested an instrument (Safer Dx Patient Instrument) to help patients identify concerns related to several dimensions of the diagnostic process based on notes review and recall of recent "at-risk" visits. Additional questions assessed patients' trust in their providers and their general feelings about the visit. The primary outcome was a self-reported diagnostic concern. Multivariate logistic regression tested whether the primary outcome was predicted by instrument variables. RESULTS: Of 293 566 visits, the algorithm identified 1282 eligible patients, of whom 486 responded. After applying exclusion criteria, 418 patients were included in the analysis. Fifty-one patients (12.2%) identified a diagnostic concern. Patients were more likely to report a concern if they disagreed with statements "the care plan the provider developed for me addressed all my medical concerns" [odds ratio (OR), 2.65; 95% confidence interval [CI], 1.45-4.87) and "I trust the provider that I saw during my visit" (OR, 2.10; 95% CI, 1.19-3.71) and agreed with the statement "I did not have a good feeling about my visit" (OR, 1.48; 95% CI, 1.09-2.01). CONCLUSION: Patients can identify diagnostic concerns based on a proactive online structured evaluation of visit notes. This surveillance strategy could potentially improve transparency in the diagnostic process.

Technology-Based Closed-Loop Tracking for Improving Communication and Follow-up of Pathology Results.

OBJECTIVE: Failure to follow-up on laboratory test results can lead to missed diagnoses, diagnostic delays, patient harm, and potential malpractice claims against providers. State-of-the-art tracking technologies such as the radio frequency identification (RFID) can potentially improve laboratory order processing and test result communication. We conducted a comparative evaluation of differences in completion rates for 5 testing process milestones and time to reach these process milestones, with and without RFID order tracking for skin biopsy orders. METHODS: This observational study analyzed 48,515 orders from 20 dermatology providers, sent to 8 pathology laboratories in 2016 to 2017. Descriptive t tests and multiple Cox proportional hazard regressions were used to examine the differences in completion rates and times to the 5 testing process milestones, namely, (1) provider receipt of results, (2) provider review of results, (3) patient notification, (4) follow-up scheduling, and (5) order case closure, for orders processed with and without RFID order tracking. RESULTS: Descriptive statistics illustrated that all 5 testing process milestone completion rates were statistically higher for RFID tracked orders compared with non-RFID tracked orders, and RFID tracked orders took 3 to 5 days lesser than non-RFID tracked orders to reach the 5 testing process milestones. Multiple cox proportional hazard regressions showed that the process milestones were achieved faster if orders were RFID tracked versus not (hazard ratios ranged from 1.3 to 4.9). CONCLUSIONS: The RFID tracking technology considerably improved test result communication timeliness and reliability. Such technologies can be beneficial for laboratory order processing, and their effectiveness should be explored in other practice settings.

Why Test Results Are Still Getting "Lost" to Follow-up: a Qualitative Study of Implementation Gaps.

BACKGROUND: Lack of timely follow-up of abnormal test results is common and has been implicated in missed or delayed diagnosis, resulting in potential for patient harm. OBJECTIVE: As part of a larger project to implement change strategies to improve follow-up of diagnostic test results, this study sought to identify specifically where implementation gaps exist, as well as possible solutions identified by front-line staff. DESIGN: We used a semi-structured interview guide to collect qualitative data from Veterans Affairs (VA) facility staff who had experience with test results management and patient safety. SETTING: Twelve VA facilities across the USA. PARTICIPANTS: Facility staff members (n = 27), including clinicians, lab and imaging professionals, nursing staff, patient safety professionals, and leadership. APPROACH: We conducted a content analysis of interview transcripts to identify perceived barriers and high-risk areas for effective test result management, as well as recommendations for improvement. RESULTS: We identified seven themes to guide further development of interventions to improve test result follow-up. Themes related to trainees, incidental findings, tracking systems for electronic health record notifications, outdated contact information, referrals, backup or covering providers, and responsibility for test results pending at discharge. Participants provided recommendations for improvement within each theme. CONCLUSIONS: Perceived barriers and recommendations for improving test result follow-up often reflected previously known problems and their corresponding solutions, which have not been consistently implemented in practice. Better policy solutions and improvement methods, such as quality improvement collaboratives, may bridge the implementation gaps between knowledge and practice.

Patient and clinician experiences of uncertainty in the diagnostic process: Current understanding and future directions.

OBJECTIVE: Uncertainty occurs throughout the diagnostic process and must be managed to facilitate accurate and timely diagnoses and treatments. Better characterization of uncertainty can inform strategies to manage it more effectively in clinical practice. We provide a comprehensive overview of current literature on diagnosis-related uncertainty describing (1) where patients and clinicians experience uncertainty within the diagnostic process, (2) how uncertainty affects the diagnostic process, (3) roots of uncertainty related to probability/risk, ambiguity, or complexity, and (4) strategies to manage uncertainty. DISCUSSION: Each diagnostic process step involves uncertainty, including patient engagement with the healthcare system; information gathering, interpretation, and integration; formulating working diagnoses; and communicating diagnoses to patients. General management strategies include acknowledging uncertainty, obtaining more contextual information from patients (e.g., gathering occupations and family histories), creating diagnostic safety nets (e.g., informing patients what red flags to look for), engaging in worst case/best case scenario planning, and communicating diagnostic uncertainty to patients, families, and colleagues. Potential strategies tailored to various aspects of diagnostic uncertainty are also outlined. CONCLUSION: Scientific knowledge on diagnostic uncertainty, while previously elusive, is now becoming more clearly defined. Next steps include research to evaluate relationships between management and communication of diagnostic uncertainty and improved patient outcomes.

Use of patient complaints to identify diagnosis-related safety concerns: a mixed-method evaluation.

BACKGROUND: Patient complaints are associated with adverse events and malpractice claims but underused in patient safety improvement. OBJECTIVE: To systematically evaluate the use of patient complaint data to identify safety concerns related to diagnosis as an initial step to using this information to facilitate learning and improvement. METHODS: We reviewed patient complaints submitted to Geisinger, a large healthcare organisation in the USA, from August to December 2017 (cohort 1) and January to June 2018 (cohort 2). We selected complaints more likely to be associated with diagnostic concerns in Geisinger's existing complaint taxonomy. Investigators reviewed all complaint summaries and identified cases as 'concerning' for diagnostic error using the National Academy of Medicine's definition of diagnostic error. For all 'concerning' cases, a clinician-reviewer evaluated the associated investigation report and the patient's medical record to identify any missed opportunities in making a correct or timely diagnosis. In cohort 2, we selected a 10% sample of 'concerning' cases to test this smaller pragmatic sample as a proof of concept for future organisational monitoring. RESULTS: In cohort 1, we reviewed 1865 complaint summaries and identified 177 (9.5%) concerning reports. Review and analysis identified 39 diagnostic errors. Most were categorised as 'Clinical Care issues' (27, 69.2%), defined as concerns/questions related to the care that is provided by clinicians in any setting. In cohort 2, we reviewed 2423 patient complaint summaries and identified 310 (12.8%) concerning reports. The 10% sample (n=31 cases) contained five diagnostic errors. Qualitative analysis of cohort 1 cases identified concerns about return visits for persistent and/or worsening symptoms, interpersonal issues and diagnostic testing. CONCLUSIONS: Analysis of patient complaint data and corresponding medical record review identifies patterns of failures in the diagnostic process reported by patients and families. Health systems could systematically analyse available data on patient complaints to monitor diagnostic safety concerns and identify opportunities for learning and improvement.

An Operational Framework to Study Diagnostic Errors in Emergency Departments: Findings From A Consensus Panel.

OBJECTIVE: To create an operational definition and framework to study diagnostic error in the emergency department setting. METHODS: We convened a 17-member multidisciplinary panel with expertise in general and pediatric emergency medicine, nursing, patient safety, informatics, cognitive psychology, social sciences, human factors, and risk management and a patient/caregiver advocate. We used a modified nominal group technique to develop a shared understanding to operationally define diagnostic errors in emergency care and modify the National Academies of Sciences, Engineering, and Medicine's conceptual process framework to this setting. RESULTS: The expert panel defined diagnostic errors as "a divergence from evidence-based processes that increases the risk of poor outcomes despite the availability of sufficient information to provide a timely and accurate explanation of the patient's health problem(s)." Diagnostic processes include tasks related to (a) acuity recognition, information and synthesis, evaluation coordination, and (b) communication with patients/caregivers and other diagnostic team members. The expert panel also modified the National Academies of Sciences, Engineering, and Medicine's diagnostic process framework to incorporate influence of mode of arrival, triage level, and interventions during emergency care and underscored the importance of outcome feedback to emergency department providers to promote learning and improvement related to diagnosis. CONCLUSIONS: The proposed operational definition and modified diagnostic process framework can potentially inform the development of measurement tools and strategies to study the epidemiology and interventions to improve emergency care diagnosis.

Identifying trigger concepts to screen emergency department visits for diagnostic errors.

OBJECTIVES: The diagnostic process is a vital component of safe and effective emergency department (ED) care. There are no standardized methods for identifying or reliably monitoring diagnostic errors in the ED, impeding efforts to enhance diagnostic safety. We sought to identify trigger concepts to screen ED records for diagnostic errors and describe how they can be used as a measurement strategy to identify and reduce preventable diagnostic harm. METHODS: We conducted a literature review and surveyed ED directors to compile a list of potential electronic health record (EHR) trigger (e-triggers) and non-EHR based concepts. We convened a multidisciplinary expert panel to build consensus on trigger concepts to identify and reduce preventable diagnostic harm in the ED. RESULTS: Six e-trigger and five non-EHR based concepts were selected by the expert panel. E-trigger concepts included: unscheduled ED return to ED resulting in hospital admission, death following ED visit, care escalation, high-risk conditions based on symptom-disease dyads, return visits with new diagnostic/therapeutic interventions, and change of treating service after admission. Non-EHR based signals included: cases from mortality/morbidity conferences, risk management/safety office referrals, ED medical director case referrals, patient complaints, and radiology/laboratory misreads and callbacks. The panel suggested further refinements to aid future research in defining diagnostic error epidemiology in ED settings. CONCLUSIONS: We identified a set of e-trigger concepts and non-EHR based signals that could be developed further to screen ED visits for diagnostic safety events. With additional evaluation, trigger-based methods can be used as tools to monitor and improve ED diagnostic performance.

Health Care Provider Factors Associated with Patient-Reported Adverse Events and Harm.

BACKGROUND: Patients can provide valuable information missing from traditional sources of safety data, thus adding new insights about factors that lead to preventable harm. In this study, researchers determined associations between patient-reported contributory factors and patient-reported harms experienced after an adverse event (AE). METHODS: A secondary analysis was conducted of a national sample of patient-reported AEs (surgical, medication, diagnostic, and hospital-acquired infection) gathered through an online questionnaire between January 2010 and February 2016. Generalized logit multivariable regression was used to assess the association between patient-reported contributory factors and patient-reported harms (grouped as nonphysical harm only, physical harm only, physical harm and emotional or financial harm, and all three harms) and adjusted for patient and AE characteristics. RESULTS: One third of patients (32.6%) reported experiencing all three harms, 27.3% reported physical harms and one additional harm, 25.5% reported physical harms only, and 14.7% reported nonphysical harms only. Patients reporting all three harms were 2.5 times more likely to have filed a report with a responsible authority (95% confidence interval [CI] = 1.23-5.01) and 3.3 times more likely to have also experienced a surgical complication (95% CI = 1.42-7.51). Odds of reporting problems related to communication between clinician and patients/families or clinician-related behavioral issues was 13% higher in those experiencing all three harm types (95% CI = 1.07-1.19). CONCLUSION: Patients' experiences are important to identify safety issues and reduce harm and should be included in patient safety measurement and improvement activities. These findings underscore the need for policy and practice changes to identify, address, and support harmed patients.

Patient Perspectives on the Usefulness of an Artificial Intelligence-Assisted Symptom Checker: Cross-Sectional Survey Study.

BACKGROUND: Patients are increasingly seeking Web-based symptom checkers to obtain diagnoses. However, little is known about the characteristics of the patients who use these resources, their rationale for use, and whether they find them accurate and useful. OBJECTIVE: The study aimed to examine patients' experiences using an artificial intelligence (AI)-assisted online symptom checker. METHODS: An online survey was administered between March 2, 2018, through March 15, 2018, to US users of the Isabel Symptom Checker within 6 months of their use. User characteristics, experiences of symptom checker use, experiences discussing results with physicians, and prior personal history of experiencing a diagnostic error were collected. RESULTS: A total of 329 usable responses was obtained. The mean respondent age was 48.0 (SD 16.7) years; most were women (230/304, 75.7%) and white (271/304, 89.1%). Patients most commonly used the symptom checker to better understand the causes of their symptoms (232/304, 76.3%), followed by for deciding whether to seek care (101/304, 33.2%) or where (eg, primary or urgent care: 63/304, 20.7%), obtaining medical advice without going to a doctor (48/304, 15.8%), and understanding their diagnoses better (39/304, 12.8%). Most patients reported receiving useful information for their health problems (274/304, 90.1%), with half reporting positive health effects (154/302, 51.0%). Most patients perceived it to be useful as a diagnostic tool (253/301, 84.1%), as a tool providing insights leading them closer to correct diagnoses (231/303, 76.2%), and reported they would use it again (278/304, 91.4%). Patients who discussed findings with their physicians (103/213, 48.4%) more often felt physicians were interested (42/103, 40.8%) than not interested in learning about the tool's results (24/103, 23.3%) and more often felt physicians were open (62/103, 60.2%) than not open (21/103, 20.4%) to discussing the results. Compared with patients who had not previously experienced diagnostic errors (missed or delayed diagnoses: 123/304, 40.5%), patients who had previously experienced diagnostic errors (181/304, 59.5%) were more likely to use the symptom checker to determine where they should seek care (15/123, 12.2% vs 48/181, 26.5%; P=.002), but they less often felt that physicians were interested in discussing the tool's results (20/34, 59% vs 22/69, 32%; P=.04). CONCLUSIONS: Despite ongoing concerns about symptom checker accuracy, a large patient-user group perceived an AI-assisted symptom checker as useful for diagnosis. Formal validation studies evaluating symptom checker accuracy and effectiveness in real-world practice could provide additional useful information about their benefit.

An Exploration of Barriers, Facilitators, and Suggestions for Improving Electronic Health Record Inbox-Related Usability: A Qualitative Analysis.

Importance: Managing messages in the electronic health record (EHR) inbox consumes substantial amounts of physician time. Certain factors associated with inbox management, such as poor usability and excessive and unnecessary inbox messages, have been associated with physician burnout. Additionally, inbox design, usability, and workflows are associated with physicians' situational awareness (ie, perception, comprehension, and projection of clinical status) and efficiency of processing EHR inbox messages. Understanding factors associated with inbox usability could improve future EHR inbox designs and workflows, thus reducing risk of burnout while improving patient safety. Objective: To determine barriers, facilitators, and suggestions associated with EHR inbox-related usability. Design, Setting, and Participants: This qualitative study included cognitive walkthroughs of EHR inbox management with 25 physicians (17 primary care physicians and 8 specialists) at 6 large health care organizations using 4 different EHR systems between May 6, 2015, and September 19, 2016. While processing EHR inbox messages, participants identified facilitators and barriers associated with EHR inbox situational awareness and processing efficiency and potential interventions to address such barriers. A qualitative analysis was performed on transcribed recordings using an inductive thematic approach with an 8-dimension sociotechnical model as a theoretical lens from May 6, 2015, to August 15, 2019. Results: The cognitive walkthroughs identified 60 barriers, 32 facilitators, and 28 suggestions for improving the EHR inbox. Emergent data fit within 5 major themes: message processing complexity, inbox interface design, cognitive load, team communication, and inbox message content. Within these themes, similar barriers were identified across sites, such as poor usability due the high numbers of clicks needed to accomplish actions. In certain instances, an identified facilitator at one site provided the exact solution needed to address a barrier identified at another site. Conclusions and Relevance: This qualitative study found that usability of the EHR inbox is often suboptimal and variable across sites, suggesting lack of shared best practices related to information management. Implementation of optimized design features and workflows will require EHR developers and health care organizations to collectively share this responsibility. Development of regional or national consortia to support collaborative sharing and implementation of EHR system best practices across EHR developers and health care organizations could also improve safety and efficiency and reduce physician burnout.

Practicing Clinicians' Recommendations to Reduce Burden from the Electronic Health Record Inbox: a Mixed-Methods Study.

BACKGROUND: Workload from electronic health record (EHR) inbox notifications leads to information overload and contributes to job dissatisfaction and physician burnout. Better understanding of physicians' inbox requirements and workflows could optimize inbox designs, enhance efficiency, and reduce safety risks from information overload. DESIGN: We conducted a mixed-methods study to identify strategies to enhance EHR inbox design and workflow. First, we performed a secondary analysis of national survey data of all Department of Veterans Affairs (VA) primary care practitioners (PCP) to identify major themes in responses to a free-text question soliciting suggestions to improve EHR inbox design and workflows. We then conducted expert interviews of clinicians at five health care systems (1 VA and 4 non-VA settings using 4 different EHRs) to understand existing optimal strategies to improve efficiency and situational awareness related to EHR inbox use. Themes from survey data were cross-validated with interview findings. RESULTS: We analyzed responses from 2104 PCPs who completed the free-text inbox question (of 5001 PCPs who responded to survey) and used an inductive approach to identify five themes: (1) Inbox notification content should be actionable for patient care and relevant to recipient clinician, (2) Inboxes should reduce risk of losing messages, (3) Inbox functionality should be optimized to improve efficiency of processing notifications, (4) Team support should be leveraged to help with EHR inbox notification burden, (5) Sufficient time should be provided to all clinicians to process EHR inbox notifications. We subsequently interviewed 15 VA and non-VA clinicians and identified 11 unique strategies, each corresponding directly with one of these five themes. CONCLUSION: Feedback from practicing end-user clinicians provides robust evidence to improve content and design of the EHR inbox and related clinical workflows and organizational policies. Several strategies we identified could improve clinicians' EHR efficiency and satisfaction as well as empower them to work with their local administrators, health IT personnel, and EHR developers to improve these systems.

Pediatric clinician perspectives on communicating diagnostic uncertainty.

OBJECTIVE: Diagnosis often evolves over time, involves uncertainty, and is vulnerable to errors. We examined pediatric clinicians' perspectives on communicating diagnostic uncertainty to patients' parents and how this occurs. DESIGN: We conducted semi-structured interviews, which were audiotaped, transcribed, and analyzed using content analysis. Two researchers independently coded transcripts and then discussed discrepancies to reach consensus. SETTING: A purposive sample of pediatric clinicians at two large academic medical institutions in Texas. PARTICIPANTS: Twenty pediatric clinicians participated: 18 physicians, 2 nurse practitioners; 7 males, 13 females; 7 inpatient, 11 outpatient, and 2 practicing in mixed settings; with 0-16 years' experience post-residency. INTERVENTION(S): None. MAIN OUTCOME MEASURE(S): Pediatric clinician perspectives on communication of diagnostic uncertainty. RESULTS: Pediatric clinicians commonly experienced diagnostic uncertainty and most were comfortable seeking help and discussing with colleagues. However, when communicating uncertainty to parents, clinicians used multiple considerations to adjust the degree to which they communicated. Considerations included parent characteristics (education, socioeconomic status, emotional response, and culture) and strength of parent-clinician relationships. Communication content included setting expectations, explaining the diagnostic process, discussing most relevant differentials, and providing reassurance. Responses to certain parent characteristics, however, were variable. For example, some clinicians were more open to discussing diagnostic uncertainty with more educated parents- others were less. CONCLUSIONS: While pediatric clinicians are comfortable discussing diagnostic uncertainty with colleagues, how they communicate uncertainty to parents appears variable. Parent characteristics and parent-clinician relationships affect extent of communication and content discussed. Development and implementation of optimal strategies for managing and communicating diagnostic uncertainty can improve the diagnostic process.

Test results management and distributed cognition in electronic health record-enabled primary care.

Managing abnormal test results in primary care involves coordination across various settings. This study identifies how primary care teams manage test results in a large, computerized healthcare system in order to inform health information technology requirements for test results management and other distributed healthcare services. At five US Veterans Health Administration facilities, we interviewed 37 primary care team members, including 16 primary care providers, 12 registered nurses, and 9 licensed practical nurses. We performed content analysis using a distributed cognition approach, identifying patterns of information transmission across people and artifacts (e.g. electronic health records). Results illustrate challenges (e.g. information overload) as well as strategies used to overcome challenges. Various communication paths were used. Some team members served as intermediaries, processing information before relaying it. Artifacts were used as memory aids. Health information technology should address the risks of distributed work by supporting awareness of team and task status for reliable management of results.