Exploring Anesthesia Provider Preferences for Precision Feedback: Preference Elicitation Study.

Background: Health care professionals must learn continuously as a core part of their work. As the rate of knowledge production in biomedicine increases, better support for health care professionals' continuous learning is needed. In health systems, feedback is pervasive and is widely considered to be essential for learning that drives improvement. Clinical quality dashboards are one widely deployed approach to delivering feedback, but engagement with these systems is commonly low, reflecting a limited understanding of how to improve the effectiveness of feedback about health care. When coaches and facilitators deliver feedback for improving performance, they aim to be responsive to the recipient's motivations, information needs, and preferences. However, such functionality is largely missing from dashboards and feedback reports. Precision feedback is the delivery of high-value, motivating performance information that is prioritized based on its motivational potential for a specific recipient, including their needs and preferences. Anesthesia care offers a clinical domain with high-quality performance data and an abundance of evidence-based quality metrics. Objective: The objective of this study is to explore anesthesia provider preferences for precision feedback. Methods: We developed a test set of precision feedback messages with balanced characteristics across 4 performance scenarios. We created an experimental design to expose participants to contrasting message versions. We recruited anesthesia providers and elicited their preferences through analysis of the content of preferred messages. Participants additionally rated their perceived benefit of preferred messages to clinical practice on a 5-point Likert scale. Results: We elicited preferences and feedback message benefit ratings from 35 participants. Preferences were diverse across participants but largely consistent within participants. Participants' preferences were consistent for message temporality (α=.85) and display format (α=.80). Ratings of participants' perceived benefit to clinical practice of preferred messages were high (mean rating 4.27, SD 0.77). Conclusions: Health care professionals exhibited diverse yet internally consistent preferences for precision feedback across a set of performance scenarios, while also giving messages high ratings of perceived benefit. A "one-size-fits-most approach" to performance feedback delivery would not appear to satisfy these preferences. Precision feedback systems may hold potential to improve support for health care professionals' continuous learning by accommodating feedback preferences.

Ten simple rules to make computable knowledge shareable and reusable.

Computable biomedical knowledge (CBK) is: "the result of an analytic and/or deliberative process about human health, or affecting human health, that is explicit, and therefore can be represented and reasned upon using logic, formal standards, and mathematical approaches." Representing biomedical knowledge in a machine-interpretable, computable form increases its ability to be discovered, accessed, understood, and deployed. Computable knowledge artifacts can greatly advance the potential for implementation, reproducibility, or extension of the knowledge by users, who may include practitioners, researchers, and learners. Enriching computable knowledge artifacts may help facilitate reuse and translation into practice. Following the examples of 10 Simple Rules papers for scientific code, software, and applications, we present 10 Simple Rules intended to make shared computable knowledge artifacts more useful and reusable. These rules are mainly for researchers and their teams who have decided that sharing their computable knowledge is important, who wish to go beyond simply describing results, algorithms, or models via traditional publication pathways, and who want to both make their research findings more accessible, and to help others use their computable knowledge. These rules are roughly organized into 3 categories: planning, engineering, and documentation. Finally, while many of the following examples are of computable knowledge in biomedical domains, these rules are generalizable to computable knowledge in any research domain.

Early intranasal medication administration in out-of-hospital cardiac arrest: Two randomized simulation trials.

Objective: Intranasal medications have been proposed as adjuncts to out-of-hospital cardiac arrest (OHCA) care. We sought to quantify the effects of intranasal medication administration (INMA) in OHCA workflows. Methods: We conducted separate randomized OHCA simulation trials with lay rescuers (LRs) and first responders (FRs). Participants were randomized to groups performing hands-only cardiopulmonary resuscitation (CPR)/automated external defibrillator with or without INMA during the second analysis phase. Time to compression following the second shock (CPR2) was the primary outcome and compression quality (chest compression rate (CCR) and fraction (CCF)) was the secondary outcome. We fit linear regression models adjusted for CPR training in the LR group and service years in the FR group. Results: Among LRs, INMA was associated with a significant increase in CPR2 (mean diff. 44.1 s, 95% CI: 14.9, 73.3), which persisted after adjustment (p = 0.005). We observed a significant decrease in CCR (INMA 95.1 compressions per min (cpm) vs control 104.2 cpm, mean diff. -9.1 cpm, 95% CI -16.6, -1.6) and CCF (INMA 62.4% vs control 69.8%, mean diff. -7.5%, 95% CI -12.0, -2.9). Among FRs, we found no significant CPR2 delays (mean diff. -2.1 s, 95% CI -15.9, 11.7), which persisted after adjustment (p = 0.704), or difference in quality (CCR INMA 115.5 cpm vs control 120.8 cpm, mean diff. -5.3 cpm, 95% CI -12.6, 2.0; CCF INMA 79.6% vs control 81.2% mean diff. -1.6%, 95% CI -7.4, 4.3%). Conclusions: INMA in LR resuscitation was associated with diminished resuscitation performance. INMA by FR did not impede key times or quality.

Barriers to the Initiation of Telecommunicator-CPR during 9-1-1 Out-of-Hospital Cardiac Arrest Calls: A Qualitative Study.

INTRODUCTION: Fewer than 10% of individuals who suffer out-of-hospital cardiac arrest (OHCA) survive with good neurologic function. Bystander CPR more than doubles the chance of survival, and telecommunicator-CPR (T-CPR) during a 9-1-1 call substantially improves the frequency of bystander CPR. OBJECTIVE: We examined the barriers to initiation of T-CPR. METHODS: We analyzed the 9-1-1 call audio from 65 EMS-treated OHCAs from a single US 9-1-1 dispatch center. We initially conducted a thematic analysis aimed at identifying barriers to the initiation of T-CPR. We then conducted a conversation analysis that examined the interactions between telecommunicators and bystanders during the recognition phase (i.e., consciousness and normal breathing). RESULTS: We identified six process themes related to barriers, including incomplete or delayed recognition assessment, delayed repositioning, communication gaps, caller emotional distress, nonessential questions and assessments, and caller refusal, hesitation, or inability to act. We identified three suboptimal outcomes related to arrest recognition and delivery of chest compressions, which are missed OHCA identification, delayed OHCA identification and treatment, and compression instructions not provided following OHCA identification. A primary theme observed during missed OHCA calls was incomplete or delayed recognition assessment and included failure to recognize descriptors indicative of agonal breathing (e.g., "snoring", "slow") or to confirm that breathing was effective in an unconscious victim. CONCLUSIONS: We observed that modifiable barriers identified during 9-1-1 calls where OHCA was missed, or treatment was delayed, were often related to incomplete or delayed recognition assessment. Repositioning delays were a common barrier to the initiation of chest compressions.

Applying AI and Guidelines to Assist Medical Students in Recognizing Patients With Heart Failure: Protocol for a Randomized Trial.

BACKGROUND: The integration of artificial intelligence (AI) into clinical practice is transforming both clinical practice and medical education. AI-based systems aim to improve the efficacy of clinical tasks, enhancing diagnostic accuracy and tailoring treatment delivery. As it becomes increasingly prevalent in health care for high-quality patient care, it is critical for health care providers to use the systems responsibly to mitigate bias, ensure effective outcomes, and provide safe clinical practices. In this study, the clinical task is the identification of heart failure (HF) prior to surgery with the intention of enhancing clinical decision-making skills. HF is a common and severe disease, but detection remains challenging due to its subtle manifestation, often concurrent with other medical conditions, and the absence of a simple and effective diagnostic test. While advanced HF algorithms have been developed, the use of these AI-based systems to enhance clinical decision-making in medical education remains understudied. OBJECTIVE: This research protocol is to demonstrate our study design, systematic procedures for selecting surgical cases from electronic health records, and interventions. The primary objective of this study is to measure the effectiveness of interventions aimed at improving HF recognition before surgery, the second objective is to evaluate the impact of inaccurate AI recommendations, and the third objective is to explore the relationship between the inclination to accept AI recommendations and their accuracy. METHODS: Our study used a 3 × 2 factorial design (intervention type × order of prepost sets) for this randomized trial with medical students. The student participants are asked to complete a 30-minute e-learning module that includes key information about the intervention and a 5-question quiz, and a 60-minute review of 20 surgical cases to determine the presence of HF. To mitigate selection bias in the pre- and posttests, we adopted a feature-based systematic sampling procedure. From a pool of 703 expert-reviewed surgical cases, 20 were selected based on features such as case complexity, model performance, and positive and negative labels. This study comprises three interventions: (1) a direct AI-based recommendation with a predicted HF score, (2) an indirect AI-based recommendation gauged through the area under the curve metric, and (3) an HF guideline-based intervention. RESULTS: As of July 2023, 62 of the enrolled medical students have fulfilled this study's participation, including the completion of a short quiz and the review of 20 surgical cases. The subject enrollment commenced in August 2022 and will end in December 2023, with the goal of recruiting 75 medical students in years 3 and 4 with clinical experience. CONCLUSIONS: We demonstrated a study protocol for the randomized trial, measuring the effectiveness of interventions using AI and HF guidelines among medical students to enhance HF recognition in preoperative care with electronic health record data. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/49842.

Factors Impacting Treatment of Out-of-Hospital Cardiac Arrest: A Qualitative Study of Emergency Responders.

Background Of the more than 250 000 emergency medical services-treated out-of-hospital cardiac arrests that occur each year in the United States, only about 8% survive to hospital discharge with good neurologic function. Treatment for out-of-hospital cardiac arrest involves a system of care that includes complex interactions among multiple stakeholders. Understanding the factors inhibiting optimal care is fundamental to improving outcomes. Methods and Results We conducted group interviews with emergency responders including 911 telecommunicators, law enforcement officers, firefighters, and transporting emergency medical services personnel (ie, emergency medical technicians and paramedics) who responded to the same out-of-hospital cardiac arrest incident. We used the American Heart Association System of Care as the framework for our analysis to identify themes and their contributory factors from these interviews. We identified 5 themes under the structure domain, which included workload, equipment, prehospital communication structure, education and competency, and patient attitudes. In the process domain, 5 themes were identified focusing on preparedness, field response and access to patient, on-scene logistics, background information acquisition, and clinical interventions. We identified 3 system themes including emergency responder culture; community support, education, and engagement; and stakeholder relationships. Three continuous quality improvement themes were identified, which included feedback provision, change management, and documentation. Conclusions We identified structure, process, system, and continuous quality improvement themes that may be leveraged to improve outcomes for out-of-hospital cardiac arrest. Interventions or programs amenable to rapid implementation include improving prearrival communication between agencies, appointing patient care and logistical leadership on-scene, interstakeholder team training, and providing more standardized feedback to all responder groups.

CBK model composition using paired web services and executable functions: A demonstration for individualizing preventive services.

Introduction: Learning health systems are challenged to combine computable biomedical knowledge (CBK) models. Using common technical capabilities of the World Wide Web (WWW), digital objects called Knowledge Objects, and a new pattern of activating CBK models brought forth here, we aim to show that it is possible to compose CBK models in more highly standardized and potentially easier, more useful ways. Methods: Using previously specified compound digital objects called Knowledge Objects, CBK models are packaged with metadata, API descriptions, and runtime requirements. Using open-source runtimes and a tool we developed called the KGrid Activator, CBK models can be instantiated inside runtimes and made accessible via RESTful APIs by the KGrid Activator. The KGrid Activator then serves as a gateway and provides a means to interconnect CBK model outputs and inputs, thereby establishing a CBK model composition method. Results: To demonstrate our model composition method, we developed a complex composite CBK model from 42 CBK submodels. The resulting model called CM-IPP is used to compute life-gain estimates for individuals based their personal characteristics. Our result is an externalized, highly modularized CM-IPP implementation that can be distributed and made runnable in any common server environment. Discussion: CBK model composition using compound digital objects and the distributed computing technologies is feasible. Our method of model composition might be usefully extended to bring about large ecosystems of distinct CBK models that can be fitted and re-fitted in various ways to form new composites. Remaining challenges related to the design of composite models include identifying appropriate model boundaries and organizing submodels to separate computational concerns while optimizing reuse potential. Conclusion: Learning health systems need methods for combining CBK models from a variety of sources to create more complex and useful composite models. It is feasible to leverage Knowledge Objects and common API methods in combination to compose CBK models into complex composite models.

Assessment of telecommunicator cardiopulmonary resuscitation performance during out-of-hospital cardiac arrest using a standardized tool for audio review.

OBJECTIVE: Telecommunicator cardiopulmonary resuscitation (T-CPR) is a critical component of optimized out-of-hospital cardiac arrest (OHCA) care. We assessed a pilot tool to capture American Heart Association (AHA) T-CPR measures and T-CPR coaching by telecommunicators using audio review. METHODS: Using a pilot tool, we conducted a retrospective review of 911 call audio from 65 emergency medical services-treated out-of-hospital cardiac arrest (OHCA) patients. Data collection included events (e.g., OHCA recognition), time intervals, and coaching quality measures. We calculated summary statistics for all performance and quality measures. RESULTS: Among 65 cases, the patients' mean age was 64.7 years (SD: 14.6) and 17 (26.2%) were women. Telecommunicator recognition occurred in 72% of cases (47/65). Among 18 non-recognized cases, reviewers determined 12 (66%) were not recognizable based on characteristics of the call. Median time-to-recognition was 76 seconds (n = 40; IQR:39-138), while median time-to-first-instructed-compression was 198 seconds (n = 26; IQR:149-233). In 36 cases where coaching was needed, coaching on compression-depth occurred in 27 (75%); -rate in 28 (78%); and chest recoil in 10 (28%) instances. In 30 cases where repositioning was needed, instruction to position the patient's body flat occurred in 18 (60%) instances, on-back in 22 (73%) instances, and on-ground in 22 (73%) instances. CONCLUSIONS: Successful collection of data to calculate AHA T-CPR measures using a pilot tool for audio review revealed performance near AHA benchmarks, although coaching instructions did not occur in many instances. Application of this standardized tool may aid in T-CPR quality review.

Evaluating Completeness of Discrete Data on Physical Functioning for Children With Cerebral Palsy in a Pediatric Rehabilitation Learning Health System.

OBJECTIVE: The purpose of this study was to determine the extent that physical function discrete data elements (DDE) documented in electronic health records (EHR) are complete within pediatric rehabilitation settings. METHODS: A descriptive analysis on completeness of EHR-based DDEs detailing physical functioning for children with cerebral palsy was conducted. Data from an existing pediatric rehabilitation research learning health system data network, consisting of EHR data from 20 care sites in a pediatric specialty health care system, were leveraged. Completeness was calculated for unique data elements, unique outpatient visits, and unique outpatient records. RESULTS: Completeness of physical function DDEs was low across 5766 outpatient records (10.5%, approximately 2 DDEs documented). The DDE for Gross Motor Function Classification System level was available for 21% (n = 3746) outpatient visits and 38% of patient records. Ambulation level was the most frequently documented DDE. Intercept only mixed effects models demonstrated that 21.4% and 45% of the variance in completeness for DDEs and the Gross Motor Function Classification System, respectively, across unique patient records could be attributed to factors at the individual care site level. CONCLUSION: Values of physical function DDEs are missing in designated fields of the EHR infrastructure for pediatric rehabilitation providers. Although completeness appears limited for these DDEs, our observations indicate that data are not missing at random and may be influenced by system-level standards in clinical documentation practices between providers and factors specific to individual care sites. The extent of missing data has significant implications for pediatric rehabilitation quality measurement. More research is needed to understand why discrete data are missing in EHRs and to further elucidate the professional and system-level factors that influence completeness and missingness. IMPACT: Completeness of DDEs reported in this study is limited and presents a significant opportunity to improve documentation and standards to optimize EHR data for learning health system research and quality measurement in pediatric rehabilitation settings.

Partnership development of the COVID-19 Front Door: a best evidence resource.

This project describes the creation of a single searchable resource during the pandemic, called the COVID-19 Best Evidence Front Door, with a primary goal of providing direct access to high-quality meta-analyses, literature syntheses, and clinical guidelines from a variety of trusted sources. The Front Door makes relevant evidence findable and accessible with a single search to aggregated evidence-based resources, optimizing time, discovery, and improved access to quality scientific evidence while reducing the burden of frontline health care providers and other knowledge-seekers in needing to separately identify, locate, and explore multiple websites.

Use of electronic health records to support a public health response to the COVID-19 pandemic in the United States: a perspective from 15 academic medical centers.

Our goal is to summarize the collective experience of 15 organizations in dealing with uncoordinated efforts that result in unnecessary delays in understanding, predicting, preparing for, containing, and mitigating the COVID-19 pandemic in the US. Response efforts involve the collection and analysis of data corresponding to healthcare organizations, public health departments, socioeconomic indicators, as well as additional signals collected directly from individuals and communities. We focused on electronic health record (EHR) data, since EHRs can be leveraged and scaled to improve clinical care, research, and to inform public health decision-making. We outline the current challenges in the data ecosystem and the technology infrastructure that are relevant to COVID-19, as witnessed in our 15 institutions. The infrastructure includes registries and clinical data networks to support population-level analyses. We propose a specific set of strategic next steps to increase interoperability, overall organization, and efficiencies.

Reconsidering hospital EHR adoption at the dawn of HITECH: implications of the reported 9% adoption of a "basic" EHR.

OBJECTIVE: In 2009, a prominent national report stated that 9% of US hospitals had adopted a "basic" electronic health record (EHR) system. This statistic was widely cited and became a memetic anchor point for EHR adoption at the dawn of HITECH. However, its calculation relies on specific treatment of the data; alternative approaches may have led to a different sense of US hospitals' EHR adoption and different subsequent public policy. MATERIALS AND METHODS: We reanalyzed the 2008 American Heart Association Information Technology supplement and complementary sources to produce a range of estimates of EHR adoption. Estimates included the mean and median number of EHR functionalities adopted, figures derived from an item response theory-based approach, and alternative estimates from the published literature. We then plotted an alternative definition of national progress toward hospital EHR adoption from 2008 to 2018. RESULTS: By 2008, 73% of hospitals had begun the transition to an EHR, and the majority of hospitals had adopted at least 6 of the 10 functionalities of a basic system. In the aggregate, national progress toward basic EHR adoption was 58% complete, and, when accounting for measurement error, we estimate that 30% of hospitals may have adopted a basic EHR. DISCUSSION: The approach used to develop the 9% figure resulted in an estimate at the extreme lower bound of what could be derived from the available data and likely did not reflect hospitals' overall progress in EHR adoption. CONCLUSION: The memetic 9% figure shaped nationwide thinking and policy making about EHR adoption; alternative representations of the data may have led to different policy.

Understanding the roles of three academic communities in a prospective learning health ecosystem for diagnostic excellence.

Inaccurate, untimely, and miscommunicated medical diagnoses represent a wicked problem requiring comprehensive and coordinated approaches, such as those demonstrated in the characteristics of learning health systems (LHSs). To appreciate a vision for how LHS methods can optimize processes and outcomes in medical diagnosis (diagnostic excellence), we interviewed 32 individuals with relevant expertise: 18 who have studied diagnostic processes using traditional behavioral science and health services research methods, six focused on machine learning (ML) and artificial intelligence (AI) approaches, and eight multidisciplinary researchers experienced in advocating for and incorporating LHS methods, ie, scalable continuous learning in health care. We report on barriers and facilitators, identified by these subjects, to applying their methods toward optimizing medical diagnosis. We then employ their insights to envision the emergence of a learning ecosystem that leverages the tools of each of the three research groups to advance diagnostic excellence. We found that these communities represent a natural fit forward, in which together, they can better measure diagnostic processes and close the loop of putting insights into practice. Members of the three academic communities will need to network and bring in additional stakeholders before they can design and implement the necessary infrastructure that would support ongoing learning of diagnostic processes at an economy of scale and scope.