Design and implementation of electronic health record-based tools to support a weight management program in primary care.

Objectives: This paper reports on a mixed methods formative evaluation to support the design and implementation of information technology (IT) tools for a primary care weight management intervention delivered through the patient portal using primary care staff as coaches. Methods: We performed a qualitative needs assessment, designed the IT tools to support the weight management program, and developed implementation tracking metrics. Implementation tracking metrics were designed to use real world electronic health record (EHR) data. Results: The needs assessment revealed IT requirements as well as barriers and facilitators to implementation of EHR-based weight management interventions in primary care. We developed implementation metrics for the IT tools. These metrics were used in weekly project team calls to make sure that project resources were allocated to areas of need. Conclusion: This study identifies the important role of IT in supporting weight management through patient identification, weight and activity tracking in the patient portal, and the use of the EHR as a population management tool. An intensive multi-level implementation approach is required for successful primary care-based weight management interventions including well-designed IT tools, comprehensive involvement of clinic leadership, and implementation tracking metrics to guide the process of workflow integration. This study helps to bridge the gap between informatics and implementation by using socio-technical formative evaluation methods early in order to support the implementation of IT tools. Trial registration: clinicaltrials.gov, NCT04420936. Registered June 9, 2020.

Artificial intelligence predictive analytics in heart failure: results of the pilot phase of a pragmatic randomized clinical trial.

OBJECTIVES: We conducted an implementation planning process during the pilot phase of a pragmatic trial, which tests an intervention guided by artificial intelligence (AI) analytics sourced from noninvasive monitoring data in heart failure patients (LINK-HF2). MATERIALS AND METHODS: A mixed-method analysis was conducted at 2 pilot sites. Interviews were conducted with 12 of 27 enrolled patients and with 13 participating clinicians. iPARIHS constructs were used for interview construction to identify workflow, communication patterns, and clinician's beliefs. Interviews were transcribed and analyzed using inductive coding protocols to identify key themes. Behavioral response data from the AI-generated notifications were collected. RESULTS: Clinicians responded to notifications within 24 hours in 95% of instances, with 26.7% resulting in clinical action. Four implementation themes emerged: (1) High anticipatory expectations for reliable patient communications, reduced patient burden, and less proactive provider monitoring. (2) The AI notifications required a differential and tailored balance of trust and action advice related to role. (3) Clinic experience with other home-based programs influenced utilization. (4) Responding to notifications involved significant effort, including electronic health record (EHR) review, patient contact, and consultation with other clinicians. DISCUSSION: Clinician's use of AI data is a function of beliefs regarding the trustworthiness and usefulness of the data, the degree of autonomy in professional roles, and the cognitive effort involved. CONCLUSION: The implementation planning analysis guided development of strategies that addressed communication technology, patient education, and EHR integration to reduce clinician and patient burden in the subsequent main randomized phase of the trial. Our results provide important insights into the unique implications of implementing AI analytics into clinical workflow.

"Are we there yet?" Ten persistent hazards and inefficiencies with the use of medication administration technology from the perspective of practicing nurses.

OBJECTIVES: (1) Characterize persistent hazards and inefficiencies in inpatient medication administration; (2) Explore cognitive attributes of medication administration tasks; and (3) Discuss strategies to reduce medication administration technology-related hazards. MATERIALS AND METHODS: Interviews were conducted with 32 nurses practicing at 2 urban, eastern and western US health systems. Qualitative analysis using inductive and deductive coding included consensus discussion, iterative review, and coding structure revision. We abstracted hazards and inefficiencies through the lens of risks to patient safety and the cognitive perception-action cycle (PAC). RESULTS: Persistent safety hazards and inefficiencies related to MAT organized around the PAC cycle included: (1) Compatibility constraints create information silos; (2) Missing action cues; (3) Intermittent communication flow between safety monitoring systems and nurses; (4) Occlusion of important alerts by other, less helpful alerts; (5) Dispersed information: Information required for tasks is not collocated; (6) Inconsistent data organization: Mismatch of the display and the user's mental model; (7) Hidden medication administration technologies (MAT) limitations: Inaccurate beliefs about MAT functionality contribute to overreliance on the technology; (8) Software rigidity causes workarounds; (9) Cumbersome dependencies between technology and the physical environment; and (10) Technology breakdowns require adaptive actions. DISCUSSION: Errors might persist in medication administration despite successful Bar Code Medication Administration and Electronic Medication Administration Record deployment for reducing errors. Opportunities to improve MAT require a deeper understanding of high-level reasoning in medication administration, including control over the information space, collaboration tools, and decision support. CONCLUSION: Future medication administration technology should consider a deeper understanding of nursing knowledge work for medication administration.

Pediatric Septic Shock Care Pathways in General Emergency Departments: A Qualitative Study Targeting How to Really Make it Work.

OBJECTIVES: Many academic pediatric emergency departments (PEDs) have successfully implemented pediatric septic shock care pathways. However, many general emergency departments (GEDs), who see the majority of pediatric ED visits, have not. This study aims to compare the workflow, resources, communication, and decision making across these 2 settings to inform the future implementation of a standardized care pathway for children with septic shock in the GED. METHODS: We used the critical incident technique to conduct semistructured interviews with 24 ED physicians, nurses, and technicians at one PED and 2 GEDs regarding pediatric septic shock care. We performed a thematic analysis using the Framework Method to develop our coding schema through inductive and deductive analyses. We continued an iterative process of revising the schema until we reached consensus agreement and thematic saturation. RESULTS: We identified the following 6 themes: (1) functioning like a "well-oiled machine" may be key to high performance; (2) experiencing the sequence of care for children with sepsis as invariant and predictable may be essential to high-quality performance; (3) resilience and flexibility are characteristic of high levels of performance; (4) believing that "the buck stops here" may contribute to more accountability; (5) continuous system learning is essential; and (6) computerized clinical decision support may not be optimized to drive decision-making at the point of care. Commentary from GED and PED participants differed across the 6 themes, providing insight into the approach for standardized care pathway implementation in GEDs. CONCLUSIONS: Pediatric septic shock workflow, decision making, and system performance differ between the PED and GEDs. Implementation of a standardized care pathway in GEDs will require a tailored approach. Specific recommendations include (1) improving shared situation awareness; (2) simulation for knowledge, skill, and team-based training; and (3) promoting a culture of continuous learning.

Clinical trial recruitment in primary care: exploratory factor analysis of a questionnaire to measure barriers and facilitators to primary care providers' involvement.

BACKGROUND: Recruitment of sufficient participants for clinical trials remains challenging. Primary care is an important avenue for patient recruitment but is underutilized. We developed and pilot tested a questionnaire to measure relevant barriers and facilitators to primary care providers' involvement in recruiting patients for clinical trials. METHODS: Prior research informed the development of the questionnaire. The initial instrument was revised using feedback obtained from cognitive interviews. We invited all primary care providers practicing within the University of Utah Health system to complete the revised questionnaire. We used a mixed-mode design to collect paper responses via in-person recruitment and email contacts to collect responses online. Descriptive statistics, exploratory factor analysis, Cronbach's alpha, and multivariable regression analyses were conducted. RESULTS: Sixty-seven primary care providers participated in the survey. Exploratory factor analysis suggested retaining five factors, representing the importance of clinical trial recruitment in providers' professional identity, clinic-level interventions to facilitate referral, patient-related barriers, concerns about patient health management, and knowledge gaps. The five factors exhibited good or high internal consistency reliability. Professional identity and clinic-level intervention factors were significant predictors of providers' intention to participate in clinical trial recruitment activities. CONCLUSIONS: Results of this exploratory analysis provide preliminary evidence of the internal structure, internal consistency reliability, and predictive validity of the questionnaire to measure factors relevant to primary care providers' involvement in clinical trial recruitment.

Accuracies of Training Labels and Machine Learning Models: Experiments on Delirium and Simulated Data.

Supervised predictive models require labeled data for training purposes. Complete and accurate labeled data is not always available, and imperfectly labeled data may need to serve as an alternative. An important question is if the accuracy of the labeled data creates a performance ceiling for the trained model. In this study, we trained several models to recognize the presence of delirium in clinical documents using data with annotations that are not completely accurate. In the external evaluation, the support vector machine model with a linear kernel performed best, achieving an area under the curve of 89.3% and accuracy of 88%, surpassing the 80% accuracy of the training sample. We then generated a set of simulated data and carried out a series of experiments which demonstrated that models trained on imperfect data can (but do not always) outperform the accuracy of the training data.

Implementing lung cancer screening in primary care: needs assessment and implementation strategy design.

Lung cancer screening with low-dose computed tomography (CT) could help avert thousands of deaths each year. Since the implementation of screening is complex and underspecified, there is a need for systematic and theory-based strategies. Explore the implementation of lung cancer screening in primary care, in the context of integrating a decision aid into the electronic health record. Design implementation strategies that target hypothesized mechanisms of change and context-specific barriers. The study had two phases. The Qualitative Analysis phase included semi-structured interviews with primary care physicians to elicit key task behaviors (e.g., ordering a low-dose CT) and understand the underlying behavioral determinants (e.g., social influence). The Implementation Strategy Design phase consisted of defining implementation strategies and hypothesizing causal pathways to improve screening with a decision aid. Three key task behaviors and four behavioral determinants emerged from 14 interviews. Implementation strategies were designed to target multiple levels of influence. Strategies included increasing provider self-efficacy toward performing shared decision making and using the decision aid, improving provider performance expectancy toward ordering a low-dose CT, increasing social influence toward performing shared decision making and using the decision aid, and addressing key facilitators to using the decision aid. This study contributes knowledge about theoretical determinants of key task behaviors associated with lung cancer screening. We designed implementation strategies according to causal pathways that can be replicated and tested at other institutions. Future research is needed to evaluate the effectiveness of these strategies and to determine the contexts in which they can be effectively applied.

Computerized Mortality Prediction for Community-acquired Pneumonia at 117 Veterans Affairs Medical Centers.

Rationale: Computerized severity assessment for community-acquired pneumonia could improve consistency and reduce clinician burden. Objectives: To develop and compare 30-day mortality-prediction models using electronic health record data, including a computerized score with all variables from the original Pneumonia Severity Index (PSI) except confusion and pleural effusion ("ePSI score") versus models with additional variables. Methods: Among adults with community-acquired pneumonia presenting to emergency departments at 117 Veterans Affairs Medical Centers between January 1, 2006, and December 31, 2016, we compared an ePSI score with 10 novel models employing logistic regression, spline, and machine learning methods using PSI variables, age, sex and 26 physiologic variables as well as all 69 PSI variables. Models were trained using encounters before January 1, 2015; tested on encounters during and after January 1, 2015; and compared using the areas under the receiver operating characteristic curve, confidence intervals, and patient event rates at a threshold PSI score of 970. Results: Among 297,498 encounters, 7% resulted in death within 30 days. When compared using the ePSI score (confidence interval [CI] for the area under the receiver operating characteristic curve, 0.77-0.78), performance increased with model complexity (CI for the logistic regression PSI model, 0.79-0.80; CI for the boosted decision-tree algorithm machine learning PSI model using the Extreme Gradient Boosting algorithm [mlPSI] with the 19 original PSI factors, 0.83-0.85) and the number of variables (CI for the logistic regression PSI model using all 69 variables, 0.84-085; CI for the mlPSI with all 69 variables, 0.86-0.87). Models limited to age, sex, and physiologic variables also demonstrated high performance (CI for the mlPSI with age, sex, and 26 physiologic factors, 0.84-0.85). At an ePSI score of 970 and a mortality-risk cutoff of <2.7%, the ePSI score identified 31% of all patients as being at "low risk"; the mlPSI with age, sex, and 26 physiologic factors identified 53% of all patients as being at low risk; and the mlPSI with all 69 variables identified 56% of all patients as being at low risk, with similar rates of mortality, hospitalization, and 7-day secondary hospitalization being determined. Conclusions: Computerized versions of the PSI accurately identified patients with pneumonia who were at low risk of death. More complex models classified more patients as being at low risk of death and as having similar adverse outcomes.

Shedding Light on the Black Box: Explaining Deep Neural Network Prediction of Clinical Outcomes.

Deep neural network models are emerging as an important method in healthcare delivery, following the recent success in other domains such as image recognition. Due to the multiple non-linear inner transformations, deep neural networks are viewed by many as black boxes. For practical use, deep learning models require explanations that are intuitive to clinicians. In this study, we developed a deep neural network model to predict outcomes following major cardiovascular procedures, using temporal image representation of past medical history as input. We created a novel explanation for the prediction of the model by defining impact scores that associate clinical observations with the outcome. For comparison, a logistic regression model was fitted to the same dataset. We compared the impact scores and log odds ratios by calculating three types of correlations, which provided a partial validation of the impact scores. The deep neural network model achieved an area under the receiver operating characteristics curve (AUC) of 0.787, compared to 0.746 for the logistic regression model. Moderate correlations were found between the impact scores and the log odds ratios. Impact scores generated by the explanation algorithm has the potential to shed light on the "black box" deep neural network model and could facilitate its adoption by clinicians.

Feeling and thinking: can theories of human motivation explain how EHR design impacts clinician burnout?

The psychology of motivation can help us understand the impact of electronic health records (EHRs) on clinician burnout both directly and indirectly. Informatics approaches to EHR usability tend to focus on the extrinsic motivation associated with successful completion of clearly defined tasks in clinical workflows. Intrinsic motivation, which includes the need for autonomy, sense-making, creativity, connectedness, and mastery is not well supported by current designs and workflows. This piece examines existing research on the importance of 3 psychological drives in relation to healthcare technology: goal-based decision-making, sense-making, and agency/autonomy. Because these motives are ubiquitous, foundational to human functioning, automatic, and unconscious, they may be overlooked in technological interventions. The results are increased cognitive load, emotional distress, and unfulfilling workplace environments. Ultimately, we hope to stimulate new research on EHR design focused on expanding functionality to support intrinsic motivation, which, in turn, would decrease burnout and improve care.

Integrated displays to improve chronic disease management in ambulatory care: A SMART on FHIR application informed by mixed-methods user testing.

OBJECTIVE: The study sought to evaluate a novel electronic health record (EHR) add-on application for chronic disease management that uses an integrated display to decrease user cognitive load, improve efficiency, and support clinical decision making. MATERIALS AND METHODS: We designed a chronic disease management application using the technology framework known as SMART on FHIR (Substitutable Medical Applications and Reusable Technologies on Fast Healthcare Interoperability Resources). We used mixed methods to obtain user feedback on a prototype to support ambulatory providers managing chronic obstructive pulmonary disease. Each participant managed 2 patient scenarios using the regular EHR with and without access to our prototype in block-randomized order. The primary outcome was the percentage of expert-recommended ideal care tasks completed. Timing, keyboard and mouse use, and participant surveys were also collected. User experiences were captured using a retrospective think-aloud interview analyzed by concept coding. RESULTS: With our prototype, the 13 participants completed more recommended care (81% vs 48%; P < .001) and recommended tasks per minute (0.8 vs 0.6; P = .03) over longer sessions (7.0 minutes vs 5.4 minutes; P = .006). Keystrokes per task were lower with the prototype (6 vs 18; P < .001). Qualitative themes elicited included the desire for reliable presentation of information which matches participants' mental models of disease and for intuitive navigation in order to decrease cognitive load. DISCUSSION: Participants completed more recommended care by taking more time when using our prototype. Interviews identified a tension between using the inefficient but familiar EHR vs learning to use our novel prototype. Concept coding of user feedback generated actionable insights. CONCLUSIONS: Mixed methods can support the design and evaluation of SMART on FHIR EHR add-on applications by enhancing understanding of the user experience.

Standards-Based Clinical Decision Support Platform to Manage Patients Who Meet Guideline-Based Criteria for Genetic Evaluation of Familial Cancer.

PURPOSE: The ubiquitous adoption of electronic health records (EHRs) with family health history (FHH) data provides opportunities for tailoring cancer screening strategies to individuals. We aimed to enable a standards-based clinical decision support (CDS) platform for identifying and managing patients who meet guidelines for genetic evaluation of hereditary cancer. METHODS: The CDS platform (www.opencds.org) was used to implement algorithms based on the 2018 National Comprehensive Cancer Network guidelines for genetic evaluation of hereditary breast/ovarian and colorectal cancer. The platform was designed to be interfaced with different EHR systems via the Health Level Seven International Fast Healthcare Interoperability Resources standard. The platform was integrated with the Epic EHR and evaluated in a pilot study at an academic health care system. RESULTS: The CDS platform was executed against a target population of 143,012 patients; 5,245 (3.7%) met criteria for genetic evaluation based on the FHH recorded in the EHR. In a clinical pilot study, genetic counselors attempted to reach out to 71 of the patients. Of those patients, 25 (35%) scheduled an appointment, 10 (14%) declined, 2 (3%) did not need genetic counseling, 7 (10%) said they would consider it in the future, and 27 (38%) were unreachable. To date, 13 (52%) of the scheduled patients completed visits, and 2 (15%) of those were found to have pathogenic variants in cancer predisposition genes. CONCLUSION: A standards-based CDS platform integrated with EHR systems is a promising population-based approach to identify patients who are appropriate candidates for genetic evaluation of hereditary cancers.

Discordance in Information Exchange Between Providers During Care Transitions for Surgical Patients.

BACKGROUND: The exchange of health information between primary care providers (PCPs) and surgeons is critical during transitions of care for older patients with multiple comorbidities; however, it is unknown to what extent this process occurs. This study was designed to characterize the extent to which factors associated with older patient's recovery, such as functional status, cognitive status, social status, and emotional factors, are shared among PCPs and surgical providers during care transitions. MATERIALS AND METHODS: We prospectively identified 15 patients aged over 60 y with ≥3 comorbidities referred for general and vascular surgery procedures at a Veterans Administrative and academic medical center. Semistructured Critical Decision Method interviews were conducted with patients along with their surgical providers and referring PCPs. Thematic content analysis was performed independently by five reviewers on the cognitive processes associated with functional status, cognitive status, social status, and emotional factors. Interrater reliability between providers and patients was assessed using Cohen's kappa. RESULTS: Forty-seven Critical Decision Method interviews were conducted, which included 20 paired interviews between a PCP and a surgeon and 16 paired interviews that involved a patient and a provider. The majority of patients reported experiencing poor information exchange between their PCP and surgeon (58%) and feeling they were primarily responsible for communicating their own health information during care transitions (67%). In paired interviews between PCPs and surgeons, there was nearly perfect agreement for the shared knowledge of cognitive (kappa: 0.83) and emotional (kappa 1) factors. In contrast, there was only minimal agreement for shared knowledge of functional status (kappa 0.38) and social status (kappa: 0.34). CONCLUSIONS: Information exchange between PCPs and surgical providers is often discordant during transitions of surgical care for medically complex older patients, particularly when it pertains to communicating their functional or social status.

CDS in a Learning Health Care System: Identifying Physicians' Reasons for Rejection of Best-Practice Recommendations in Pneumonia through Computerized Clinical Decision Support.

BACKGROUND: Local implementation of guidelines for pneumonia care is strongly recommended, but the context of care that affects implementation is poorly understood. In a learning health care system, computerized clinical decision support (CDS) provides an opportunity to both improve and track practice, providing insights into the implementation process. OBJECTIVES: This article examines physician interactions with a CDS to identify reasons for rejection of guideline recommendations. METHODS: We implemented a multicenter bedside CDS for the emergency department management of pneumonia that integrated patient data with guideline-based recommendations. We examined the frequency of adoption versus rejection of recommendations for site-of-care and antibiotic selection. We analyzed free-text responses provided by physicians explaining their clinical reasoning for rejection, using concept mapping and thematic analysis. RESULTS: Among 1,722 patient episodes, physicians rejected recommendations to send a patient home in 24%, leaving text in 53%; reasons for rejection of the recommendations included additional or alternative diagnoses beyond pneumonia, and comorbidities or signs of physiologic derangement contributing to risk of outpatient failure that were not processed by the CDS. Physicians rejected broad-spectrum antibiotic recommendations in 10%, leaving text in 76%; differences in pathogen risk assessment, additional patient information, concern about antibiotic properties, and admitting physician preferences were given as reasons for rejection. CONCLUSION: While adoption of CDS recommendations for pneumonia was high, physicians rejecting recommendations frequently provided feedback, reporting alternative diagnoses, additional individual patient characteristics, and provider preferences as major reasons for rejection. CDS that collects user feedback is feasible and can contribute to a learning health system.

Incorporating Guideline Adherence and Practice Implementation Issues into the Design of Decision Support for Beta-Blocker Titration for Heart Failure.

BACKGROUND: The recognition of and response to undertreatment of heart failure (HF) patients can be complicated. A clinical reminder can facilitate use of guideline-concordant ß-blocker titration for HF patients with depressed ejection fraction. However, the design must consider the cognitive demands on the providers and the context of the work. OBJECTIVE: This study's purpose is to develop requirements for a clinical decision support tool (a clinical reminder) by analyzing the cognitive demands of the task along with the factors in the Cabana framework of physician adherence to guidelines, the health information technology (HIT) sociotechnical framework, and the Promoting Action on Research Implementation in Health Services (PARIHS) framework of health services implementation. It utilizes a tool that extracts information from medical records (including ejection fraction in free text reports) to identify qualifying patients at risk of undertreatment. METHODS: We conducted interviews with 17 primary care providers, 5 PharmDs, and 5 Registered Nurses across three Veterans Health Administration outpatient clinics. The interviews were based on cognitive task analysis (CTA) methods and enhanced through the inclusion of the Cabana, HIT sociotechnical, and PARIHS frameworks. The analysis of the interview data led to the development of requirements and a prototype design for a clinical reminder. We conducted a small pilot usability assessment of the clinical reminder using realistic clinical scenarios. RESULTS: We identified organizational challenges (such as time pressures and underuse of pharmacists), knowledge issues regarding the guideline, and information needs regarding patient history and treatment status. We based the design of the clinical reminder on how to best address these challenges. The usability assessment indicated the tool could help the decision and titration processes. CONCLUSION: Through the use of CTA methods enhanced with adherence, sociotechnical, and implementation frameworks, we designed a decision support tool that considers important challenges in the decision and execution of ß-blocker titration for qualifying HF patients at risk of undertreatment.

Patient and caregiver perspectives on care coordination during transitions of surgical care.

Care coordination for patients with chronic disease commonly involves multiple transitions between primary care and surgical providers. These transitions often cross healthcare settings, providers, and information systems. We performed a cross-sectional qualitative study to gain a better understanding of the factors that influence how patients and caregivers perceive care coordination during transitions of surgical care. Eight focus groups were conducted among individuals from three different U.S. states who had experienced an episode of surgical care within the past year. We included patients who had undergone major surgery for a chronic condition, as well as caregivers. We used Atlas.ti qualitative software and engaged in an iterative process of thematic analysis of focus group transcripts. After five-rounds of review, five main themes emerged that define chronic care coordination for surgical patients and caregivers: (a) Care coordination is embedded in the unwritten social con tract patients share with their surgical providers; (b) Patients expect all surgical and nonsurgical healthcare providers to be "on the same page"; (c) Patients are frightened and vulnerable during surgical care transitions; (d) Patients need to have accurate expectations of the processes associated with care coordination; and (e) Care coordination relies upon establishing patient trust with their surgical team and needs to be continually reaffirmed. Surgical patients and caregivers expect care coordination processes to involve informatics infrastructure, patient education, and information exchange between providers. Unfortunately, these aspects of care coordination are often lacking during transitions. These findings have implications for designing patient-centered interventions to improve coordination of chronic care.

Understanding Primary Care Providers' Information Gathering Strategies in the Care of Children and Youth with Special Health Care Needs.

Background: Effective care coordination of children and youth with special health care needs (CYSHCN) is critical but challenging. Objective: To investigate clinicians' information-gathering strategies while preparing for visits with CYSHCN. Methods: Critical incident interviews with primary care physicians and care coordinators. Results: Six themes emerged indicating 1) substantial reliance on the electronic health record; 2) a central role of the problem list in organizing and summarizing information; 3) Medical Home's central role in organizing clinical documentation; 4) universal need to integrate information from external records; 5) lack of well-organized and labeled encounter documentation; and 6) lack of tools reconcile medication lists. Conclusion: Our findings have important implications to the design of informatics tools to support information-gathering in the care of CYSHCN.

A Pragmatic Guide to Establishing Clinical Decision Support Governance and Addressing Decision Support Fatigue: a Case Study.

There is limited guidance available in the literature for establishing clinical decision support (CDS) governance and improving CDS effectiveness in a pragmatic, resource-efficient manner. Here, we describe how University of Utah Health established enterprise CDS governance in 2015 leveraging existing resources. Key components of the governance include a multi-stakeholder CDS Committee that vets new requests and reviews existing content; a requirement that proposed CDS is actually desired by intended recipients; coordination with other governance bodies; basic data analytics to identify high-frequency, low-value CDS and monitor progress; active solicitation of user issues; the transition of alert and reminder content to other, more appropriate areas in the electronic health record; and the judicious use of experimental designs to guide decision-making regarding CDS effectiveness. In the three years since establishing this governance, new CDS has been continuously added while the overall burden of clinician-facing alerts and reminders has been reduced by 53.8%.

Integration of Clinical Decision Support and Electronic Clinical Quality Measurement: Domain Expert Insights and Implications for Future Direction.

The integration of clinical decision support (CDS) and electronic clinical quality measurement (eCQM) could help improve consistency, reduce redundancy, and ultimately help improve value. To guide efforts in this area, 15 leading experts in CDS and eCQM were interviewed to obtain insights on how CDS and eCQM could be better integrated. Four main themes emerged: cultural and business considerations for CDS, eCQM, and their integration trump the technical considerations; the purpose and goals of CDS and eCQM differ, and these differences must be accounted for; there is an oftentimes invisible overlap between CDS and eCQM, and with the larger domain of quality improvement; and despite the differences, synergies between CDS and eCQM should be pursued to amplify the effectiveness of each approach. A key implication of these findings is the need to bridge cultural and business differences between CDS and eCQM, with informaticists potentially playing a critical role.