What AHRQ Learned While Working to Transform Primary Care.

Building on previous efforts to transform primary care, the Agency for Healthcare Research and Quality (AHRQ) launched EvidenceNOW: Advancing Heart Health in 2015. This 3-year initiative provided external quality improvement support to small and medium-size primary care practices to implement evidence-based cardiovascular care. Despite challenges, results from an independent national evaluation demonstrated that the EvidenceNOW model successfully boosted the capacity of primary care practices to improve quality of care, while helping to advance heart health. Reflecting on AHRQ's own learnings as the funder of this work, 3 key lessons emerged: (1) there will always be surprises that will require flexibility and real-time adaptation; (2) primary care transformation is about more than technology; and (3) it takes time and experience to improve care delivery and health outcomes. EvidenceNOW taught us that lasting practice transformation efforts need to be responsive to anticipated and unanticipated changes, relationship-oriented, and not tied to a specific disease or initiative. We believe these lessons argue for a national primary care extension service that provides ongoing support for practice transformation.

A Critical Appraisal of AHRQ's "Diagnostic Errors" Report.

The Agency for Healthcare Research and Quality (AHRQ) report "Diagnostic Errors in the Emergency Department" generated significant mass media interest with negative implications for the safety of contemporary emergency care. The assumptions and methodology underlying this report are problematic, while multiple ongoing efforts to improve the quality and quantity of diagnostic research are missed, neglected, or ignored. The AHRQ report identifies reasonable target diseases for targeting diagnostic quality improvement efforts, as well as viable methods by which to measure any initiatives impact on diagnostic error. We note additional opportunities to improve the status quo by funding emergency department-based diagnostic research and healthcare system-level patient safety research and highlighting innovative approaches to diagnostic science within emergency medicine.

Update of the agency for healthcare research and quality guidance on using nonrandomized studies in evidence syntheses.

▪.

Feasibility of Capturing Adverse Events From Insurance Claims Data Using International Classification of Diseases, Tenth Revision, Codes Coupled to Present on Admission Indicators.

OBJECTIVE: The aim of the study was to investigate the feasibility of using administrative data to screen adverse events in Korea. METHODS: We used a diagnosis-related groups claims data set and the information of the checklist of healthcare quality improvement (a part of the value incentive program) to verify adverse events in fiscal year 2018. Adverse events were identified using patient safety indicator (PSI) clusters and a present on admission indicator (POA). The PSIs consisted of 19 clusters representing subcategories of adverse events, such as hospital-acquired infection. Among the adverse events identified using PSI clusters, "POA = N," which means not present at the time of admission, was only deemed as the case in the final stage. We compared the agreement on the occurrence of adverse events from claims data with a reference standard data set (i.e., checklist of healthcare quality improvement) and presented them by PSI cluster and institution. RESULTS: The cases of global PSI for any adverse event numbered 27,320 (2.32%) among all diagnostic codes in 2018. In terms of institutional distribution, considerable variation was observed throughout the clusters. For example, only 13.2% of institutions (n = 387) reported any global PSI for any adverse event throughout the whole year. The agreement between the reference standard and the claims data was poor, in the range of 2.2% to 10.8%, in 3 types of adverse events. The current claims data system (i.e., diagnostic codes coupled to POA indicators) failed to capture a large majority of adverse events identified using the reference standard. CONCLUSIONS: Our results imply that the coding status of International Classification of Diseases, Tenth Revision, codes and POA indicators should be refined before using them as quality indicators.

Relationships Between Pediatric Safety Indicators Across a National Sample of Pediatric Hospitals: Dispelling the Myth of the "Safest" Hospital.

OBJECTIVE: There are many measures of healthcare quality, but no obvious summary measures to simplify ranking of hospital performance. With public reporting and accountability for hospital performance, the validity of composite measures for performance rankings has increased importance. This study aimed to explore the covariance of pediatric hospital quality indicators and evaluate the use of a single composite score. METHODS: We performed an observational study of pediatric hospital performance across 13 safety indicators extracted from the Pediatric Health Information System, a comparative database of children's hospitals in the United States. We included patients discharged from 36 hospitals from January 1, 2016, to December 31, 2019. Using principal components analysis, we investigate relationships among patient safety measures from the Agency for Healthcare Research and Quality pediatric quality indicators and Center for Medicare and Medicaid Services hospital-acquired conditions. We compare and summarize rankings based on individual safety indicators and calculate alternative composite scores. RESULTS: We identified 5 orthogonal variance components accounting for 68% of variation in pediatric hospital quality indicators. Rankings demonstrated greater within-hospital variation compared with between-hospital variation. We observed discordant rankings across commonly used summary measures and conclude that these pediatric safety measures demonstrate at least 2 underlying variance components. CONCLUSIONS: This study demonstrates the multifactorial nature of patient safety. This implies no unique ordering of hospitals based on these measures, and thus, no pediatric hospital can claim to be "the safest." This raises further questions about appropriate methods to rank hospitals by safety.

Implementation of an Antibiotic Stewardship Program in Long-term Care Facilities Across the US.

IMPORTANCE: Antibiotic overuse in long-term care (LTC) is common, prompting calls for antibiotic stewardship programs (ASPs) designed for specific use in these settings. The optimal approach to establish robust, sustainable ASPs in LTC facilities is unknown. OBJECTIVES: To determine if the Agency for Healthcare Research and Quality (AHRQ) Safety Program for Improving Antibiotic Use, an educational initiative to establish ASPs focusing on patient safety, is associated with reductions in antibiotic use in LTC settings. DESIGN, SETTING, AND PARTICIPANTS: This quality improvement study including 439 LTC facilities in the US assessed antibiotic therapy data following a pragmatic quality-improvement program, which was implemented to assist facilities in establishing ASPs and with antibiotic decision-making. Training was conducted between December 2018 and November 2019. Data were analyzed from January 2019 to December 2019. INTERVENTIONS: Fifteen webinars occurred over 12 months (December 2018 to November 2019), accompanied by additional tools, activities, posters, and pocket cards. All clinical staff were encouraged to participate. MAIN OUTCOMES AND MEASURES: The primary outcome was antibiotic starts per 1000 resident-days. Secondary outcomes included days of antibiotic therapy (DOT) per 1000 resident-days, the number of urine cultures per 1000 resident-days, and Clostridioides difficile laboratory-identified events per 10 000 resident-days. All outcomes compared data from the baseline (January-February 2019) to the completion of the program (November-December 2019). Generalized linear mixed models with random intercepts at the site level assessed changes over time. RESULTS: Of a total 523 eligible LTC facilities, 439 (83.9%) completed the safety program. The mean difference for antibiotic starts from baseline to study completion per 1000 resident-days was -0.41 (95% CI, -0.76 to -0.07; P = .02), with fluoroquinolones showing the greatest decrease at -0.21 starts per 1000 resident-days (95% CI, -0.35 to -0.08; P = .002). The mean difference for antibiotic DOT per 1000 resident-days was not significant (-3.05; 95% CI, -6.34 to 0.23; P = .07). Reductions in antibiotic starts and use were greater in facilities with greater program engagement (as measured by webinar attendance). While antibiotic starts and DOT in these facilities decreased by 1.12 per 1000 resident-days (95% CI, -1.75 to -0.49; P < .001) and 9.97 per 1000 resident-days (95% CI, -15.4 to -4.6; P < .001), respectively, no significant reductions occurred in low engagement facilities. Urine cultures per 1000 resident-days decreased by 0.38 (95% CI, -0.61 to -0.15; P = .001). There was no significant change in facility-onset C difficile laboratory-identified events. CONCLUSIONS AND RELEVANCE: Participation in the AHRQ safety program was associated with the development of ASPs that actively engaged clinical staff in the decision-making processes around antibiotic prescriptions in participating LTC facilities. The reduction in antibiotic DOT and starts, which was more pronounced in more engaged facilities, indicates that implementation of this multifaceted program may support successful ASPs in LTC settings.

Realizing the Dream: The Future of Primary Care Research.

Primary care research is central to the successful transformation of care delivery, providing the crucial evidence to overcome the longstanding and widespread threats and challenges to the realization of primary care's full potential. The Agency for Healthcare Research and Quality (AHRQ), as the federal agency specifically charged with conducting and supporting primary care research, plays a pivotal role in supporting the research and generating the evidence needed to advance primary care. Drawing upon decades of AHRQ-supported research studies, extensive stakeholder consultation, and a Primary Care Research Summit held in fall 2020, we discuss the primary care research central to successful primary care transformation and for realizing the vision of a high-performing US health system to effectively serve all Americans and their communities while advancing health equity.Realizing the potential of primary care will require wise investments in primary care research. Newly generated evidence needs to be rapidly incorporated into the design of the delivery system, clinical care, and community interventions. Investments in evidence-informed primary care redesign can catalyze progress to achieving the quintuple aim-improved health outcomes, increased value, better patient and clinician experience, and health equity. Primary care research can provide the evidence to help stem the twin epidemics of clinician burnout and lack of trust in the health system. Actualizing this vision will require a concerted and coordinated effort by policy makers, researchers, clinicians, and community members and a commitment to ensuring people and communities have ready access to primary care.Appeared as Annals "Online First" article.

Incorporating harms into the weighting of the revised Agency for Healthcare Research and Quality Patient Safety for Selected Indicators Composite (Patient Safety Indicator 90).

OBJECTIVE: To reweight the Agency for Healthcare Research and Quality Patient Safety for Selected Indicators Composite (Patient Safety Indicator [PSI] 90) from weights based solely on the frequency of component PSIs to those that incorporate excess harm reflecting patients' preferences for outcome-related health states. DATA SOURCES: National administrative and claims data involving hospitalizations in nonfederal, nonrehabilitation, acute care hospitals. STUDY DESIGN: We estimated the average excess aggregate harm associated with the occurrence of each component PSI using a cohort sample for each indicator based on denominator-eligible records. We used propensity scores to account for potential confounding in the risk models for each PSI and weighted observations to estimate the "average treatment effect in the treated" for those with the PSI event. We fit separate regression models for each harm outcome. Final PSI weights reflected both the disutilities and the frequencies of the harms. DATA COLLECTION/EXTRACTION METHODS: We estimated PSI frequencies from the 2012 Healthcare Cost and Utilization Project State Inpatient Databases with present on admission data and excess harms using 2012-2013 Centers for Medicare & Medicaid Services Medicare Fee-for-Service data. PRINCIPAL FINDINGS: Including harms in the weighting scheme changed individual component weights from the original frequency-based weighting. In the reweighted composite, PSIs 11 ("Postoperative Respiratory Failure"), 13 ("Postoperative Sepsis"), and 12 ("Perioperative Pulmonary Embolism or Deep Vein Thrombosis") contributed the greatest harm, with weights of 29.7%, 21.1%, and 20.4%, respectively. Regarding reliability, the overall average hospital signal-to-noise ratio for the reweighted PSI 90 was 0.7015. Regarding discrimination, among hospitals with greater than median volume, 34% had significantly better PSI 90 performance, and 41% had significantly worse performance than benchmark rates (based on percentiles). CONCLUSIONS: Reformulation of PSI 90 with harm-based weights is feasible and results in satisfactory reliability and discrimination, with a more clinically meaningful distribution of component weights.

Gaps in standardized postoperative pain management quality measures: A systematic review.

BACKGROUND: The goal of this study was an assessment of availability postoperative pain management quality measures and National Quality Forum-endorsed measures. Postoperative pain is an important clinical timepoint because poor pain control can lead to patient suffering, chronic opiate use, and/or chronic pain. Quality measures can guide best practices, but it is unclear whether there are measures for managing pain after surgery. METHODS: The National Quality Forum Quality Positioning System, Agency for Healthcare Research and Quality Indicators, and Centers for Medicare and Medicaid Services Measures Inventory Tool databases were searched in November 2019. We conducted a systematic literature review to further identify quality measures in research publications, clinical practice guidelines, and gray literature for the period between March 11, 2015 and March 11, 2020. RESULTS: Our systematic review yielded 1,328 publications, of which 206 were pertinent. Nineteen pain management quality measures were identified from the quality measure databases, and 5 were endorsed by National Quality Forum. The National Quality Forum measures were not specific to postoperative pain management. Three of the non-endorsed measures were specific to postoperative pain. CONCLUSION: The dearth of published postoperative pain management quality measures, especially National Quality Forum-endorsed measures, highlights the need for more rigorous evidence and widely endorsed postoperative pain quality measures to guide best practices.

Utility of the AHRQ Learning Collaboratives Taxonomy for Analyzing Innovations from an Australian Collaborative.

BACKGROUND: Despite the proliferation of learning collaborations such as innovation platforms, the factors contributing to their success or failure are rarely documented. The Agency for Healthcare Research and Quality learning collaboratives taxonomy provides a framework for understanding how such collaborations work in different settings according to four primary elements: innovation, communication, time, and social systems. This study applied the taxonomy to assess an innovation platform and the utility of applying the taxonomy. METHODS: The study focus was a five-year national research collaboration operating as an innovation platform to strengthen primary health care quality improvement efforts for Indigenous Australians. The study team analyzed project records, reports and publications, and interviews that were conducted with 35 stakeholders. Data were mapped retrospectively against the taxonomy domains and thematically analyzed. RESULTS: The taxonomy proved useful in understanding how and why the innovation platform generated innovations. It revealed that time was particularly important, both to see innovations through and to establish a social system that enabled interconnectivity between members. However, the taxonomy did not provide useful guidance on identifying the types of innovations from the collaboration or the importance of a culture of continuous adaptation and learning. The study also found that the primary and secondary elements of the taxonomy were not discrete, which meant that it was difficult to align themes with only one element. CONCLUSION: To improve the utility of the taxonomy, several elaborations are proposed, including reconfiguring it to a more dynamic form that recognizes the interconnections and links between the elements.

Clinical impact of compound sarcopenia in hospitalized older adult patients with heart failure.

OBJECTIVES: Skeletal muscle loss or sarcopenia is a frequent complication in heart failure (HF) and contributes to adverse clinical outcomes. We evaluated if age (primary) and chronic disease (secondary) related sarcopenia, that we refer to as compound sarcopenia, impacts clinical outcomes in hospitalized patients with HF. DESIGN: Cross-sectional study using hospitalized patient data. SETTING: Data from the Agency for Healthcare Research and Quality through the Healthcare Cost and Utilization Project (HCUP). PARTICIPANTS: Hospitalized adult patients with a primary or secondary diagnosis of HF (n = 64,476) and a concurrent random 2% sample of general medical population (GMP; n = 322,217) stratified by age (<50 years of age [y], 51-65y, >65y) from the Nationwide Inpatient Sample (NIS) database (years 2010-2014). MEASUREMENTS: In-hospital mortality, length of stay (LoS), cost of hospitalization per admission (CoH), comorbidities and discharge disposition, with and without muscle loss phenotype, were analyzed. Muscle loss phenotype was defined using a comprehensive code set from international classification of diseases-9 (ICD-9). RESULTS: Muscle loss phenotype was observed in 8673 (13.5%) patients with HF compared to 5213 (1.6%) GMP across all age strata. In patients with HF, muscle loss phenotype was associated with higher mortality, LoS, and CoH. Patients with HF (>65y) and muscle loss phenotype had higher mortality (adjusted OR: 1.81; 95% CI 1.56-2.10), CoH (adjusted OR 1.48; 95% CI 1.44-1.1.52), and LoS (adjusted OR 1.40; 95% CI 1.37-1.43) compared to >65y GMP with muscle loss phenotype. CONCLUSION: Muscle loss phenotype is more commonly associated with increasing age in hospitalized patients with HF. Clinical outcomes were significantly worse in patients with HF aged >65y compared to younger patients with HF and all age strata in GMP with and without a muscle loss phenotype.

Association of a Safety Program for Improving Antibiotic Use With Antibiotic Use and Hospital-Onset Clostridioides difficile Infection Rates Among US Hospitals.

Importance: Regulatory agencies and professional organizations recommend antibiotic stewardship programs (ASPs) in US hospitals. The optimal approach to establish robust, sustainable ASPs across diverse hospitals is unknown. Objective: To assess whether the Agency for Healthcare Research and Quality Safety Program for Improving Antibiotic Use is associated with reductions in antibiotic use across US hospitals. Design, Setting, and Participants: A pragmatic quality improvement program was conducted and evaluated over a 1-year period in US hospitals. A total of 437 hospitals were enrolled. The study was conducted from December 1, 2017, to November 30, 2018. Data analysis was performed from March 1 to October 31, 2019. Interventions: The Safety Program assisted hospitals with establishing ASPs and worked with frontline clinicians to improve their antibiotic decision-making. All clinical staff (eg, clinicians, pharmacists, and nurses) were encouraged to participate. Seventeen webinars occurred over 12 months, accompanied by additional durable educational content. Topics focused on establishing ASPs, the science of safety, improving teamwork and communication, and best practices for the diagnosis and management of infectious processes. Main Outcomes and Measures: The primary outcome was overall antibiotic use (days of antibiotic therapy [DOT] per 1000 patient days [PD]) comparing the beginning (January-February 2018) and end (November-December 2018) of the Safety Program. Data analysis occurred using linear mixed models with random hospital unit effects. Antibiotic use from 614 hospitals in the Premier Healthcare Database from the same period was analyzed to evaluate contemporary US antibiotic trends. Quarterly hospital-onset Clostridioides difficile laboratory-identified events per 10 000 PD were a secondary outcome. Results: Of the 437 hospitals enrolled, 402 (92%) remained in the program until its completion, including 28 (7%) academic medical centers, 122 (30%) midlevel teaching hospitals, 167 (42%) community hospitals, and 85 (21%) critical access hospitals. Adherence to key components of ASPs (ie, interventions before and after prescription of antibiotics, availability of local antibiotic guidelines, ASP leads with dedicated salary support, and quarterly reporting of antibiotic use) improved from 8% to 74% over the 1-year period (P < .01). Antibiotic use decreased by 30.3 DOT per 1000 PD (95% CI, -52.6 to -8.0 DOT; P = .008). Similar changes in antibiotic use were not observed in the Premier Healthcare Database. The incidence rate of hospital-onset C difficile laboratory-identified events decreased by 19.5% (95% CI, -33.5% to -2.4%; P = .03). Conclusions and Relevance: The Agency for Healthcare Research and Quality Safety Program appeared to enable diverse hospitals to establish ASPs and teach frontline clinicians to self-steward their antibiotic use. Safety Program content is publicly available.

NxGen evidence: Redesigning the Agency for Healthcare Research and Quality Effective Health Care website to promote engagement, interactivity, and usability of systematic reviews.

Evidence-based decision-making is predicated on the ability of users to find and comprehend results from systematic review. Evidence producers have an obligation to support evidence users in this process. The Agency for Healthcare Research and Quality (AHRQ) Evidence-based Practice Center (EPC) program-a producer of rigorous and comprehensive systematic reviews for two decades-has set a gold standard for reliability in health evidence reviews in the United States. It has recently begun a program of active support for evidence dissemination and uptake beyond mere publication of lengthy reports. This Brief Methods Note critiques the current paper-based format for systematic reviews and describes the development of a next generation (NxGen) AHRQ EPC Effective Health Care website. This redesigned platform will allow end-users of all types to find and share the evidence they need through data visualizations and other interactive displays. Several design principles guided the development of NxGen to make systematic review findings more accessible, customizable, adaptable, interactive, and shareable. NxGen will include visualizations for report results that are expressed as meta-analyses as well as those with narrative syntheses, through forest or bubble plots, respectively. Visual and interactive evidence heat maps are also planned. The NxGen version of the Effective Health Care website is planned to go live in the latter half of 2020 or early 2021.

Assessment of Automating Safety Surveillance From Electronic Health Records: Analysis for the Quality and Safety Review System.

BACKGROUND AND OBJECTIVES: In an effort to improve and standardize the collection of adverse event data, the Agency for Healthcare Research and Quality is developing and testing a patient safety surveillance system called the Quality and Safety Review System (QSRS). Its current abstraction from medical records is through manual human coders, taking an average of 75 minutes to complete the review and abstraction tasks for one patient record. With many healthcare systems across the country adopting electronic health record (EHR) technology, there is tremendous potential for more efficient abstraction by automatically populating QSRS. In the absence of real-world testing data and models, which require a substantial investment, we provide a heuristic assessment of the feasibility of automatically populating QSRS questions from EHR data. METHODS: To provide an assessment of the automation feasibility for QSRS, we first developed a heuristic framework, the Relative Abstraction Complexity Framework, to assess relative complexity of data abstraction questions. This framework assesses the relative complexity of characteristics or features of abstraction questions that should be considered when determining the feasibility of automating QSRS. Questions are assigned a final relative complexity score (RCS) of low, medium, or high by a team of clinicians, human factors, and natural language processing researchers. RESULTS: One hundred thirty-four QSRS questions were coded using this framework by a team of natural language processing and clinical experts. Fifty-five questions (41%) had high RCS and would be more difficult to automate, such as "Was use of a device associated with an adverse outcome(s)?" Forty-two questions (31%) had medium RCS, such as "Were there any injuries as a result of the fall(s)?" and 37 questions (28%) had low RCS, such as "Did the patient deliver during this stay?" These results suggest that Blood and Hospital Acquired Infections-Clostridium Difficile Infection (HAI-CDI) modules would be relatively easier to automate, whereas Surgery and HAI-Surgical Site Infection would be more difficult to automate. CONCLUSIONS: Although EHRs contain a wealth of information, abstracting information from these records is still very challenging, particularly for complex questions, such as those concerning patient adverse events. In this work, we developed a heuristic framework, which can be applied to help guide conversations around the feasibility of automating QSRS data abstraction. This framework does not aim to replace testing with real data but complement the process by providing initial guidance and direction to subject matter experts to help prioritize, which abstraction questions to test for feasibility using real data.

Patient and caregiver engagement in the Patient-Centered Outcomes Research Institute (PCORI) Health Care Horizon Scanning System (HCHSS) process.

OBJECTIVE: The Patient-Centered Outcomes Research Institute (PCORI) horizon scanning system is an early warning system for healthcare interventions in development that could disrupt standard care. We report preliminary findings from the patient engagement process. METHODS: The system involves broadly scanning many resources to identify and monitor interventions up to 3 years before anticipated entry into U.S. health care. Topic profiles are written on included interventions with late-phase trial data and circulated with a structured review form for stakeholder comment to determine disruption potential. Stakeholders include patients and caregivers recruited from credible community sources. They view an orientation video, comment on topic profiles, and take a survey about their experience. RESULTS: As of March 2020, 312 monitored topics (some of which were archived) were derived from 3,500 information leads; 121 met the criteria for topic profile development and stakeholder comment. We invited fifty-four patients and caregivers to participate; thirty-nine reviewed at least one report. Their perspectives informed analyst nominations for fourteen topics in two 2019 High Potential Disruption Reports. Thirty-four patient stakeholders completed the user-experience survey. Most agreed (68 percent) or somewhat agreed (26 percent) that they were confident they could provide useful comments. Ninety-four percent would recommend others to participate. CONCLUSIONS: The system has successfully engaged patients and caregivers, who contributed unique and important perspectives that informed the selection of topics deemed to have high potential to disrupt clinical care. Most participants would recommend others to participate in this process. More research is needed to inform optimal patient and caregiver stakeholder recruitment and engagement methods and reduce barriers to participation.

Developing Criteria and Associated Instructions for Consistent and Useful Quality Improvement Study Data Extraction for Health Systems.

BACKGROUND: The Agency for Healthcare Research and Quality (AHRQ) could devote resources to collate and assess quality improvement studies to support learning health systems (LHS) but there is no reliable data on the consistency of data extraction for important criteria. METHODS: We identified quality improvement studies and evaluated the consistency of data extraction from two experienced independent reviewers at three time points: baseline, first revision (where explicit instructions for each criterion were created), and final revision (where the instructions were revised). Six investigators looked at the data extracted by the two systematic reviewers and determined the extent of similarity on a scale of 0 to 10 (where 0 represented no similarity and 10 perfect similarity). There were 42 assessments for baseline, 42 assessments for the first revision, and 42 assessments for the final revision. We asked two LHS participants to assess the relative value of our criteria. RESULTS: The consistency of extraction improved from 1.17 ± 1.85 at baseline to 6.07 ± 2.76 after revision 1 (P < 0.001) and to 6.81 ± 1.94 out of 10 for the final revision (P < 0.001). However, the final revision was not significantly improved over the first revision (P = 0.14). One key informant rated the difficulty in finding and using quality improvement studies a 6 (moderately difficult) while the other a 4 (moderately difficult). When asked how valuable it would be if AHRQ found and collated the demographic information about the health systems and the interventions used in published quality improvement studies, they rated it a 9 (highly valuable) and a 6 (moderately valuable). CONCLUSION: Creating explicit instructions for extracting data for quality improvement studies helps enhance the consistency of data extraction. This is important because it is difficult for LHS to vet these quality improvement studies on their own and they would value AHRQ's support in that regard.