Improving the utility of evidence synthesis for decision makers in the face of insufficient evidence.

OBJECTIVE: To identify and suggest strategies to make insufficient evidence ratings in systematic reviews more actionable. STUDY DESIGN AND SETTING: A workgroup comprising members from the Evidence-Based Practice (EPC) Program of the Agency for Healthcare Research and Quality convened throughout 2020. We conducted iterative discussions considering information from three data sources: a literature review for relevant publications and frameworks, a review of a convenience sample of past systematic reviews conducted by the EPCs, and an audit of methods used in past EPC technical briefs. RESULTS: We identified five strategies for supplementing systematic review findings when evidence on benefits or harms is expected to be, or found to be, insufficient: 1) reconsider eligible study designs, 2) summarize indirect evidence, 3) summarize contextual and implementation evidence, 4) consider modelling, and 5) incorporate unpublished health system data in the evidence synthesis. While these strategies may not increase the strength of evidence, they may improve the utility of reports for decision makers. Adopting these strategies depends on feasibility, timeline, funding, and expertise of the systematic reviewers. CONCLUSION: Throughout the process of evidence synthesis of early scoping, protocol development, review conduct, and review presentation, authors can consider these five strategies to supplement evidence with insufficient rating to make it more actionable for end-users.

A prospective comparison of evidence synthesis search strategies developed with and without text-mining tools.

OBJECTIVE: We compared the process of developing searches with and without using text-mining tools (TMTs) for evidence synthesis products. STUDY DESIGN: This descriptive comparative analysis included seven systematic reviews, classified as simple or complex. Two librarians created MEDLINE strategies for each review, using either usual practice (UP) or TMTs. For each search we calculated sensitivity, number-needed-to-read (NNR) and time spent developing the search strategy. RESULTS: We found UP searches were more sensitive (UP 92% (95% CI, 85-99); TMT 84.9% (95% CI, 74.4-95.4)), with lower NNR (UP 83 (SD 34); TMT 90 (SD 68)). UP librarians spent an average of 12 h (SD 8) developing search strategies, compared to TMT librarians' 5 hours (SD 2). CONCLUSION: Across all reviews, TMT searches were less sensitive than UP searches, but confidence intervals overlapped. For simple SR topics, TMT searches were faster and slightly less sensitive than UP. For complex SR topics, TMT searches were faster and less sensitive than UP searches but identified unique eligible citations not found by the UP searches.

Retention Strategies for Medications for Opioid Use Disorder in Adults: A Rapid Evidence Review.

OBJECTIVES: Although medications for opioid use disorder (MOUD) save lives, treatment retention remains challenging. Identification of interventions to improve MOUD retention is of interest to policymakers and researchers. On behalf of the Agency for Healthcare Research and Quality, we conducted a rapid evidence review on interventions to improve MOUD retention. METHODS: We searched MEDLINE and the Cochrane Library from February 2009 through August 2019 for systematic reviews and randomized trials of care settings, services, logistical support, contingency management, health information technology (IT), extended-release (XR) formulations, and psychosocial interventions that assessed retention at least 3 months. RESULTS: Two systematic reviews and 39 primary studies were included; most did not focus on retention as the primary outcome. Initiating MOUD in soon-to-be-released incarcerated people improved retention following release. Contingency management may improve retention using antagonist but not agonist MOUD. Retention with interventions integrating medical, psychiatric, social services, or IT did not differ from in-person treatment-as-usual approaches. Retention was comparable with XR- compared to daily buprenorphine formulations and conflicting with XR-naltrexone monthly injection compared to daily buprenorphine. Most psychosocial interventions did not improve retention. DISCUSSION: Consistent but sparse evidence supports criminal justice prerelease MOUD initiation, and contingency management interventions for antagonist MOUD. Integrating MOUD with medical, psychiatric, social services, delivering through IT, or administering via XR-MOUD formulations did not worsen retention. Fewer than half of the studies we identified focused on retention as a primary outcome. Studies used different measures of retention, making it difficult to compare effectiveness. Additional inquiry into the causes of low retention would inform future interventions.Registration: PROSPERO: CRD42019134739.