ROMPER: The RAND/USC OPTIC Method for Policy Expert Ratings.

We developed an expert panel approach for identifying expert views on the effectiveness and implementability of population-level policy interventions. ROMPER-the RAND/USC OPTIC Method for Policy Expert Ratings-involves an online, three-round, modified-Delphi process:•Experts rate and comment on policies according to domains of the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Evidence-to-Decision framework.•To identify consensus on policy effectiveness and implementability, expert ratings are analyzed using the Inter-Percentile Range Adjusted for Symmetry (IPRAS) technique from the RAND/UCLA Appropriateness Method and visualized using a forest plot. To explain consensus, expert comments are analyzed using reflexive thematic analysis and reported following the Standards for Reporting Qualitative Research.•To provide actionable information for decisionmakers, each policy is summarized in a "Policy Profile" adapted from GRADEPro Evidence-to-Decision tables.We validated ROMPER in two studies that successfully recruited the targeted sample size, retained experts through all three rounds, and examined consensus on which policies are (not) effective and implementable. ROMPER protocols, materials, data, and code are openly available on the Open Science Framework with Creative Commons licensing for replication and reuse. ROMPER provides a validated, replicable, open access approach for eliciting expert views on both policy effectiveness and implementability-and for summarizing (lack of) consensus specifically for policymakers.

Using the Delphi Process to Prioritize an Agenda for Care Transition Research for Patients With Substance Use Disorders.

Medical hospitalizations are increasingly recognized as important opportunities to engage individuals with substance use disorders (SUD) and offer treatment. While a growing number of hospitals have instituted interventions to support the provision of SUD care during medical admissions, post-hospitalization transitions of care remain a challenge for patients and clinicians and an understudied area of SUD care. Evidence is lacking on the most effective and feasible models of care to improve post-hospitalization care transitions for people with SUD. In the absence of strong empirical evidence to guide practice and policy, consensus-based research methods such as the Delphi process can play an important role in efficiently prioritizing existing models of care for future study and implementation. We conducted a Delphi study that convened a group of 25 national interdisciplinary experts with direct clinical experience facilitating post-hospitalization care transitions for people with SUD. Our panelists rated 10 existing care transition models according to anticipated effectiveness and facility of implementation based on the GRADE Evidence to Decision framework. Qualitative data on each care model were also gathered through comments and an online moderated discussion board. Our results help establish a hierarchy of SUD care transition models to inform future study and program development.

A guide for social science journal editors on easing into open science.

Journal editors have a large amount of power to advance open science in their respective fields by incentivising and mandating open policies and practices at their journals. The Data PASS Journal Editors Discussion Interface (JEDI, an online community for social science journal editors: www.dpjedi.org ) has collated several resources on embedding open science in journal editing ( www.dpjedi.org/resources ). However, it can be overwhelming as an editor new to open science practices to know where to start. For this reason, we created a guide for journal editors on how to get started with open science. The guide outlines steps that editors can take to implement open policies and practices within their journal, and goes through the what, why, how, and worries of each policy and practice. This manuscript introduces and summarizes the guide (full guide: https://doi.org/10.31219/osf.io/hstcx ).

Expert Views on State Policies to Improve Engagement and Retention in Treatment for Opioid Use Disorder: A Qualitative Analysis of an Online Modified Delphi Process.

OBJECTIVES: The aim of this study was to examine expert views on the effectiveness and implementability of state policies to improve engagement and retention in treatment for opioid use disorder (OUD). METHODS: We conducted a 3-round modified Delphi process using the online ExpertLens platform. Participants included 66 experts on OUD treatment policies. Experts commented on 14 hypothetical state policies targeting treatment engagement and quality of care. Using the GRADE Evidence-to-Decision framework, we conducted reflexive thematic analysis to develop patterns of meaning from the dataset. RESULTS: Only policies for providing continued access to evidence-based treatment for highly at-risk populations, settings, and periods were seen as effective in meaningfully reducing population-level opioid-related overdose mortality. Experts commented that, although the general public increasingly supports policies expanding medications for OUD and evidence-based care, ongoing stigma about OUD encourages public acceptance of punitive and paternalistic policies. Experts viewed all policies as at least moderately feasible given the current infrastructure and resources, with affordability reliant on long-term cost savings from reduced opioid-related harms. Equitability depended on whether experts perceived a policy as inherently equitable in its design as well as concerns about the potential for inequitable implementation due to structural oppression and interpersonal biases in criminal-legal, healthcare, and other systems. CONCLUSIONS: Experts believe that supportive (rather than punitive) policies improve engagement and retention in OUD treatment. States could prioritize implementing supportive policies that are patient-centered and take a harm-reduction approach to enhance medications for OUD access and utilization. States could consider deimplementing punitive policies that are coercive, take an abstinence-only approach, and use punitive and restrictive measures.

Disciplinary trends in the use of the Delphi method: A bibliometric analysis.

The Delphi method is an iterative, anonymous, group-based process for eliciting and aggregating opinion on a topic to explore the existence of consensus among experts. The year 2023 marks the 60th anniversary of the first peer-reviewed journal article on the Delphi method. Originally developed for operations research, this method is now applied extensively by researchers representing diverse scientific fields. We used a bibliometric analysis to describe general trends in the expansion of its use across disciplines over time. We conducted a systematic literature search for all English-language, peer-reviewed journal articles on the Delphi method through its first 60 years. We found 19,831 articles: 96.8% (n = 19,204) on the actual use of the Delphi method in an empirical study and 3.2% (n = 627) describing, examining, or providing some guidance on how to use the Delphi method. Almost half (49.9%) of all articles were published in the 2010s and an additional third (32.5%) in the first few years of the 2020s. Nearly two-thirds (65%, n = 12,883) of all published articles have appeared in medical journals, compared to 15% in science and technology (n = 3,053) or social science (n = 3,016) journals. We conclude that the expanded use of the Delphi method has been driven largely by the medical field, though social scientists and technologists continue to be at the forefront of methodological work on the Delphi method. Therefore, we call for greater transdisciplinary collaboration on methodological guidance and standards for the Delphi method.

Open Science Standards at Journals that Inform Evidence-Based Policy.

Evidence-based policy uses intervention research to inform consequential decisions about resource allocation. Research findings are often published in peer-reviewed journals. Because detrimental research practices associated with closed science are common, journal articles report more false-positives and exaggerated effect sizes than would be desirable. Journal implementation of standards that promote open science-such as the transparency and openness promotion (TOP) guidelines-could reduce detrimental research practices and improve the trustworthiness of research evidence on intervention effectiveness. We evaluated TOP implementation at 339 peer-reviewed journals that have been used to identify evidence-based interventions for policymaking and programmatic decisions. Each of ten open science standards in TOP was not implemented in most journals' policies (instructions to authors), procedures (manuscript submission systems), or practices (published articles). Journals implementing at least one standard typically encouraged, but did not require, an open science practice. We discuss why and how journals could improve implementation of open science standards to safeguard evidence-based policy.

Exploring enablers and barriers to implementing the Transparency and Openness Promotion Guidelines: a theory-based survey of journal editors.

The Transparency and Openness Promotion (TOP) Guidelines provide a framework to help journals develop open science policies. Theories of behaviour change can guide understanding of why journals do (not) implement open science policies and the development of interventions to improve these policies. In this study, we used the Theoretical Domains Framework to survey 88 journal editors on their capability, opportunity and motivation to implement TOP. Likert-scale questions assessed editor support for TOP, and enablers and barriers to implementing TOP. A qualitative question asked editors to provide reflections on their ratings. Most participating editors supported adopting TOP at their journal (71%) and perceived other editors in their discipline to support adopting TOP (57%). Most editors (93%) agreed their roles include maintaining policies that reflect current best practices. However, most editors (74%) did not see implementing TOP as a high priority compared with other editorial responsibilities. Qualitative responses expressed structural barriers to implementing TOP (e.g. lack of time, resources and authority to implement changes) and varying support for TOP depending on study type, open science standard, and level of implementation. We discuss how these findings could inform the development of theoretically guided interventions to increase open science policies, procedures and practices.

Establishing open science research priorities in health psychology: a research prioritisation Delphi exercise.

OBJECTIVE: Research on Open Science practices in Health Psychology is lacking. This meta-research study aimed to identify research question priorities and obtain consensus on the Top 5 prioritised research questions for Open Science in Health Psychology. METHODS AND MEASURES: An international Delphi consensus study was conducted. Twenty-three experts in Open Science and Health Psychology within the European Health Psychology Society (EHPS) suggested research question priorities to create a 'long-list' of items (Phase 1). Forty-three EHPS members rated the importance of these items, ranked their top five and suggested their own additional items (Phase 2). Twenty-four EHPS members received feedback on Phase 2 responses and then re-rated and re-ranked their top five research questions (Phase 3). RESULTS: The top five ranked research question priorities were: 1. 'To what extent are Open Science behaviours currently practised in Health Psychology?', 2. 'How can we maximise the usefulness of Open Data and Open Code resources?', 3. 'How can Open Data be increased within Health Psychology?', 4. 'What interventions are effective for increasing the adoption of Open Science in Health Psychology?' and 5. 'How can we increase free Open Access publishing in Health Psychology?'. CONCLUSION: Funding and resources should prioritise the research questions identified here.

Expert Panel Consensus on State-Level Policies to Improve Engagement and Retention in Treatment for Opioid Use Disorder.

Importance: In the US, recent legislation and regulations have been considered, proposed, and implemented to improve the quality of treatment for opioid use disorder (OUD). However, insufficient empirical evidence exists to identify which policies are feasible to implement and successfully improve patient and population-level outcomes. Objective: To examine expert consensus on the effectiveness and the ability to implement state-level OUD treatment policies. Evidence Review: This qualitative study used the ExpertLens online platform to conduct a 3-round modified Delphi process to convene 66 stakeholders (health care clinicians, social service practitioners, addiction researchers, health policy decision-makers, policy advocates, and persons with lived experience). Stakeholders participated in 1 of 2 expert panels on 14 hypothetical state-level policies targeting treatment engagement and linkage, evidence-based and integrated care, treatment flexibility, and monitoring or support services. Participants rated policies in round 1, discussed results in round 2, and provided final ratings in round 3. Participants used 4 criteria associated with either the effectiveness or implementability to rate and discuss each policy. The effectiveness panel (n = 29) considered policy effects on treatment engagement, treatment retention, OUD remission, and opioid overdose mortality. The implementation panel (n = 34) considered the acceptability, feasibility, affordability, and equitability of each policy. We measured consensus using the interpercentile range adjusted for symmetry analysis technique from the RAND/UCLA appropriateness method. Findings: Both panels reached consensus on all items. Experts viewed 2 policies (facilitated access to medications for OUD and automatic Medicaid enrollment for citizens returning from correctional settings) as highly implementable and highly effective in improving patient and population-level outcomes. Participants rated hub-and-spoke-type policies and provision of financial incentives to emergency departments for treatment linkage as effective; however, they also rated these policies as facing implementation barriers associated with feasibility and affordability. Coercive policies and policies levying additional requirements on individuals with OUD receiving treatment (eg, drug toxicology testing, counseling requirements) were viewed as low-value policies (ie, decreasing treatment engagement and retention, increasing overdose mortality, and increasing health inequities). Conclusions and Relevance: The findings of this study may provide urgently needed consensus on policies for states to consider either adopting or deimplementing in their efforts to address the opioid overdose crisis.

Feasibility of an Audit and Feedback Intervention to Facilitate Journal Policy Change Towards Greater Promotion of Transparency and Openness in Sports Science Research.

OBJECTIVES: To evaluate (1) the feasibility of an audit-feedback intervention to facilitate sports science journal policy change, (2) the reliability of the Transparency of Research Underpinning Social Intervention Tiers (TRUST) policy evaluation form, and (3) the extent to which policies of sports science journals support transparent and open research practices. METHODS: We conducted a cross-sectional, audit-feedback, feasibility study of transparency and openness standards of the top 38 sports science journals by impact factor. The TRUST form was used to evaluate journal policies support for transparent and open research practices. Feedback was provided to journal editors in the format of a tailored letter. Inter-rater reliability and agreement of the TRUST form was assessed using intraclass correlation coefficients and the standard error of measurement, respectively. Time-based criteria, fidelity of intervention delivery and qualitative feedback were used to determine feasibility. RESULTS: The audit-feedback intervention was feasible based on the time taken to rate journals and provide tailored feedback. The mean (SD) score on the TRUST form (range 0-27) was 2.05 (1.99), reflecting low engagement with transparent and open practices. Inter-rater reliability of the overall score of the TRUST form was moderate [ICC (2,1) = 0.68 (95% CI 0.55-0.79)], with standard error of measurement of 1.17. However, some individual items had poor reliability. CONCLUSION: Policies of the top 38 sports science journals have potential for improved support for transparent and open research practices. The feasible audit-feedback intervention developed here warrants large-scale evaluation as a means to facilitate change in journal policies. REGISTRATION: OSF ( https://osf.io/d2t4s/ ).

Expert views on state-level naloxone access laws: a qualitative analysis of an online modified-Delphi process.

BACKGROUND: Expanding availability to naloxone is a core harm reduction strategy in efforts to address the opioid epidemic. In the US, state-level legislation is a prominent mechanism to expand naloxone availability through various venues, such as community pharmacies. This qualitative study aimed to identify and summarize the views of experts on state-level naloxone access laws. METHODS: We conducted a three-round modified-Delphi process using the online ExpertLens platform. Participants included 46 key stakeholders representing various groups (advocates, healthcare providers, human/social service practitioners, policymakers, and researchers) with expertise naloxone access laws. Participants commented on the effectiveness and implementability of 15 state-level naloxone access laws (NALs). We thematically analyzed participant comments to summarize views on NALs overall and specific types of NAL. RESULTS: Participants commented that the effectiveness of NALs in reducing opioid-related mortality depends on their ability to make sustained, significant impacts on population-level naloxone availability. Participants generally believed that increased naloxone availability does not have appreciable negative impacts on the prevalence of opioid misuse, opioid use disorder (OUD), and non-fatal opioid overdoses. Implementation barriers include stigma among the general public, affordability of naloxone, and reliance on an inequitable healthcare system. CONCLUSIONS: Experts believe NALs that significantly increase naloxone access are associated with less overdose mortality without risking substantial unintended public health outcomes. To maximize impacts, high-value NALs should explicitly counter existing healthcare system inequities, address stigmatization of opioid use and naloxone, maintain reasonable prices for purchasing naloxone, and target settings beyond community pharmacies to distribute naloxone.

Developing a medication adherence technologies repository: proposed structure and protocol for an online real-time Delphi study.

INTRODUCTION: An online interactive repository of available medication adherence technologies may facilitate their selection and adoption by different stakeholders. Developing a repository is among the main objectives of the European Network to Advance Best practices and technoLogy on medication adherencE (ENABLE) COST Action (CA19132). However, meeting the needs of diverse stakeholders requires careful consideration of the repository structure. METHODS AND ANALYSIS: A real-time online Delphi study by stakeholders from 39 countries with research, practice, policy, patient representation and technology development backgrounds will be conducted. Eleven ENABLE members from 9 European countries formed an interdisciplinary steering committee to develop the repository structure, prepare study protocol and perform it. Definitions of medication adherence technologies and their attributes were developed iteratively through literature review, discussions within the steering committee and ENABLE Action members, following ontology development recommendations. Three domains (product and provider information (D1), medication adherence descriptors (D2) and evaluation and implementation (D3)) branching in 13 attribute groups are proposed: product and provider information, target use scenarios, target health conditions, medication regimen, medication adherence management components, monitoring/measurement methods and targets, intervention modes of delivery, target behaviour determinants, behaviour change techniques, intervention providers, intervention settings, quality indicators and implementation indicators. Stakeholders will evaluate the proposed definition and attributes' relevance, clarity and completeness and have multiple opportunities to reconsider their evaluations based on aggregated feedback in real-time. Data collection will stop when the predetermined response rate will be achieved. We will quantify agreement and perform analyses of process indicators on the whole sample and per stakeholder group. ETHICS AND DISSEMINATION: Ethical approval for the COST ENABLE activities was granted by the Malaga Regional Research Ethics Committee. The Delphi protocol was considered compliant regarding data protection and security by the Data Protection Officer from University of Basel. Findings from the Delphi study will form the basis for the ENABLE repository structure and related activities.

Transparent, Open, and Reproducible Prevention Science.

The field of prevention science aims to understand societal problems, identify effective interventions, and translate scientific evidence into policy and practice. There is growing interest among prevention scientists in the potential for transparency, openness, and reproducibility to facilitate this mission by providing opportunities to align scientific practice with scientific ideals, accelerate scientific discovery, and broaden access to scientific knowledge. The overarching goal of this manuscript is to serve as a primer introducing and providing an overview of open science for prevention researchers. In this paper, we discuss factors motivating interest in transparency and reproducibility, research practices associated with open science, and stakeholders engaged in and impacted by open science reform efforts. In addition, we discuss how and why different types of prevention research could incorporate open science practices, as well as ways that prevention science tools and methods could be leveraged to advance the wider open science movement. To promote further discussion, we conclude with potential reservations and challenges for the field of prevention science to address as it transitions to greater transparency, openness, and reproducibility. Throughout, we identify activities that aim to strengthen the reliability and efficiency of prevention science, facilitate access to its products and outputs, and promote collaborative and inclusive participation in research activities. By embracing principles of transparency, openness, and reproducibility, prevention science can better achieve its mission to advance evidence-based solutions to promote individual and collective well-being.

Modern Meta-Analytic Methods in Prevention Science: Introduction to the Special Issue.

Meta-analyses that statistically synthesize evidence from multiple research studies can play an important role in advancing evidence-informed prevention science. When done in the context of a well-conducted systematic review, meta-analysis is a powerful tool for synthesizing evidence and exploring research questions that are difficult to address in individual studies, such as the association of individual study limitations on intervention effect estimates, replicability of empirical findings, and variation of effect estimates across populations and settings. Alongside the rapid growth in the number of published reviews and meta-analyses, there has been a parallel growth in the development of meta-analytic techniques to handle the increasingly complex types of questions and types of evidence relevant to prevention science. Despite this rapid evolution of meta-analytic techniques and approaches, there is still a lag between the development of new techniques and their uptake by researchers in the field. This paper serves as a brief introduction to this special issue of Prevention Science, entitled "Modern Meta-Analytic Methods in Prevention Science," which highlights recent developments in meta-analytic methods and demonstrates their application to prevention research. This special issue makes an important contribution to the field by ensuring these methodological advances are widely accessible to prevention science researchers, thereby improving their uptake and utilization, and ultimately improving the utility and rigor of research syntheses for informing evidence-based decision making in prevention.

Clearinghouse Standards of Evidence on the Transparency, Openness, and Reproducibility of Intervention Evaluations.

Clearinghouses are influential repositories of information on the effectiveness of social interventions. To identify which interventions are "evidence-based," clearinghouses review intervention evaluations using published standards of evidence that focus primarily on internal validity and causal inferences. Open science practices can improve trust in evidence from evaluations on the effectiveness of social interventions. Including open science practices in clearinghouse standards of evidence is one of many efforts that could increase confidence in designations of interventions as "evidence-based." In this study, we examined the policies, procedures, and practices of 10 federal evidence clearinghouses that review preventive interventions-an important and influential subset of all evidence clearinghouses. We found that seven consider at least one open science practice when evaluating interventions: replication (6 of 10 clearinghouses), public availability of results (6), investigator conflicts of interest (3), design and analysis transparency (3), study registration (2), and protocol sharing (1). We did not identify any policies, procedures, or practices related to analysis plan registration, data sharing, code sharing, material sharing, and citation standards. We provide a framework with specific recommendations to help federal and other evidence clearinghouses implement the Transparency and Openness Promotion (TOP) Guidelines. Our proposed "TOP Guidelines for Clearinghouses" includes reporting whether evaluations used open science practices, incorporating open science practices in their standards for receiving "evidence-based" designations, and verifying that evaluations used open science practices. Doing so could increase the trustworthiness of evidence used for policy making and support improvements throughout the evidence ecosystem.

PROTOCOL: Guidance for stakeholder engagement in guideline development: A systematic review.

This is the protocol for a Campbell systematic review. The objectives are as follows: to identify, describe, and summarize existing guidance and methods for multistakeholder engagement throughout the health guideline development process.

PROTOCOL: Barriers and facilitators to stakeholder engagement in health guideline development: A qualitative evidence synthesis.

Background: There is a need for the development of comprehensive, global, evidence-based guidance for stakeholder engagement in guideline development. Stakeholders are any individual or group who is responsible for or affected by health- and healthcare-related decisions. This includes patients, the public, providers of health care and policymakers for example. As part of the guidance development process, Multi-Stakeholder Engagement (MuSE) Consortium set out to conduct four concurrent systematic reviews to summarise the evidence on: (1) existing guidance for stakeholder engagement in guideline development, (2) barriers and facilitators to stakeholder engagement in guideline development, (3) managing conflicts of interest in stakeholder engagement in guideline development and (4) measuring the impact of stakeholder engagement in guideline development. This protocol addresses the second systematic review in the series. Objectives: The objective of this review is to identify and synthesise the existing evidence on barriers and facilitators to stakeholder engagement in health guideline development. We will address this objective through two research questions: (1) What are the barriers to multi-stakeholder engagement in health guideline development across any of the 18 steps of the GIN-McMaster checklist? (2) What are the facilitators to multi-stakeholder engagement in health guideline development across any of the 18 steps of the GIN-McMaster checklist? Search Methods: A comprehensive search strategy will be developed and peer-reviewed in consultation with a medical librarian. We will search the following databases: MEDLINE, Cumulative Index to Nursing & Allied Health Literature (CINAHL), EMBASE, PsycInfo, Scopus, and Sociological Abstracts. To identify grey literature, we will search the websites of agencies who actively engage stakeholder groups such as the AHRQ, Canadian Institutes of Health Research (CIHR) Strategy for Patient-Oriented Research (SPOR), INVOLVE, the National Institute for Health and Care Excellence (NICE) and the PCORI. We will also search the websites of guideline-producing agencies, such as the American Academy of Pediatrics, Australia's National Health Medical Research Council (NHMRC) and the WHO. We will invite members of the team to suggest grey literature sources and we plan to broaden the search by soliciting suggestions via social media, such as Twitter. Selection Criteria: We will include empirical qualitative and mixed-method primary research studies which qualitatively report on the barriers or facilitators to stakeholder engagement in health guideline development. The population of interest is stakeholders in health guideline development. Building on previous work, we have identified 13 types of stakeholders whose input can enhance the relevance and uptake of guidelines: Patients, caregivers and patient advocates; Public; Providers of health care; Payers of health services; Payers of research; Policy makers; Program managers; Product makers; Purchasers; Principal investigators and their research teams; and Peer-review editors/publishers. Eligible studies must describe stakeholder engagement at any of the following steps of the GIN-McMaster Checklist for Guideline Development. Data Collection and Analysis: All identified citations from electronic databases will be imported into Covidence software for screening and selection. Documents identified through our grey literature search will be managed and screened using an Excel spreadsheet. A two-part study selection process will be used for all identified citations: (1) a title and abstract review and (2) full-text review. At each stage, teams of two review authors will independently assess all potential studies in duplicate using a priori inclusion and exclusion criteria. Data will be extracted by two review authors independently and in duplicate according to a standardised data extraction form. Main Results: The results of this review will be used to inform the development of guidance for multi-stakeholder engagement in guideline development and implementation. This guidance will be official GRADE (Grading of Recommendations Assessment, Development and Evaluation) Working Group guidance. The GRADE system is internationally recognised as a standard for guideline development. The findings of this review will assist organisations who develop healthcare, public health and health policy guidelines, such as the World Health Organization, to involve multiple stakeholders in the guideline development process to ensure the development of relevant, high quality and transparent guidelines.

Strengthening women's empowerment and gender equality in fragile contexts towards peaceful and inclusive societies: A systematic review and meta-analysis.

Background: Across the globe, gender disparities still exist with regard to equitable access to resources, participation in decision-making processes, and gender and sexual-based violence. This is particularly true in fragile and conflict-affected settings, where women and girls are affected by both fragility and conflict in unique ways. While women have been acknowledged as key actors in peace processes and post-conflict reconstruction (e.g., through the United Nations Security Council Resolution 1325 and the Women, Peace and Security Agenda) evidence on the effectiveness of gender-specific and gender-transformative interventions to improve women's empowerment in fragile and conflict-affected states and situations (FCAS) remains understudied. Objectives: The purpose of this review was to synthesize the body of evidence around gender-specific and gender-transformative interventions aimed at improving women's empowerment in fragile and conflict-affected settings with high levels of gender inequality. We also aimed to identify barriers and facilitators that could affect the effectiveness of these interventions and to provide implications for policy, practice and research designs within the field of transitional aid. Methods: We searched for and screened over 100,000 experimental and quasi-experimental studies focused on FCAS at the individual and community levels. We used standard methodological procedures outlined by the Campbell Collaboration for the data collection and analysis, including quantitative and qualitative analyses, and completed the Grading of Recommendations, Assessment, Development and Evaluations (GRADE) methodology to assess the certainty around each body of evidence. Results: We identified 104 impact evaluations (75% randomised controlled trials) assessing the effects of 14 different types of interventions in FCAS. About 28% of included studies were assessed as having a high risk of bias (45% among quasi-experimental designs). Interventions supporting women's empowerment and gender equality in FCAS produced positive effects on the outcomes related to the primary focus of the intervention. There are no significant negative effects of any included interventions. However, we observe smaller effects on behavioural outcomes further along the causal chain of empowerment. Qualitative syntheses indicated that gender norms and practices are potential barriers to intervention effectiveness, while working with local powers and institutions can facilitate the uptake and legitimacy of interventions. Conclusions: We observe gaps of rigorous evidence in certain regions (notably MENA and Latin America) and in interventions specifically targeting women as actors of peacebuilding. Gender norms and practices are important elements to consider in programme design and implementation to maximise potential benefits: focusing on empowerment only might not be enough in the absence of targeting the restrictive gender norms and practices that may undermine intervention effectiveness. Lastly, programme designers and implementation should consider explicitly targeting specific empowerment outcomes, promoting social capital and exchange, and tailoring the intervention components to the desired empowerment-related outcomes.

Reporting all results efficiently: A RARE proposal to open up the file drawer.

While the social sciences have made impressive progress in adopting transparent research practices that facilitate verification, replication, and reuse of materials, the problem of publication bias persists. Bias on the part of peer reviewers and journal editors, as well as the use of outdated research practices by authors, continues to skew literature toward statistically significant effects, many of which may be false positives. To mitigate this bias, we propose a framework to enable authors to report all results efficiently (RARE), with an initial focus on experimental and other prospective empirical social science research that utilizes public study registries. This framework depicts an integrated system that leverages the capacities of existing infrastructure in the form of public registries, institutional review boards, journals, and granting agencies, as well as investigators themselves, to efficiently incentivize full reporting and thereby, improve confidence in social science findings. In addition to increasing access to the results of scientific endeavors, a well-coordinated research ecosystem can prevent scholars from wasting time investigating the same questions in ways that have not worked in the past and reduce wasted funds on the part of granting agencies.