A protocol for ongoing systematic scoping reviews of World Trade Center Health research.

BACKGROUND: The World Trade Center (WTC) Health Program ("Program") seeks to assess the inventory, quality, and impact of its funded research in the context of all clinical and translational research involving WTC populations. This paper presents a protocol for ongoing scoping reviews of WTC-related health research. METHODS: Using terms relevant to the September 11 attacks, we will search OVID MEDLINE, PsycINFO, Scopus, Web of Science, CINAHL, and Embase for records of peer-reviewed publications. Title, abstract, and full text screening will be used to exclude records according to a priori criteria. Data abstraction will be performed on all articles that meet inclusion criteria using a standardized query form that was developed in collaboration with NIOSH. A team of reviewers will be trained to abstract data from included articles. Articles will be double-reviewed, and disagreements will be adjudicated. RESULTS: We will summarize existing research involving WTC populations. The summary will assess the extent, nature, and signals of impact of WTC-related health research. CONCLUSIONS: Our review will lay the groundwork for additional study of research impact by identifying population, clinical, and translational topics that can be assessed through future focused reviews. It will also support planning activities by Program policy makers and stakeholders as they work to achieve the Program's research goals. SYSTEMATIC REVIEW REGISTRATION: This publication serves as documentation of the protocol.

Protocol for the development of guidance for collaborator and partner engagement in health care evidence syntheses.

BACKGROUND: Involving collaborators and partners in research may increase relevance and uptake, while reducing health and social inequities. Collaborators and partners include people and groups interested in health research: health care providers, patients and caregivers, payers of health research, payers of health services, publishers, policymakers, researchers, product makers, program managers, and the public. Evidence syntheses inform decisions about health care services, treatments, and practice, which ultimately affect health outcomes. Our objectives are to: A. Identify, map, and synthesize qualitative and quantitative findings related to engagement in evidence syntheses B. Explore how engagement in evidence synthesis promotes health equity C. Develop equity-oriented guidance on methods for conducting, evaluating, and reporting engagement in evidence syntheses METHODS: Our diverse, international team will develop guidance for engagement with collaborators and partners throughout multiple sequential steps using an integrated knowledge translation approach: 1. Reviews. We will co-produce 1 scoping review, 3 systematic reviews and 1 evidence map focusing on (a) methods, (b) barriers and facilitators, (c) conflict of interest considerations, (d) impacts, and (e) equity considerations of engagement in evidence synthesis. 2. Methods study, interviews, and survey. We will contextualise the findings of step 1 by assessing a sample of evidence syntheses reporting on engagement with collaborators and partners and through conducting interviews with collaborators and partners who have been involved in producing evidence syntheses. We will use these findings to develop draft guidance checklists and will assess agreement with each item through an international survey. 3. CONSENSUS: The guidance checklists will be co-produced and finalised at a consensus meeting with collaborators and partners. 4. DISSEMINATION: We will develop a dissemination plan with our collaborators and partners and work collaboratively to improve adoption of our guidance by key organizations. CONCLUSION: Our international team will develop guidance for collaborator and partner engagement in health care evidence syntheses. Incorporating partnership values and expectations may result in better uptake, potentially reducing health inequities.

Policy Decisionmaking in Long-Term Care: Lessons from Infection Control During the COVID-19 Pandemic.

The COVID-19 pandemic focused attention on long-term care facilities' need for infection-control policies that balanced community safety and individual well-being. Infection-control policies were often developed, implemented, and mandated without the input or involvement of those who are most affected: residents and their family members, administrators, and staff. This failure led to declines in residents' physical and mental health. The pandemic exposed an opportunity-and an imperative-to reimagine long-term care in a way that is centered on the needs and preferences of those who receive care, their family members, and those who provide care. This study lays the groundwork for cultural change and a move toward inclusive policy decisionmaking in long-term care through a review of infection-control policy decisions and action items proposed in guided discussions with a diversity of stakeholders-long-term care residents, direct care staff, and consumer advocates to facility administrators, clinicians, researchers, and industry organizations. Transforming the culture of long-term care to elevate the needs of residents will require attention to facility leadership, along with steps to increase inclusiveness, transparency, and accountability in decisionmaking.

Key issues for stakeholder engagement in the development of health and healthcare guidelines.

Established in 2015, the Multi-Stakeholder Engagement (MuSE) Consortium is an international network of over 120 individuals interested in stakeholder engagement in research and guidelines. The MuSE group is developing guidance for stakeholder engagement in the development of health and healthcare guideline development. The development of this guidance has included multiple meetings with stakeholders, including patients, payers/purchasers of health services, peer review editors, policymakers, program managers, providers, principal investigators, product makers, the public, and purchasers of health services and has identified a number of key issues. These include: (1) Definitions, roles, and settings (2) Stakeholder identification and selection (3) Levels of engagement, (4) Evaluation of engagement, (5) Documentation and transparency, and (6) Conflict of interest management. In this paper, we discuss these issues and our plan to develop guidance to facilitate stakeholder engagement in all stages of the development of health and healthcare guideline development.

A group of international researchers, patient partners, and other stakeholders are working together to create a checklist for when and how to involve stakeholders in health guideline development. Health guidelines include clinical practice guidelines, which your healthcare provider uses to determine treatments for health conditions. While working on this checklist, the team identified key issues to work on, including: (1) Definitions, roles, and settings (2) Stakeholder identification and selection (3) Levels of engagement, (4) Evaluation of engagement, (5) Documentation and transparency, and (6) Conflict of interest management. This paper describes each issue and how the team plans to produce guidance papers to address them.

Treatment goals for rheumatoid arthritis: patient engagement and goal collection.

Aim: We developed the Patient-Engaged Health Technology Assessment strategy for survey-based goal collection from patients to yield patient-important outcomes suitable for use in multi-criteria decision analysis. Methods: Rheumatoid arthritis patients were recruited from online patient networks for proof-of-concept testing of goal collection and prioritization using a survey. A Project Steering Committee and Expert Panel rated the feasibility of scaling to larger samples. Results: Survey respondents (n = 47) completed the goal collection exercise. Finding effective treatments was rated by respondents as the most important goal, and reducing stiffness was rated as the least important. Feedback from our steering committee and expert panel support the approach's feasibility for goal identification and ranking. Conclusion: Goals relevant for treatment evaluation can be identified and rated for importance by patients to permit wide input from patients with lived experience of disease.

Translational Impacts of World Trade Center Health Program Research: A Mixed Methods Study.

The World Trade Center (WTC) Health Program (hereafter Program) provides medical monitoring and treatment for its Members: the responders and survivors who were at the WTC and related sites or located within the New York City disaster area during and after the terrorist attacks of September 11, 2001. Scientific research supported by the Program provides an evidence base for the health monitoring and clinical care of Members. In this article, the authors describe the findings and recommendations of a four-year study of the Program's research portfolio and its translational impact. They present results from a mixed methods assessment that integrates (1) a scoping review of nearly 1,000 peer-reviewed publications and thousands of pages of grey literature that reference research related to the health effects of 9/11, (2) stakeholder perspectives gathered through focus groups and interviews, and (3) a review of Program documentation. The study is intended to guide planning by Program leadership as it aims to maximize the impacts of Program research and achieve its goal of translating research into care for those affected by the attacks on 9/11.

Factors to Consider During Identification and Invitation of Individuals in a Multi-stakeholder Research Partnership.

BACKGROUND: Health research teams increasingly partner with stakeholders to produce research that is relevant, accessible, and widely used. Previous work has covered stakeholder group identification. OBJECTIVE: We aimed to develop factors for health research teams to consider during identification and invitation of individual representatives in a multi-stakeholder research partnership, with the aim of forming equitable and informed teams. DESIGN: Consensus development. PARTICIPANTS: We involved 16 stakeholders from the international Multi-Stakeholder Engagement (MuSE) Consortium, including patients and the public, providers, payers of health services/purchasers, policy makers, programme managers, peer review editors, and principal investigators. APPROACH: We engaged stakeholders in factor development and as co-authors of this manuscript. Using a modified Delphi approach, we gathered stakeholder views concerning a preliminary list of 18 factors. Over two feedback rounds, using qualitative and quantitative analysis, we concentrated these into ten factors. KEY RESULTS: We present seven highly desirable factors: 'expertise or experience', 'ability and willingness to represent the stakeholder group', 'inclusivity (equity, diversity and intersectionality)', 'communication skills', 'commitment and time capacity', 'financial and non-financial relationships and activities, and conflict of interest', 'training support and funding needs'. Additionally, three factors are desirable: 'influence', 'research relevant values', 'previous stakeholder engagement'. CONCLUSIONS: We present factors for research teams to consider during identification and invitation of individual representatives in a multi-stakeholder research partnership. Policy makers and guideline developers may benefit from considering the factors in stakeholder identification and invitation. Research funders may consider stipulating consideration of the factors in funding applications. We outline how these factors can be implemented and exemplify how their use has the potential to improve the quality and relevancy of health research.

Use of Real-World Evidence for Regulatory Approval and Coverage of Medical Devices: A Landscape Assessment.

OBJECTIVES: To enhance the generalizability of the evidence it reviews, the US Food and Drug Administration (FDA) has encouraged manufacturers to expand the submission of real-world evidence (RWE). The extent to which this evidence, which is generated outside of research settings, can support decision making remains unclear. We described the current use of RWE for medical devices, assessed manufacturers' challenges in generating and using it for regulatory and coverage decisions, and identified opportunities to expand its use. METHODS: We conducted 27 solo and group interviews with FDA officials and representatives of device manufacturers, payers, and health technology assessment organizations. All interviews used a semistructured protocol and were transcribed to allow thematic analysis. RESULTS: Accessing and linking real-world data sources, identifying unique devices, capturing longitudinal data, limited staff expertise, and uncertain return on investment have hampered efforts to use real-world data. Many companies in our sample were conducting research using real-world data, but none had submitted RWE as the primary evidence supporting a premarket approval. FDA guidance was helpful, but regulatory requirements remained ambiguous and examples of successful regulatory decisions based on RWE were limited. Payers mainly used RWE to supplement experimental evidence in coverage decisions, evaluated both types of evidence in similar ways, and had concerns about the rigor of RWE. CONCLUSIONS: Technical challenges may slow efforts to generate and use RWE in the near term. Additional regulatory guidance and examples, greater use of rigorous study designs and analytic methods, and continued stakeholder engagement could accelerate the use of RWE.

"I'm not gonna be able to do anything about it, then what's the point?": A broad group of stakeholders identify barriers and facilitators to HCV testing in a Massachusetts jail.

BACKGROUND: Despite national guidelines promoting hepatitis C virus (HCV) testing in prisons, there is substantial heterogeneity on the implementation of HCV testing in jails. We sought to better understand barriers and opportunities for HCV testing by interviewing a broad group of stakeholders involved in HCV testing and treatment policies and procedures in Massachusetts jails. METHODS: We conducted semi-structured interviews with people incarcerated in Middlesex County Jail (North Billerica, MA), clinicians working in jail and community settings, corrections administrators, and representatives from public health, government, and industry between November 2018-April 2019. RESULTS: 51/120 (42%) of people agreed to be interviewed including 21 incarcerated men (mean age 32 [IQR 25, 39], 60% non-White). Themes that emerged from these interviews included gaps in knowledge about HCV testing and treatment opportunities in jail, the impact of captivity and transience, and interest in improving linkage to HCV care after release. Many stakeholders discussed stigma around HCV infection as a factor in reluctance to provide HCV testing or treatment in the jail setting. Some stakeholders expressed that stigma often led decisionmakers to estimate a lower "worth" of incarcerated individuals living with HCV and therefore to decide against paying for HCV testing.". CONCLUSION: All stakeholders agreed that HCV in the jail setting is a public health issue that needs to be addressed. Exploring stakeholders' many ideas about how HCV testing and treatment can be approached is the first step in developing feasible and acceptable strategies.

Impact of World Trade Center-Related Health Research: An Application of the NIEHS Translational Framework.

The World Trade Center Health Program (WTCHP) has a research mission to identify physical and mental health conditions that may be related to the 9/11 terrorist attacks as well as effective diagnostic procedures and treatments for WTC-related health conditions. The ability of the WTCHP to serve its members and realize positive impacts on all of its stakeholders depends on effective translation of research findings. As part of an ongoing assessment of the translational impact of World Trade Center (WTC)-related research, we applied the National Institute of Environmental Health Sciences (NIEHS) translational framework to two case studies: WTC-related research on post-traumatic stress disorder (PTSD) and cancer. We conducted a review of 9/11 health-related research in the peer-reviewed literature through October 2017, grey literature, and WTCHP program documentation. We mapped peer-reviewed studies in the literature to the NIEHS framework and used WTCHP program documentation and grey literature to find evidence of translation of research into clinical practice and policy. Using the NIEHS framework, we identified numerous translational milestones and bridges, as well as areas of opportunity, within each case study. This application demonstrates the utility of the NIEHS framework for documenting progress toward public health impact and for setting future research goals.

GRADE Concept Paper 1: Validating the "F.A.C.E" instrument using stakeholder perceptions of feasibility, acceptability, cost, and equity in guideline implement.

BACKGROUND AND OBJECTIVE: To present a structured approach for assessing stakeholder perceptions and implementing the approach in guideline development. METHODS: This work was carried out by the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) Equity and Stakeholder Engagement Project Groups through brainstorming and iterative frameworks, stakeholder engagement, pilot testing, refinement of ideas, using input from workshops, and discussions at GRADE Working Group meetings to produce this document, which constitutes a GRADE conceptual article on implementation. RESULTS: We introduce the FACE implementation criteria, feasibility, acceptability, cost, and equity; priority; and "intent to implement" criterion. We outline the implementation importance of networks and approaches to patient and other stakeholder engagement. Implementation is often highly contextual and can benefit from stakeholder engagement and other assessments. Our FACE approach provides stakeholder questions and language to inform guideline implementation and tools. CONCLUSION: The FACE criteria propose a series of knowledge translation questions to guide the assessment of implementation for evidence-based guidelines. It is desirable for guideline developers to use a conceptual approach, such as FACE, to tailor implementation and inform end of guideline dissemination and knowledge translation activities.

Identifying patient decisions and related information needs during decision making related to total knee arthroplasty.

Aim: Research regarding decisions patients make about total knee arthroplasty, apart from having the procedure or not, are limited. Understanding patient decision making and related information needs is essential for shared decision making. Methods: Focus groups with an online community-based sample identified decisions about total knee arthroplasty beyond the decision to have the surgery itself. An online survey was used to determine relative importance of five major decisions and evaluate related information available. Results: Patients did not feel they have enough information to make important decisions of surgeon, device type, surgical approach, facility, or timing, for their total knee arthroplasty. Conclusion: Although further research is needed to generalize these findings, physicians should consider these questions during shared decision making with patients considering total knee arthroplasty.

Why clinical trials may not help patients make treatment decisions: results from focus group discussions with 22 patients.

Aim: Despite broad interest in advancing personalized medicine, most evidence is currently derived from average results of clinical trials that may obscure heterogeneity of trial participants. Little is known currently about how patients view heterogeneity in trials and whether they can participate in methodological discussions about this concept. Materials & methods: In structured discussions with three focus groups involving 22 participants, we assessed how representatives of patient communities have used research to guide individual treatment decisions. Discussion themes were organized into a framework describing patient decision-making in four steps: decisions patients make in the course of care; information used to make decisions; sources for information; and quality of information. Results/conclusion: Patients prioritize information that reflects their own characteristics, preferences and values. They struggle applying clinical research to their own case.

Facilitating stakeholder engagement in early stage translational research.

INTRODUCTION: Stakeholder engagement can play an important role in increasing public trust and the understanding of scientific research and its impact. Frameworks for stakeholder identification exist, but these frameworks may not apply well to basic science and early stage translational research. METHODS: Four Clinical and Translational Science Award (CTSA) hubs led six focus groups and two semi-structured interviews using a semi-structured discussion guide to learn from basic science researchers about stakeholder engagement in their work. The 24 participants represented fourteen clinical and academic disciplines. RESULTS: Early stage translational researchers reported engagement with a broad array of stakeholders. Those whose research has a clinical focus reported working with a more diverse range of stakeholders than those whose work did not. Common barriers to stakeholder engagement were grouped into three major themes: a poor definition of concepts, absence of guidance, and limited resources. DISCUSSION: The National Center for Advancing Translational Sciences (NCATS), the consortium of CTSAs, and the individual CTSA "hubs" are three actors that can help early stage translational researchers develop shared terms of reference, build the necessary skills, and assemble the appropriate resources for engaging stakeholders in Clinical and Translational Research. Getting this right will involve a coordinated push by all three entities.

Protocol for the development of guidance for stakeholder engagement in health and healthcare guideline development and implementation.

BACKGROUND: Stakeholder engagement has become widely accepted as a necessary component of guideline development and implementation. While frameworks for developing guidelines express the need for those potentially affected by guideline recommendations to be involved in their development, there is a lack of consensus on how this should be done in practice. Further, there is a lack of guidance on how to equitably and meaningfully engage multiple stakeholders. We aim to develop guidance for the meaningful and equitable engagement of multiple stakeholders in guideline development and implementation. METHODS: This will be a multi-stage project. The first stage is to conduct a series of four systematic reviews. These will (1) describe existing guidance and methods for stakeholder engagement in guideline development and implementation, (2) characterize barriers and facilitators to stakeholder engagement in guideline development and implementation, (3) explore the impact of stakeholder engagement on guideline development and implementation, and (4) identify issues related to conflicts of interest when engaging multiple stakeholders in guideline development and implementation. DISCUSSION: We will collaborate with our multiple and diverse stakeholders to develop guidance for multi-stakeholder engagement in guideline development and implementation. We will use the results of the systematic reviews to develop a candidate list of draft guidance recommendations and will seek broad feedback on the draft guidance via an online survey of guideline developers and external stakeholders. An invited group of representatives from all stakeholder groups will discuss the results of the survey at a consensus meeting which will inform the development of the final guidance papers. Our overall goal is to improve the development of guidelines through meaningful and equitable multi-stakeholder engagement, and subsequently to improve health outcomes and reduce inequities in health.

Engaging Patients and Other Non-Researchers in Health Research: Defining Research Engagement.

With the increase in patient and consumer activism through the late twentieth century and into this century, patient roles in research evolved into a new model of research engagement, with patients serving as active advisors and co-leading or leading clinical research. By requiring active engagement of patients and other stakeholders, several government research funders have advanced this model, particularly in Canada, the United States (US), United Kingdom (UK), and Australia. A consortium of individuals from these countries formed a Multi-Stakeholder Engagement (MuSE) consortium to examine critical issues in engaged research, establish consensus on definitions, and provide guidance for the field, beginning with an overview of how to involve stakeholders in health research (Concannon et al. J Gen Intern Med. 2019;34(3):458-463) and continuing here with an examination of definitions of research engagement. The political and advocacy roots of engaged research are reflected in definitions. Engagement is conceptualized with reference to research project goals, from informing specific clinical decisions to informing health-system level decisions. Political and cultural differences across countries are evident. Some of these government funders focus on empirical rather than ethical rationales. In countries with centralized health technology assessment, the link between societal values and engaged research is explicit. Ethical rationales for engagement are explicit in most of the published literature on research engagement. Harmonization of definitions is recommended so that research engagement elements, methods, and outcomes and impacts can be clearly examined and understood, and so that the field of research engagement can proceed from a clear conceptual foundation. Specific recommendations for terminology definitions are provided. Placing engaged research on a continuum from specific clinical decisions to more global public and social justice concerns clarifies the type of engaged research, supports appropriate comparisons, and improves the rigor of engaged research methods. The results help identify knowledge gaps in this growing field.

Stakeholder Perspectives on the Use of Community Health Workers To Improve Palliative Care Use by African Americans with Cancer.

BACKGROUND: African Americans in the United States have worse end-of-life care and cancer outcomes than whites. Palliative care may improve this disparity. Community Health Workers may provide a means to improve palliative care disparities. METHODS: Semistructured in-depth interviews (five) and stakeholder focus groups (four) were conducted with cancer patients, caregivers, health care administrators, oncologists, and community health workers (CHWs). Patients were recruited through snowball sampling. Three raters coded interviews independently. Data were analyzed using interpretative phenomenological analysis. RESULTS: Seventy-one individuals were contacted to participate with 24 stakeholders (34%) participating in individual interviews or across 4 stakeholder engagements. Eleven constructs were identified and grouped in three broader themes: "hub of the wheel," understanding palliative care, and patient-provider relationships. Participants felt that the role of a CHW should be central, bridging patients with their providers, information, and resources, including psychosocial support and advance care planning documents. They also placed an emphasis on the background of CHWs, saying individuals selected should be familiar with the history, culture, and norms of the communities from which they operate. Stakeholders reported that a CHW could activate a patient to contact their primary care physician or oncologist who may refer to or provide palliative care. Stakeholders reported that given the barriers to palliative care, a CHW could contribute to patient-centered multidisciplinary care while addressing palliative care domains with patients and families in a culturally sensitive manner. CONCLUSION: Based on feedback from patients, caregivers, and providers, a culturally adapted CHW intervention may improve palliative care use for African American patients with advanced malignancies.

Practical Guidance for Involving Stakeholders in Health Research.

Stakeholder engagement is increasingly common in health research, with protocols for engaging multiple stakeholder groups becoming normative in patient-centered outcomes research. Previous work has focused on identifying relevant stakeholder groups with whom to work and on working with stakeholders in evidence implementation. This paper draws on the expertise of a team from four countries-Canada, Australia, the UK, and the USA-to provide researchers with practical guidance for carrying out multi-stakeholder-engaged projects: we present a list of questions to assist in selecting appropriate roles and modes of engagement; we introduce a matrix to help summarize engagement activities; and we provide a list of online resources. This guidance, matrix, and list of resources can assist researchers to consider more systematically which stakeholder groups to involve, in what study roles, and by what modes of engagement. By documenting how stakeholders are paired up with specific roles, the matrix also provides a potential structure for evaluating the impact of stakeholder engagement.