Return of aggregate results to study participants: Facilitators, barriers, and recommendations.

Background: Most researchers and study participants believe that the summary, or aggregate, results of health research should be returned to study participants. However, researchers often do not return aggregate results. A better understanding of the impediments to results return could support improvements in this practice. Methods: This qualitative study convened eight virtual focus groups, four with investigators and four with patient partners from research studies funded by the Patient-Centered Outcomes Research Institute (PCORI). In total, 23 investigators and 20 partners participated. We explored perspectives, experiences, influences, and recommendations related to aggregate results return. Results: Focus group participants described the ethical importance of returning aggregate results, as well as the benefits to study participants. They also noted important impediments to results return, emphasizing IRB and logistical challenges and describing a lack of support for the practice both on the part of institutions and the field at large. Participants highlighted the value of patients and caregivers' perspectives and contributions to results return, which focused on returning the most relevant findings through effective channels and formats. They further emphasized the importance of planning and identified resources that could support results return. Conclusion: Researchers, funders, and the field can better facilitate results return by promoting standardized processes in research, such as the earmarking of funds for results return and inclusion of results returns milestones in research plans. More intentional policies, infrastructures, and resources that support results return may lead to more widespread return of study results to those who make these studies possible.

Examining how study teams manage different viewpoints and priorities in patient-centered outcomes research: Results of an embedded multiple case study.

INTRODUCTION: Limited evidence exists about which patient and stakeholder engagement practices support or hinder study teams as they negotiate different viewpoints in decisions about the design and conduct of patient-centered outcomes research. METHODS: We applied a multiple-embedded descriptive case study design for six studies funded by the Patient-Centered Outcomes Research Institute (PCORI). We interviewed 32 researchers and stakeholder partners, including patients, caregivers and clinicians, and reviewed documents related to each study (e.g., publications, and progress reports submitted to PCORI). FINDINGS: Overall, researchers reported that incorporating different viewpoints was a strength or opportunity to learn rather than something to be avoided or dreaded. Across cases, different viewpoints and priorities, often related to ethical or pragmatic considerations, emerged between researchers and stakeholders, between stakeholder groups (e.g., patients and clinicians) or within groups (e.g., amongst researchers). Examples of navigating different viewpoints arose across study phases. The length of time to resolve issues depended on how strongly people disagreed and the perceived importance or impact of decisions on the study. All cases used collaborative decision-making approaches, often described as consensus, throughout the study. Interviewees described consensus as using negotiation, compromise or working towards an agreeable decision. To encourage consensus, cases actively facilitated group discussions with an openness to diverse opinions, remained flexible and open to trying new things, referenced a ground rule or common goal and delegated decisions to partners or smaller workgroups. When viewpoints were not easily resolved, cases used different approaches to reach final decisions while maintaining relationships with partners, such as elevating decisions to leadership or agreeing to test out an approach. No one engagement structure (e.g., advisory group, coinvestigator) stood out as better able to manage different viewpoints. Teams adjusted engagement structures and behaviours to facilitate an overall culture of inclusion and respect. Partners acknowledged the intentional efforts of researchers to incorporate their perspectives, navigate challenges and communicate the value of partner input. CONCLUSION: By using collaborative decision-making in the early stages and throughout the study, researchers built trust with partners so that when decisions were difficult to resolve, partners still felt listened to and that their input mattered. PATIENT OR PUBLIC CONTRIBUTION: Members of the PCORI Patient Engagement Advisory Panel in 2019-2020 provided input into the design of the study, including the research questions and approaches to data collection and analysis.

Detection of SARS-CoV-2 by Canine Olfaction: A Pilot Study.

Background: As the number of coronavirus disease 2019 (COVID-19) cases continue to surge worldwide and new variants emerge, additional accurate, rapid, and noninvasive screening methods to detect severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) are needed. The number of COVID-19 cases reported globally is >455 million, and deaths have surpassed 6 million. Current diagnostic methods are expensive, invasive, and produce delayed results. While COVID-19 vaccinations are proven to help slow the spread of infection and prevent serious illness, they are not equitably available worldwide. Almost 40% of the world's population remains unvaccinated. Evidence suggests that SARS-CoV-2 virus-associated volatile organic compounds found in the breath, urine, and sweat of infected individuals can be detected by canine olfaction. Medical detection dogs may be a feasible, accurate, and affordable SARS-CoV-2 screening method. Methods: In this double-blinded, case-control, validation study, we obtained sweat samples from inpatients and outpatients tested for SARS-CoV-2 by a polymerase chain reaction test. Medical detection dogs were trained to distinguish SARS-CoV-2-positive samples from SARS-CoV-2-negative samples using reward-based reinforcement. Results: Samples were obtained from 584 individuals (6-97 years of age; 24% positive SARS-CoV-2 samples and 76% negative SARS-CoV-2 samples). In the testing phase, all dogs performed with high accuracy in detecting SARS-CoV-2. The overall diagnostic sensitivity was 98%, and specificity was 92%. In a follow-up phase, 1 dog screened 153 patients for SARS-CoV-2 in a hospital setting with 96% diagnostic sensitivity and 100% specificity. Conclusions: Canine olfaction is an accurate and feasible method for diagnosis of SARS-CoV-2, including asymptomatic and presymptomatic infected individuals.

Stakeholder-driven principles for advancing equity through shared measurement.

OBJECTIVE: To advance equity by developing stakeholder-driven principles of shared measurement, which is using a common set of measurable goals that reflect shared priorities across communities and systems, such as health care, public health, and human and social services. DATA SOURCES: From October 2019 to July 2021, we collected primary data from leaders in cross-systems alignment, measurement, and community engagement-including community members and community-based organization leaders-across the United States. STUDY DESIGN: In partnership with equity and community engagement experts, we conducted a mixed-methods study that included multiple formative research activities and culminated in a six-week, stakeholder-engaged modified-Delphi process. DATA COLLECTION: Formative data collection occurred through an environmental scan, interviews, focus groups, and an online survey. Principles were developed using a virtual modified Delphi with iterative rapid-analysis. Feedback on the final principles was collected through virtual focus groups, an online feedback form, and during virtual presentations. PRINCIPAL FINDINGS: We developed a set of five guiding principles. Measurement that aligns systems with communities toward equitable outcomes: (1) Requires upfront investment in communities; (2) Is co-created by communities; (3) Creates accountability to communities for addressing root causes of inequities and repairing harm; (4) Focuses on a holistic and comprehensive view of communities that highlights assets and historical context; and (5) Reflects long-term efforts to build trust. Using an equity-focused process resulted in principles with broad applicability. CONCLUSIONS: Leaders across systems and communities can use these shared measurement principles to reimagine and transform how systems create equitable health by centering the needs and priorities of the communities they serve, particularly communities that historically have been harmed the most by inequities. Intentionally centering equity across all project activities was essential to producing principles that could guide others in advancing equity.

Understanding the Influence and Impact of Stakeholder Engagement in Patient-centered Outcomes Research: a Qualitative Study.

BACKGROUND: Engaging patients and other stakeholders as partners in research offers promise in improving the relevance and usefulness of research findings. OBJECTIVE: To explore the influence and impact of patient and other stakeholder engagement on the planning and conduct of comparative effectiveness research studies. DESIGN: Qualitative study with virtual, hour-long semi-structured interviews. PARTICIPANTS: Fifty-eight researchers and fifty-one partners from a diverse purposeful sample of fifty-eight studies funded by the Patient-Centered Outcomes Research Institute (PCORI). APPROACH: Content and thematic analysis of interview data. KEY RESULTS: Described as an integral, long-term part of the research process, engagement influenced all aspects of the design and execution of studies. Partner influence was also dynamic and iterative, taking different forms over the course of the study. Across studies, we identified 387 discrete examples of influence and classified each as one of five types of influence, derived inductively from the interview data: co-producing, redirecting, refining, confirming, and limited. Most projects exhibited multiple types of influence, with 50 researchers and 41 partners reporting two or more types of influence within a project. Of the 387 examples of stakeholder influence, 306 had at least one reported impact on the study. Such impacts included changes to reflect the needs and preferences of patients or clinicians, as well as impacts on study feasibility, study quality, engagement scope or quality, and study relevance. Both researchers and partners identified multiple types of impact within projects, with 42 researchers and 38 partners reporting two or more types within a project. Because of these observable impacts, researchers and partners described engagement as worthwhile. CONCLUSIONS: Findings provide insights for funders and institutions supporting engagement, measurement efforts, and clinical researchers aiming to conduct engaged research and observe similar influences and impacts in their own studies.

Lessons Learned from Developing Plain Language Summaries of Research Studies.

BACKGROUND: Plain language translation may facilitate the public's ability to understand and use results of scientific research. Brief description of activity: This article describes the Patient-Centered Outcomes Research Institute's (PCORI) approach to and lessons learned from developing plain language summaries of PCORI-funded research for the lay public. IMPLEMENTATION: We developed and tested a standard template for the summaries, incorporating feedback in the template design from focus groups with members of the public. Between February 2017 and March 2020, we completed translation of 272 plain language summaries of PCORI-funded studies, covering topics including cardiovascular disease, obesity, cancer, mental health, asthma, HIV/AIDS, and comparative effectiveness research methods. RESULTS: Templates use a question-and-answer format, with sections on the rationale, methods, results, limitations, and how the research will help inform decisions. In addition to feedback on template heading wording and order, focus group participants stressed the importance of establishing relevance and conveying credibility and limitations. LESSONS LEARNED: Lessons learned relate to supporting consistency across individual summaries, carefully prioritizing content to include in the summaries, and balancing plain language and reading level with precision. These lessons learned from template development and implementation may be useful to other organizations or publishers contemplating similar efforts to make scientific research results more accessible. [HLRP: Health Literacy Research and Practice. 2021;5(2):e155-e161.] Plain Language Summary: The Patient-Centered Outcomes Research Institute (PCORI) funds comparative effectiveness research. This research compares the benefits and harms of two or more health care choices. In this article, we describe lessons learned from PCORI's efforts to develop plain language summaries of results from the research it funds. These lessons may help other organizations that want to share research results in plain language.

What motivates patients and caregivers to engage in health research and how engagement affects their lives: Qualitative survey findings.

BACKGROUND: US research organizations increasingly are supporting patient and stakeholder engagement in health research with a goal of producing more useful, relevant and patient-centered evidence better aligned with real-world clinical needs. The Patient-Centered Outcomes Research Institute (PCORI) engages patients, family caregivers and other health-care stakeholders, including clinicians, payers and policymakers, as active partners in prioritizing, designing, conducting and disseminating research as a key strategy to produce useful evidence for health-care decision making. OBJECTIVE: To inform effective engagement practices and policies, we sought to understand what motivates patients and caregivers to engage as partners on PCORI-funded research projects and how such engagement changed their lives. METHODS: We conducted thematic analysis of open-ended survey responses from 255 patients, family caregivers and individuals from advocacy and community-based organizations who engaged as partners on 139 PCORI-funded research projects focusing on a range of health conditions. RESULTS: Partners' motivations for engaging in research were oriented primarily towards benefiting others, including a desire to improve patients' lives and to support effective health-care interventions. In addition to feeling they made a positive difference, many partners reported direct benefits from engagement, such as new relationships and improved health habits. DISCUSSION AND CONCLUSIONS: By identifying patient and caregiver motivations for engaging in research partnerships and what they get out of the experience, our study may help research teams and organizations attract partners and foster more satisfying and sustainable partnerships. Our findings also add to evidence that engagement benefits the people involved as partners, strengthening the case for more widespread engagement.

An informed public's views on reducing antibiotic overuse.

OBJECTIVE: To understand public attitudes about and recommendations to address antibiotic overuse by employing public deliberation (a method for eliciting informed input on value-laden issues). DATA SOURCES/STUDY SETTING: Participants in 24 Community Deliberation groups (CD; n = 263), four Citizens' Panel groups (CP; n = 96), and a control group (n = 348). Data were collected in 2012 in four U.S. STUDY DESIGN: Using mixed methods, we analyzed quantitative and qualitative data from a randomized control trial. DATA COLLECTION/EXTRACTION METHODS: Using pre/postdeliberation surveys, we compared CD and CP participant attitude changes regarding antibiotic use to the control group. We analyzed deliberation transcripts using qualitative techniques to provide context for survey results. PRINCIPAL FINDINGS: Compared to control group participants, CD and CP participants had a larger postdeliberation shift in attitudes toward support of government limits on when doctors can prescribe antibiotics. Participants described unawareness about antibiotic overuse and called for education. When discussing prescription limits, participants debated tensions between preserving patient/doctor autonomy and protecting society from antibiotic-related harms. Participants saw patient, physician, and government roles in antibiotic stewardship policies/programs. CONCLUSION: When informed about individual and social consequences of antibiotic overuse, patients may be more receptive to antibiotic prescription limits. Community-physician-government partnerships are needed to create solutions.

Increasing the Use of Comparative Quality Information in Maternity Care: Results From a Randomized Controlled Trial.

This randomized controlled trial tested an intervention to increase uptake of hospital-level maternity care quality reports among 245 pregnant women in North Carolina (123 treatment; 122 control). The intervention included three enhancements to the quality report offered to the control: (a) biweekly text messages or e-mails directing women to the website, (b) videos and materials describing the relevance of quality measures to pregnant women's interests, and (c) tools to support discussions with clinicians. Compared with controls, intervention participants were significantly more likely to visit the website and report adopting behaviors to inform care, such as thinking through preferences, talking with their doctor, or creating a birth plan. Reports designed to put quality information into the larger context of what consumers want and need to know, along with targeted and timely communications, can increase consumer use of quality information and prompt them to talk with providers about care preferences and evidence-based practices.

Public Mistrust of the U.S. Health Care System's Profit Motives: Mixed-Methods Results from a Randomized Controlled Trial.

BACKGROUND: Decision makers are increasingly tasked with reducing health care costs, but the public may be mistrustful of these efforts. Public deliberation helps gather input on these types of issues by convening a group of diverse individuals to learn about and discuss values-based dilemmas. OBJECTIVE: To explore public perceptions of health care costs and how they intersect with medical mistrust. DESIGN AND PARTICIPANTS: This mixed-methods study analyzed data from a randomized controlled trial including four public deliberation groups (n = 96) and a control group (n = 348) comprising English-speaking adults aged 18 years and older. Data were collected in 2012 in four U.S. regions. APPROACH: We used data from four survey items to compare attitude shifts about costs among participants in deliberation groups to participants in the control group. We qualitatively analyzed deliberation transcripts to identify themes related to attitude shifts and to provide context for quantitative results about attitude shifts. KEY RESULTS: Deliberation participants were significantly more likely than control group participants to agree that doctors and patients should consider cost when making treatment decisions (ß = 0.59; p < 0.01) and that people should consider the effect on group premiums when making treatment decisions (ß = 0.48; p < 0.01). Qualitatively, participants mistrusted the health care system's profit motives (e.g., that systems prioritize making money over patient needs); however, after grappling with patient/doctor autonomy and learning about and examining their own views related to costs during the process of deliberation, they largely concluded that payers have the right to set some boundaries to curb costs. CONCLUSIONS: Individuals who are informed about costs may be receptive to boundaries that reduce societal health care costs, despite their mistrust of the health care system's profit motives, especially if decision makers communicate their rationale in a transparent manner. Future work should aim to develop transparent policies and practices that earn public trust.

Setting Boundaries: Public Views on Limiting Patient and Physician Autonomy in Health Care Decisions.

We obtained and qualitatively analyzed input from more than nine hundred citizens during seventy-six public deliberation sessions about patient and physician autonomy in decision making, setting health care boundaries, and the tensions among competing social values. Generally, participants resisted interference with the patient-physician relationship and believed strongly in the freedom of patient and physician to control individual medical decisions. However, during deliberation participants identified two situations where boundaries and regulations in health care were more acceptable: protecting people from harm and allocating limited resources. The core value of individual freedom was tempered in varying degrees by the values of concern for the greater good and fairness in allocating resources. Where tensions between values emerged, participants used different concepts-including accountability, transparency, trust, personal responsibility, and moral obligation-to navigate trade-offs. Fairly balancing the public's desire to protect individual freedom with their sense of responsibility for protecting the common good may be the key to developing acceptable, workable policies that promote evidence-based medical practice.

Detection of Bacteriuria by Canine Olfaction.

Background. Urinary tract infections (UTIs) are a significant medical problem , particularly for patients with neurological conditions and the elderly. Detection is often difficult in these patients, resulting in delayed diagnoses and more serious infections such as pyelonephritis and life-threatening sepsis. Many patients have a higher risk of UTIs because of impaired bladder function, catheterization, and lack of symptoms. Urinary tract infections are the most common nosocomial infection; however, better strategies are needed to improve early detection of the disease. Methods. In this double-blinded, case-control, validation study, we obtained fresh urine samples daily in a consecutive case series over a period of 16 weeks. Dogs were trained to distinguish urine samples that were culture-positive for bacteriuria from those of culture-negative controls, using reward-based clicker and treat methods. Results. Samples were obtained from 687 individuals (from 3 months to 92 years of age; 86% female and 14% male; 34% culture-positive and 66% culture-negative controls). Dogs detected urine samples positive for 100 000 colony-forming units/mL Escherichia coli (N = 250 trials; sensitivity 99.6%, specificity 91.5%). Dilution of E coli urine with distilled water did not affect accuracy at 1% (sensitivity 100%, specificity 91.1%) or 0.1% (sensitivity 100%, specificity 93.6%) concentration. Diagnostic accuracy was similar to Enterococcus (n = 50; sensitivity 100%, specificity 93.9%), Klebsiella (n = 50; sensitivity 100%, specificity 95.1%), and Staphylococcus aureus (n = 50; sensitivity 100%, specificity 96.3%). All dogs performed with similarly high accuracy: overall sensitivity was at or near 100%, and specificity was above 90%. Conclusions. Canine scent detection is an accurate and feasible method for detection of bacteriuria.

Understanding An Informed Public's Views On The Role Of Evidence In Making Health Care Decisions.

Policy makers and practitioners increasingly believe that medical evidence plays a critical role in improving care and health outcomes and lowering costs. However, public understanding of the role of evidence-based care may be different. Public deliberation is a process that convenes diverse citizens and has them learn about and consider ethical or values-based dilemmas and weigh alternative views. The Community Forum Deliberative Methods Demonstration project, sponsored by the Agency for Healthcare Research and Quality, obtained informed public views on the role of evidence in health care decisions through seventy-six deliberative groups involving 907 people overall, in the period August-November 2012. Although participants perceived evidence as being essential to high-quality care, they also believed that personal choice or clinical judgment could trump evidence. They viewed doctors as central figures in discussing evidence with patients and key arbiters of whether to follow evidence in individual cases. They found evidence of harm to individuals or the community to be more compelling than evidence of effectiveness. These findings indicate that increased public understanding of evidence can play an important role in advancing evidence-based care by helping create policies that better reflect the needs and values of the public.

Understanding Consumer Perceptions and Awareness of Hospital-Based Maternity Care Quality Measures.

OBJECTIVE: To explore factors that may influence use of comparative public reports for hospital maternity care. DATA SOURCES: Four focus groups conducted in 2013 with 41 women and preintervention survey data collected in 2014 to 2015 from 245 pregnant women in North Carolina. STUDY DESIGN: As part of a larger randomized controlled trial, we conducted qualitative formative research to develop an intervention that will be evaluated through pre- and postintervention surveys. DATA EXTRACTION METHODS: Analysis of focus group transcripts examined participants' perceptions of high-quality maternity care and the importance of different quality measures. Quantitative analysis included descriptive results of the preintervention survey and subgroup analyses to examine the impact of race, education, and being a first-time mom on outcomes. PRINCIPAL FINDINGS: When describing high-quality maternity care, participants focused on interactions with providers, including respect for preferences and communication. The importance of quality measures was influenced by the extent to which they focused on babies' health, were perceived as the hospital's responsibility, and were perceived as representing "standard care." At baseline, 28 percent of survey respondents had used quality information to choose a hospital. Survey respondents were more aware of some quality measures (e.g., breastfeeding support) than others (e.g., episiotomy rates). CONCLUSIONS: Public reporting efforts could help increase relevance of maternity care quality measures by creating measures that reflect women's concerns, clearly explaining the hospital's role in supporting quality care, and showing how available quality measures can inform decisions about childbirth.

Creating Change in Health Care: Developing a Shared Understanding and Roadmap for Action.

The Patient and Family Engagement Framework and the Roadmap for Patient and Family Engagement in Healthcare Practice and Research are tools designed to create a shared understanding of what patient and family engagement is and how it can be translated into concrete action.

Effectiveness of public deliberation methods for gathering input on issues in healthcare: Results from a randomized trial.

UNLABELLED: Public deliberation elicits informed perspectives on complex issues that are values-laden and lack technical solutions. This Deliberative Methods Demonstration examined the effectiveness of public deliberation for obtaining informed public input regarding the role of medical evidence in U.S. healthcare. We conducted a 5-arm randomized controlled trial, assigning participants to one of four deliberative methods or to a reading materials only (RMO) control group. The four deliberative methods reflected important differences in implementation, including length of the deliberative process and mode of interaction. The project convened 76 groups between August and November 2012 in four U.S. LOCATIONS: Chicago, IL; Sacramento, CA; Silver Spring, MD; and Durham, NC, capturing a sociodemographically diverse sample with specific attention to ensuring inclusion of Hispanic, African-American, and elderly participants. Of 1774 people recruited, 75% participated: 961 took part in a deliberative method and 377 participants comprised the RMO control group. To assess effectiveness of the deliberative methods overall and of individual methods, we evaluated whether mean pre-post changes on a knowledge and attitude survey were statistically different from the RMO control using ANCOVA. In addition, we calculated mean scores capturing participant views of the impact and value of deliberation. Participating in deliberation increased participants' knowledge of evidence and comparative effectiveness research and shifted participants' attitudes regarding the role of evidence in decision-making. When comparing each deliberative method to the RMO control group, all four deliberative methods resulted in statistically significant change on at least one knowledge or attitude measure. These findings were underscored by self-reports that the experience affected participants' opinions. Public deliberation offers unique potential for those seeking informed input on complex, values-laden topics affecting broad public constituencies.

Unresolved tensions in consumer engagement in CER: a US research perspective.

Efforts to engage patients and consumers in comparative effectiveness research (CER) in the USA are still in the early stages, and the outcomes of these partnerships have yet to be fully understood or realized. Our work assisting federal agencies and national organizations who engage patients and consumers in CER reveal three unresolved tensions around the representation of the patient experience in the research process, the culture of research and capacity to partner with patients and consumers, and the conflict between the methodological approaches to CER and the outcomes of interest to patients and caregivers. Several approaches to address these tensions have emerged, yet resolving these tensions will require addressing many system-level challenges and building an evidence base for consumer engagement in CER.

Patient and family engagement: a framework for understanding the elements and developing interventions and policies.

Patient and family engagement offers a promising pathway toward better-quality health care, more-efficient care, and improved population health. Since definitions of patient engagement and conceptions of how it works vary, we propose a framework. We first present the forms engagement can take, ranging from consultation to partnership. We discuss the levels at which patient engagement can occur across the health care system, from the direct care setting to incorporating patient engagement into organizational design, governance, and policy making. We also discuss the factors that influence whether and to what extent engagement occurs. We explore the implications of our multidimensional framework for the development of interventions and policies that support patient and family engagement, and we offer a research agenda to investigate how such engagement leads to improved outcomes.

Evidence that consumers are skeptical about evidence-based health care.

We undertook focus groups, interviews, and an online survey with health care consumers as part of a recent project to assist purchasers in communicating more effectively about health care evidence and quality. Most of the consumers were ages 18-64; had health insurance through a current employer; and had taken part in making decisions about health insurance coverage for themselves, their spouse, or someone else. We found many of these consumers' beliefs, values, and knowledge to be at odds with what policy makers prescribe as evidence-based health care. Few consumers understood terms such as "medical evidence" or "quality guidelines." Most believed that more care meant higher-quality, better care. The gaps in knowledge and misconceptions point to serious challenges in engaging consumers in evidence-based decision making.