Evaluating the Impacts of Patient Engagement on Health Services Research Teams: Lessons from the Veteran Consulting Network.

BACKGROUND: Despite increasing commitment to patient engagement in research, evaluation of the impact of these efforts on research processes, products, and teams is limited. OBJECTIVE: To explore the impacts of engaging patients as consultants to research studies by examining the experiences, impacts, and lessons learned from a program facilitating patient engagement at a Veterans Health Administration research center. DESIGN: We developed a logic model to articulate the activities being implemented to support patient engagement and their anticipated outcomes. Then, we conducted qualitative, semi-structured interviews with participants in the local Veteran Consulting Network to qualitatively explore these outcomes. PARTICIPANTS: Twelve researchers and eleven Veteran patients with experience working on at least one grant or funded study. APPROACH: Interview transcripts were inductively coded using a consensus-based approach. Findings were synthesized using framework analysis and mapped back onto our logic model of expected patient engagement impacts. KEY RESULTS: Patient engagement improved the perceived quality and relevance of research studies as patient consultants challenged researchers' assumptions about patient populations and clinical contexts and gave feedback that helped improve the feasibility of proposed grants, readability of study materials, comprehensiveness of study assessments, and cultural sensitivity and relevance of interventions. Patient engagement also had personal benefits to researchers and patients. Researchers reported improved communication skills and higher job satisfaction. Patients reported a sense of purpose and satisfaction from their work with greater awareness of and appreciation for research. CONCLUSIONS: Engaging patients in research can have multiple benefits to the people and work involved. Our evaluation process can serve as a template for other organizations to plan for and assess the impact of their own patient engagement programs. Creating logic models and updating them based on feedback from program users make engagement goals explicit, help verify expected mechanisms to achieve impact, and facilitate organizational learning.

Concerns of Primary Care Clinicians Practicing in an Integrated Health System: a Qualitative Study.

BACKGROUND: Clinician well-being is a major priority for healthcare organizations. However, the impact of workplace environment on clinicians' well-being is poorly understood. Integrated health systems are a particularly relevant type of practice environment to focus on, given the increasing rates of practice consolidation and integration. OBJECTIVE: To improve understanding of the concerns of primary care clinicians (PCCs) practicing in an integrated health system. DESIGN: We analyzed free-text comment box responses offered on a national survey about care coordination by 555 PCCs in the Veterans Health Administration, one of the largest integrated health systems in the USA. PARTICIPANTS: A total of 555 PCCs who left free-text comments on a national survey of care coordination in the VHA (30% out of 1862 eligible respondents). Demographics and coordination scale scores were similar between respondents who left comments vs. those who did not. APPROACH: The data were coded and analyzed in line with the grounded theory approach. Key themes were identified by team consensus and illustrative quotations were chosen to illustrate each theme. KEY RESULTS: VHA PCCs described some pressures shared across practice environments, such as prohibitive administrative burden, but also reported several concerns particular to integrated settings, including "dumping" by specialists and moral distress related to a concern for patients. Frustrations due to several aspects of responsibility around referrals may be unique to integrated health systems with salaried clinicians and/or where specialists have the ability to reject referrals. CONCLUSION: PCCs in integrated health systems feel many of the same pressures as their counterparts in non-integrated settings, but they are also confronted with unique stressors related to these systems' organizational features that restrict clinicians' autonomy. An understanding of these concerns can guide efforts to improve the well-being of PCCs in existing integrated health systems, as well as in practices on their way to integration.

Coordination of Care as Experienced by the Specialist: Validation of the CSC-Specialist Survey in the Private Sector and the Effect of a Shared Electronic Health Record.

BACKGROUND: We previously developed 2 complementary surveys to measure coordination of care as experienced by the specialist and the primary care provider (PCP). These Coordination of Specialty Care (CSC) surveys were developed in the Veterans Health Administration (VA), under an integrated organizational umbrella that includes a shared electronic health record (EHR). OBJECTIVE: To develop an augmented version of the CSC-Specialist in the private sector and use that version (CSC-Specialist 2.0) to examine the effect of a shared EHR on coordination. RESEARCH DESIGN: We administered the survey online to a national sample of clinicians from 10 internal medicine subspecialties. We used multitrait analysis and confirmatory factor analysis to evaluate the psychometric properties of the original VA-based survey and develop an augmented private sector survey (CSC-Specialist 2.0). We tested construct validity by regressing a single-item measure of overall coordination onto the 4 scales. We used analysis of variance to examine the relationship of a shared EHR to coordination. RESULTS: Psychometric assessment supported the 13-item, 4-scale structure of the original VA measure and the augmented 18-item, 4-scale structure of the CSC-Specialist 2.0. The CSC-Specialist 2.0 scales together explained 45% of the variance in overall coordination. A shared EHR was associated with significantly better scores for the Roles and Responsibilities and Data Transfer scales, and for overall coordination. CONCLUSIONS: The CSC-Specialist 2.0 is a unique survey that demonstrates adequate psychometric performance and is sensitive to use of a shared EHR. It can be used alone or with the CSC-PCP to identify coordination problems, guide interventions, and measure improvements.

Electronic consultations (E-consults) and their outcomes: a systematic review.

OBJECTIVE: Electronic consultations (e-consults) are clinician-to-clinician communications that may obviate face-to-face specialist visits. E-consult programs have spread within the US and internationally despite limited data on outcomes. We conducted a systematic review of the recent peer-reviewed literature on the effect of e-consults on access, cost, quality, and patient and clinician experience and identified the gaps in existing research on these outcomes. MATERIALS AND METHODS: We searched 4 databases for empirical studies published between 1/1/2015 and 2/28/2019 that reported on one or more outcomes of interest. Two investigators reviewed titles and abstracts. One investigator abstracted information from each relevant article, and another confirmed the abstraction. We applied the GRADE criteria for the strength of evidence for each outcome. RESULTS: We found only modest empirical evidence for effectiveness of e-consults on important outcomes. Most studies are observational and within a single health care system, and comprehensive assessments are lacking. For those outcomes that have been reported, findings are generally positive, with mixed results for clinician experience. These findings reassure but also raise concern for publication bias. CONCLUSION: Despite stakeholder enthusiasm and encouraging results in the literature to date, more rigorous study designs applied across all outcomes are needed. Policy makers need to know what benefits may be expected in what contexts, so they can define appropriate measures of success and determine how to achieve them.

Tools to improve referrals from primary care to specialty care.

OBJECTIVES: Referrals from primary to specialty care are a critical first step in coordination of specialty care, but shortcomings in the appropriateness, clarity, or completeness of referrals are common. We examined (1) whether 3 tools to coordinate specialty care are associated with better referral characteristics and (2) whether greater perceived helpfulness of these tools is associated with better referral characteristics among specialists who use all 3 of them. STUDY DESIGN: National online survey about care coordination among medical specialists receiving referrals in the Veterans Health Administration. METHODS: Adjusted odds ratios (ORs) for associations between use and helpfulness of 3 coordination tools (service agreements, referral templates, and e-consults) and perceived frequency of 3 referral characteristics (appropriateness, clarity, and completeness). RESULTS: Among specialists (N = 497), use of referral templates was associated with perceptions that referrals were more frequently appropriate (adjusted OR, 1.5; 95% CI, 1.0-2.4), clear (adjusted OR, 1.6; 95% CI, 1.0-2.5), and complete (adjusted OR, 1.9; 95% CI, 1.1-3.2). Use of e-consults was associated with more frequent referral clarity (adjusted OR, 1.7; 95% CI, 1.0-3.0). Among specialists using all 3 tools, those reporting that templates were very helpful also perceived more frequent referral clarity (adjusted OR, 3.1; 95% CI, 1.1-8.5) and completeness (adjusted OR, 3.6; 95% CI, 1.5-8.7). Service agreements were not associated with any referral characteristic. CONCLUSIONS: Well-designed referral templates may help improve the clarity and completeness of primary care-specialty care referrals. Existing templates may provide models that can be adapted in collaboration with primary care and broadly applied to improve referrals. Work is needed to improve the impact of service agreements and e-consults on referrals.