RE-AIM applied to a primary care workforce training for rural providers and nurses: the Department of Veterans Affairs' Rural Women's Health Mini-Residency.

Introduction: Application of the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework to evaluate workforce education and training programs targeting clinical health care staff has received relatively little attention. This paper aims to contribute to this area with RE-AIM findings from a women's health-focused workforce training program implemented by the U.S. Department of Veterans Affairs (VA). Over the past two decades, the rapid expansion of the women Veteran population in VA has necessitated a quick response to meet clinical demand. To address this health care need, the VA Offices of Rural Health (ORH) and Women's Health (OWH) partnered to deploy a primary care workforce development initiative for Rural Providers and Nurses-the Rural Women's Health Mini-Residency (Rural WH-MR)-to train VA clinicians in rural locations in skills for the care of women Veterans. Here we assess the applicability of RE-AIM as an evaluation framework in this context. Methods: We evaluated the Rural WH-MR, relying on a primarily quantitative approach, rooted in RE-AIM. It included longitudinal and cross-sectional measurements from multiple quantitative and qualitative data sources to develop selected metrics. Data collection instruments consisted of pre-, post-, and follow-up training surveys, course evaluations, existing VA databases, and implementation reports. We developed metrics for and assessed each RE-AIM component by combining data from multiple instruments and then triangulating findings. Results: Results from the Rural WH-MR program for fiscal years 2018-2020 indicate that RE-AIM provides an instructive evaluation framework for a rural workforce training program, particularly in eliciting clarity between measures of Reach vs. Adoption and focusing attention on both provider- and patient-level outcomes. Discussion: We describe evaluation metric development and barriers to and facilitators of utilizing RE-AIM as an evaluation framework for a provider- and nurse-facing intervention such as this workforce training program. We also reflect upon RE-AIM benefits for highlighting process and outcomes indicators of a training program's success and lessons learned for evaluating rural workforce development innovations. Several of our observations have implications for training and evaluation approaches in rural areas with more limited access to health care services.

Implementation experience and initial assessment of a rural women's health training program in support of the U.S. Department of Veterans Affairs as a learning health system.

Introduction: The increasing number of women Veterans receiving health care from the Veterans Health Administration (VHA) has spurred the need for more women's health primary care providers (PCPs) and nurses, including in rural areas nationwide. Here we report on the implementation of a women's health rural workforce training program, demonstrate initial evidence of its effectiveness within VHA as a Learning Health System, and present lessons learned and implications for other workforce training programs. Methods: The Women's Health Primary Care Mini-Residency for Rural Providers and Nurses (Rural WH-MR) is a mobile VHA training initiative adapted from a national training model. The Rural WH-MR uses asynchronous blended learning paired with in-person hands-on instruction delivered directly at rural VHA sites. Mixed methods evaluation using quantitative data, qualitative interviews, and observational feedback assessed the program's implementation feasibility, fidelity, acceptability, and appropriateness. Longitudinal survey data were used to assess the initial program impact via changes in participating PCP and nurse knowledge, attitudes, practices, and skills (KAPS). Results: Inclusive of the pilot and fiscal years 2018 and 2019 Rural WH-MR trainings, 181 PCPs, and 320 nurses were trained through 56 training events nationwide. Cumulative survey data using 5-point measures showed high participant satisfaction, achievement of program-specific objectives, and usefulness of training activities to the rural practice of both PCPs and nurses. Both a pre-training and 6-month-follow-up survey were completed by 52 PCPs (32.9%) and 93 nurses (32.2%) and revealed significant sustained improvements in 18 out of 22 KAPS (p < 0.01-0.03) areas assessed for PCPs and all 17 KAPS (p < 0.01) areas assessed for nurses. Conclusions: This adapted training program benefitted VHA's rural clinical workforce thereby contributing to the VHA goal of increasing the numbers of rural women Veterans with access to PCPs and nurses with women's health expertise.

Women Veterans' Experiences of Harassment and Perceptions of Veterans Affairs Health Care Settings During a National Anti-Harassment Campaign.

PURPOSE: In 2017, Veterans Health Administration (VA) launched a social marketing and training campaign to address harassment of women veterans at VA health care facilities. We assessed women veterans' experiences of harassment, reported perpetrators of harassment, and perceptions of VA in 2017 (before campaign launch) and 2018 (1 year after campaign implementation). METHODS: We administered surveys to women veterans attending primary care appointments (2017, n = 1,300; 2018, n = 1,711). Participants reported whether they experienced sexual harassment (e.g., catcalls) and gender harassment (e.g., questioning women's veteran status) from patients and/or staff at VA in the past 6 months. They also indicated whether they felt welcome, felt safe, and believed the VA is working to address harassment. We compared variables in 2017 versus 2018 with χ2 analyses, adjusting for facility-level clustering. RESULTS: There were no significant differences in percentages of participants reporting sexual harassment (20% vs. 17%) or gender harassment (11% vs. 11%) in 2017 versus 2018. Men veterans were the most frequently named perpetrators, but participants also reported harassment from staff. Participant beliefs that VA is working to address harassment significantly improved from 2017 to 2018 (52% vs. 57%; p = .05). CONCLUSIONS: One year after campaign launch, women veterans continued to experience harassment while accessing VA health care services. Findings confirm that ongoing efforts to address and monitor both staff- and patient-perpetrated harassment are essential. Results have implications for future anti-harassment intervention design and implementation and highlight additional opportunities for investigation.

Promoting learning health system feedback loops: Experience with a VA practice-based research network card study.

BACKGROUND: We tested the capacity of the 60-site VA Women's Health Practice-Based Research Network (WH-PBRN), embedded within VA, to employ a multisite card study to collect women Veterans' perspectives about Complementary and Integrative Health (CIH) and to rapidly return findings to participating sites and partnered national policy-makers in support of a Learning Health System (LHS) wherein evidence generation informs ongoing improvement. METHODS: VA primary care clinic clerks and nurses distributed anonymous surveys (patient feedback forms) at clinics for up to two weeks in fiscal year 2017, asking about CIH behavior and preferred delivery methods. We examined the project's feasibility, representativeness, acceptability, and impact via a tracking system, national administrative data, debriefing notes, and three surveys of WH-PBRN Site Leads. RESULTS: Twenty geographically diverse and largely representative VA Medical Centers and 11 Community-Based Outpatient Clinics volunteered to participate. Over six months, N = 1191 women Veterans responded (median 57; range 8-151 per site). In under three months, we returned local findings benchmarked against multisite findings to all participating sites and summary findings to national VA partners. Sites and partners disseminated results to clinical and leadership stakeholders, who then applied results as warranted. CONCLUSIONS: VA effectively mobilized an embedded PBRN to implement a timely, representative, acceptable and impactful operations project. IMPLICATIONS: Card studies by PBRNs within large, national healthcare systems can provide rapid feedback to participating sites and national leaders to guide policies, programs, and practices. LEVEL OF EVIDENCE: Self-selected respondents could have biased results.

Mobilizing embedded research and operations partnerships to address harassment of women Veterans at VA medical facilities.

Key insights: A: Addressing a complex problem like harassment in VA medical facilities requires committed, engaged collaboration at multiple levels of the organization. B: Timely feedback of initial research findings to operations partners enabled rapid and more responsive development of new programs and policies. C: Our research-clinical partnership has enabled us to pursue targeted change from the outset, while incorporating real-time findings from embedded researchers working to develop a comprehensive understanding of the problem.

Women Veterans' Perspectives on How to Make Veterans Affairs Healthcare Settings More Welcoming to Women.

PURPOSE: Women veterans are a rapidly increasing subset of the Veterans Affairs (VA) patient population but remain a numerical minority. Men veteran-dominated health care settings pose unique considerations for providing care to women veterans in a comfortable and welcoming environment. We analyzed patient suggestions on how to make the VA more welcoming to women. METHODS: We surveyed a convenience sample of women veteran patients who visited 1 of 26 VA locations in August and September of 2017. Women veterans were invited to complete brief anonymous questionnaires that included questions about harassment experiences and feeling welcome at the VA, and an open-ended question about suggestions to make the VA more welcoming to women. We analyzed data from the open-ended question using the constant comparison method. RESULTS: Among respondents (N = 1,303), 85% felt welcome at the VA. Overall, 29% answered the open-ended prompt for a total of 490 distinct responses: 260 comments and 230 suggestions. Comments included praise for the VA (67%) and stories about feeling uncomfortable or harassed in the VA (26%). Suggestions included those related to VA staff (31%), the environment of care (18%), additional resources for women veterans (18%), clinical services for women veterans (15%), changing men veterans' behavior toward women veterans at the VA (5%), and making the treatment of women and men the same (5%). CONCLUSIONS: Although most women veterans felt welcome in the VA, patient-centered suggestions offer opportunities for making the VA more welcoming to women. Soliciting patient suggestions and increasing awareness of how feeling welcome is experienced by patients are first steps to health care settings becoming more inclusive.

The Global HIV Archive: Facilitating the Transition from Science to Practice of Efficacious HIV Prevention Interventions.

This paper describes the development, content, and capabilities of the online Global HIV Archive (GHA). With the goal of facilitating widespread adaptation and appropriate use of efficacious HIV prevention programs throughout the globe, GHA has: first, expanded and updated the search for HIV prevention programs originating in low-resource countries; second, identified those meritorious HIV prevention programs meeting established efficacy criteria of technical merit, replicability, and positive outcomes; third, prepared both implementation and evaluation materials from the efficacious programs for public use; fourth, developed interactive wizards or capacity-building tools to facilitate appropriate program selection, implementation, and adaptation; and, fifth, made the efficacious programs and accompanying wizards available to health practitioners throughout the globe in both printed and online formats.

Update and expansion of the HIV/AIDS prevention program archive (HAPPA).

Established in 1996 with funding from CDC and NIH, the HIV/AIDS Prevention Program Archive (HAPPA) is now the biggest private sector collection of HIV-related evidence-based behavioral interventions (EBIs). Each EBI in HAPPA has been determined by a distinguished Scientist Expert Panel to have demonstrated efficacy in preventing HIV or its risk-related behaviors in the United States. The multimedia replications kits contain everything that a new site would need to implement an EBI such as a user guide that gives an overview of the program and the evidence of its effectiveness; a facilitator's manual that gives step-by-step implementation protocols for each session; and session implementation materials referenced in the facilitator's manual such as slides, video clips, participant handouts, activity masters, checklists, and homework assignments for the next session. The program packages also contain evaluation materials such as surveys and questionnaires that were used in the original demonstration of effectiveness and that may be used to re-evaluate the program as implemented in a new setting. Recently, we have expanded HAPPA's scope to include HIV EBIs developed globally and to include evidence-based structural interventions (effective in modifying the physical, social, cultural, political, economic, legal, and/or policy aspects of the HIV risk environment). This paper describes HAPPA's procedures for identifying, selecting, acquiring and packaging HIV EBIs. It also provides comprehensive lists of evidence-based HIV behavioral and structural interventions and gives information on how to access EBI program packages for implementation in new settings.