Maintaining Program Fidelity in a Changing World: National Implementation of a School-Based HIV Prevention Program.

Large-scale, evidence-based interventions face challenges to program fidelity of implementation. We developed implementation strategies to support teachers implementing an evidence-based HIV prevention program in schools, Focus on Youth in The Caribbean (FOYC) and Caribbean Informed Parents and Children Together (CImPACT) in The Bahamas. We examined the effects of these implementation strategies on teachers' implementation in the subsequent year after the initial implementation during the COVID-19 pandemic. Data were collected from 79 Grade 6 teachers in 24 government elementary schools. Teachers completed training workshops and a pre-implementation questionnaire to record their characteristics and perceptions that might affect their program fidelity. School coordinators and peer mentors provided teachers with monitoring, feedback, and mentoring. In Year 1, teachers on average taught 79.3% of the sessions and 80.8% of core activities; teachers in Year 2 covered 84.2% of sessions and 72.9% of the core activities. Teachers with "good" or "excellent" school coordinators in the second year taught significantly more sessions on average (7.8 vs. 7.0, t = 2.04, P < 0.05) and more core activities (26.3 vs. 23.0, t = 2.41, P < 0.05) than teachers with "satisfactory" coordinators. Teachers who had a "good" or "satisfactory" mentor taught more sessions than teachers who did not have a mentor (7.9 vs. 7.3; t = 2.22; P = 0.03). Two-level mixed-effects model analysis indicated that teachers' program fidelity in Year 1, confidence in the execution of core activities, and school coordinators' performance were significantly associated with Year 2 implementation dose. Implementation of FOYC + CImPACT was significantly associated with improved student outcomes. Teachers maintained high fidelity to a comprehensive HIV prevention program over 2 years during the COVID-19 pandemic. Future program implementers should consider additional implementation support to improve the implementation of school-based programs.

Comparing standard versus enhanced implementation of an evidence-based HIV prevention program among Bahamian sixth grade students: findings from nationwide implementation trials.

BACKGROUND: Effective implementation strategies are needed to address the challenges encountered by teachers in implementation of evidence-based HV prevention programs in schools. The current study: 1) compares implementation fidelity of Focus on Youth in the Caribbean (FOYC) plus Caribbean Informed Parents and Children Together (CImPACT) intervention using enhanced implementation strategies (including biweekly monitoring/feedback and site-based mentoring) to those using more traditional approach (teacher training only); and 2) evaluates the impact of school coordinators' and mentors' performance on teachers' implementation fidelity and student outcomes. METHODS: Data from an enhanced implementation trial in 2019-2020, involving 24 government primary schools, 79 teachers, and 2252 students, were compared to data from a standard implementation trial in 2011-2012, involving 35 government primary schools, 110 teachers and 2811 students using mixed-effects modeling and structural equation modeling. FINDINGS: Teachers in the 2019-2020 trial taught more core activities (28.3 vs. 16.3, t = 10.80, P < 0.001) and sessions (7.2 vs. 4.4, t = 9.14, P < 0.001) than those participating in the 2011-2012 trial. Teachers taught > 80% of the intervention curriculum in 2019-2020 compared to 50% curriculum delivery in 2011-2012. Teachers who had a "very good" or "excellent" school coordinator in their schools taught more core activities than those who had a "satisfactory" school coordinator (30.4 vs. 29.6 vs. 22.3, F = 18.54, P < 0.001). Teachers who worked in a school which had a "very good" mentor, taught more core activities than those teachers who did not have a mentor or had only a "satisfactory" mentor (30.4 vs. 27.6; t = 2.96; p = 0.004). Teachers' confidence in implementing core activities, comfort level with the curriculum, attitudes towards sex education in schools, and perceived principal support were significantly related to increased self-efficacy, which in turn was related to teachers' implementation fidelity. The degree of implementation was significantly associated with improved student outcomes. IMPLICATIONS/CONCLUSION: An evidence-based HIV prevention intervention can achieve a high degree of implementation when delivered with enhanced implementation strategies and implementation monitoring. Future program implementers should consider the purposeful selection and training of school coordinators and mentors to support low-implementing teachers as a potentially important strategy when attempting to achieve high-quality implementation of school-based interventions.

Bahamas National Implementation Project: Proposal for Sustainability of an Evidence-based HIV Prevention Intervention in a School Setting.

BACKGROUND: Sustained implementation of school-based prevention programs is low. Effective strategies are needed to enhance both high-level implementation fidelity and sustainability of prevention programs. OBJECTIVE: This proposed study aims to determine if the provision of either biweekly monitoring and feedback and site-based assistance and mentorship or both to at-risk and moderate-performing teachers with monitoring through an enhanced decision-making platform by the Ministry of Education (MOE) and Ministry of Health (MOH) based on the real-time implementation data will increase national implementation fidelity and result in sustained implementation over time. METHODS: This study will target government schools including 200 grade 6 teachers in 80 primary schools and 100 junior/middle high school teachers (and their classes) on 12 Bahamian islands. Teacher and school coordinator training will be conducted by the MOE in year 1, followed by an optimization trial among teachers in the capital island. Informed by these results, an implementation intervention will be conducted to train using different levels of educational intensity all at-risk and moderate-performing teachers. Subsequently selected training and implementation strategies will be evaluated for the national implementation of Focus on Youth in the Caribbean and Caribbean Informed Parents and Children Together in years 2 to 5. RESULTS: It is hypothesized that a more intensive training and supervision program for at-risk and moderate-performing teachers will enhance their implementation fidelity to the average level of the high-performing group (85%), an HIV prevention program delivered at the national level can be implemented with fidelity in grade 6 and sustained over time (monitored annually), and student outcomes will continue to be highly correlated with implementation fidelity and be sustained over time (assessed annually through grade 9). The proposed study is funded by the National Institute of Child Health and Human Development from August 1, 2018, through May 31, 2023. CONCLUSIONS: The study will explore several theory-driven implementation strategies to increase sustained teacher implementation fidelity and thereby increase the general public health impact of evidence-based interventions. The proposed project has potential to make significant contributions to advancing school-based HIV prevention research and implementation science and serve as a global model for the Fast Track strategy. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): PRR1-10.2196/14816.

Multi-year school-based implementation and student outcomes of an evidence-based risk reduction intervention.

BACKGROUND: Intervention effects observed in efficacy trials are rarely replicated when the interventions are broadly disseminated, underscoring the need for more information about factors influencing real-life implementation and program impact. Using data from the ongoing national implementation of an evidence-based HIV prevention program [Focus on Youth in The Caribbean (FOYC)] in The Bahamas, this study examines factors influencing teachers' patterns of implementation, the impact of teachers' initial implementation of FOYC, and subsequent delivery of the booster sessions on students' outcomes. METHODS: Data were collected from the 80 government elementary and 34 middle schools between 2011 and 2014, involving 208 grade 6, 75 grade 7, and 58 grade 8 teachers and 4411 students initially in grade 6 and followed for 3 years. Student outcomes include HIV/AIDS knowledge, reproductive health skills, self-efficacy, and intention to use protection. Data from teachers includes implementation and modification of the curriculum, attitudes towards the prevention program, comfort level with the curriculum, and attendance at training workshops. Structural equation modeling and mixed-effect modeling analyses were applied to examine the impact of teachers' implementation. RESULTS: Teachers' attitudes towards and comfort with the intervention curriculum, and attendance at the curriculum training workshop had a direct effect on teachers' patterns of implementation, which had a direct effect on student outcomes. Teachers' attitudes had a direct positive effect on student outcomes. Teachers' training in interactive teaching methods and longer duration as teachers were positively associated with teachers' comfort with the curriculum. High-quality implementation in grade 6 was significantly related to student outcomes in grades 6 and 7 post-implementation. Level of implementation of the booster sessions in grades 7 and 8 were likewise significantly related to subsequent student outcomes in both grades. CONCLUSIONS: High-quality initial implementation of a prevention program is significantly related to better program outcomes. Poor subsequent delivery of booster sessions can undermine the positive effects from the initial implementation while strong subsequent delivery of booster sessions can partially overcome poor initial implementation.

Adolescent HIV Risk Reduction in the Bahamas: Results from Two Randomized Controlled Intervention Trials Spanning Elementary School Through High School.

To address global questions regarding the timing of HIV-prevention efforts targeting youth and the possible additional benefits of parental participation, researchers from the USA and The Bahamas conducted two sequential longitudinal, randomized trials of an evidence-based intervention spanning the adolescent years. The first trial involved 1360 grade-6 students and their parents with three years of follow-up and the second 2564 grade-10 students and their parents with two years of follow-up. Through grade-12, involvement in the combined child and parent-child HIV-risk reduction interventions resulted in increased consistent condom-use, abstinence/protected sex, condom-use skills and parent-child communication about sex. Receipt of the grade-6 HIV-prevention intervention conferred lasting benefits regarding condom-use skills and self-efficacy. Youth who had not received the grade-six intervention experienced significantly greater improvement over baseline as a result of the grade-10 intervention. The HIV-risk reduction intervention delivered in either or both grade-6 and grade-10 conferred sustained benefits; receipt of both interventions appears to confer additional benefits.

The Impact of Teachers' Modifications of an Evidenced-Based HIV Prevention Intervention on Program Outcomes.

The degree to which evidence-based program outcomes are affected by modifications is a significant concern in the implementation of interventions. The ongoing national implementation of an evidence-based HIV prevention program targeting grade 6 students in The Bahamas [Focus on Youth in The Caribbean (FOYC)] offers an opportunity to explore factors associated with teachers' modification of FOYC lessons and to examine the impact of types and degrees of modifications on student outcomes. Data were collected in 2012 from 155 teachers and 3646 students in 77 government elementary schools. Results indicate that teachers taught 16 of 30 core activities, 24.5 of 46 total activities and 4.7 of 8 sessions. Over one-half of the teachers made modifications to FOYC core activities; one-fourth of the teachers modified 25 % or more core activities that they taught (heavily modified FOYC). Omitting core activities was the most common content modification, followed by lengthening FOYC lessons with reading, writing assignments or role-play games, and shortening core activities or adding educational videos. Mixed-effects modeling revealed that omitting core activities had negative impacts on all four student outcomes. Shortening core activities and adding videos into lessons had negative impacts on HIV/AIDS knowledge and/or intention to use condom protection. Heavy modifications (>1/4 core activities) were associated with diminished program effectiveness. Heavy modifications and omitting or shortening core activities were negatively related to teachers' level of implementation. We conclude that poorer student outcomes were associated with heavy modifications.

Cross-sectional data on alcohol and marijuana use and sexual behavior among male and female secondary school students in New Providence, The Bahamas.

BACKGROUND: While The Bahamas have significantly reduced poor reproductive health outcomes among adolescents and emerging adults, data indicate that youth are engaged in sexual risk behaviors. Substance use has been linked to increased risk for HIV and sexually transmitted infections in other contexts. There are limited data on Bahamian youth in relation to consumption of alcohol and marijuana use and engagement in sexual behaviors. OBJECTIVE: This study aimed to assess potential relationships between alcohol and marijuana use and engagement in sexual behavior among government secondary school students in New Providence, The Bahamas. MATERIALS AND METHODS: Total sample size was 2572, and about 56% of respondents were female. Mean age was 14.2 (SD 2.7 years). Cross-sectional data came from a baseline survey conducted as part of a longitudinal randomized controlled evaluation of a school-based HIV prevention and reproductive health program in New Providence. RESULTS: Overall, 46.5% (519) males and 44.8% (652) females reported alcohol consumption; 7.3% (82) males and 1.7% (25) females reported use of marijuana in the last 6 months. About 43% (477) male respondents and 16% (231) female respondents reported ever having vaginal sex. Logistic regression analysis indicates that increased likelihood of engaging in sex during the past 6 months is associated with being older, male, and consuming alcohol and marijuana. CONCLUSION: These data provide a 'global correlation' between substance use and engagement in sexual behaviors among Bahamian adolescents. Longitudinal research is needed to assess event specific risks and identify mediating and moderating factors. These findings indicate the importance of integrating reproductive health and substance use education.

Factors influencing implementation dose and fidelity thereof and related student outcomes of an evidence-based national HIV prevention program.

BACKGROUND: Teachers' implementation of evidence-based prevention programs in schools is inconsistent. Using data gathered from the national implementation among grade six students in The Bahamas of an evidence-based HIV intervention [Focus on Youth in the Caribbean (FOYC)], this study examines differences in the degree of implementation ("dose") and adherence to the core activities ("fidelity of implementation") by teachers according to theoretically and historically relevant teachers' characteristics, attitudes, and experiences pre-intervention and post-intervention. The relationship of implementation dose and implementation fidelity is assessed according to student outcomes. METHODS: Beginning in 2008, the Bahamian Ministry of Education (MOE) included FOYC in the grade six curriculum nationwide. Consistent with standard practice, teachers were offered MOE training workshops in FOYC prior to delivery. The MOE conducted an anonymous curricular assessment among the grade six students at the beginning and end of the school year. Teachers agreeing to participate in the research component were asked to complete a pre-implementation and post-implementation assessment of attitudes and prior experiences. RESULTS: Teachers taught 15.6 out of 30 core activities, 24 out of the 46 total activities, and 4.6 out of 8 sessions on average. Three teachers' implementation groups were identified: 1) High Implementation Group (31.7% of the teachers), characterized by high levels of implementation dose and fidelity of implementation; 2) Moderate Implementation Group (52.8%), showing moderate levels of implementation dose but high levels of fidelity of implementation; and 3) Low Implementation Group (15.6%), with low levels of implementation dose and fidelity of implementation. Low Implementation Group teachers compared to teachers in the two higher performing groups had less training in interactive teaching, limited prior exposure to the FOYC curriculum, incomplete attendance at FOYC training workshops, and low levels of comfort in teaching FOYC lessons. Students taught by teachers in the Low Implementation Group demonstrated poorer outcomes relevant to the four student outcomes (HIV/AIDS knowledge, preventive reproductive health skills, self-efficacy, and intention to use protection if they were to have sex). CONCLUSIONS: Both implementation dose and implementation fidelity are related to student outcomes. Teachers at risk for limited implementation can be identified pre-intervention, thus opening the possibility for focused pre-intervention training.

Assessing the effects of a complementary parent intervention and prior exposure to a preadolescent program of HIV risk reduction for mid-adolescents.

OBJECTIVES: We (1) evaluated the impact of an evidence-based HIV prevention program with and without a parent component among mid-adolescents living in the Caribbean and (2) determined the effect of prior receipt of a related intervention during preadolescence on intervention response. METHODS: A randomized, controlled 4-cell trial of a 10-session, theory-based HIV prevention intervention involving 2564 Bahamian grade-10 youths (some of whom had received a comparable intervention in grade 6) was conducted (2008-2011). Randomization occurred at the level of the classroom with follow-up at 6, 12, and 18 months after intervention. The 3 experimental conditions all included the youths' curriculum and either a youth-parent intervention emphasizing adolescent-parent communication, a parent-only goal-setting intervention, or no parent intervention. RESULTS: An intervention delivered to mid-adolescents in combination with a parent-adolescent sexual-risk communication intervention increased HIV/AIDS knowledge, condom-use skills, and self-efficacy and had a marginal effect on consistent condom use. Regardless of prior exposure to a similar intervention as preadolescents, youths benefited from receipt of the intervention. CONCLUSIONS: Preadolescents and mid-adolescents in HIV-affected countries should receive HIV prevention interventions that include parental participation.

Fidelity of implementation of an evidence-based HIV prevention program among Bahamian sixth grade students.

The Bahamian Ministry of Education has elected to implement at a national level in all Bahamian government grade six classes an evidence-based HIV prevention intervention [Focus on Youth in the Caribbean (FOYC)]. This study explores fidelity of implementation of the intervention, factors that may influence implementation fidelity, and the impact of variations in the implementation fidelity on student outcomes. Data were collected in the first wave of national implementation in 2011, involving 35 government primary schools and 110 teachers and 2,811 students. Structural equation modeling was performed to examine the relationships among factors which facilitated or impeded teachers' implementation of FOYC. Results indicate that teachers taught 16.3 out of 30 core activities, 24.9 out of 46 total activities, and 4.4 out of 8 sessions on average. The strongest predictor of implementation fidelity was teacher comfort level with the FOYC curriculum. Teachers who did not perceive the FOYC intervention to be important for their students or who had attended only part of a FOYC training workshop were more likely to change the curriculum. Increased duration of experience as a teacher (>10 years) was negatively associated with fidelity of implementation. Teacher's perception of the importance of the FOYC intervention and implementation fidelity had direct positive effects on students' HIV/AIDS knowledge, reproductive health skills, protective intentions, and self-efficacy. Youth did not appear to benefit from FOYC if two or fewer sessions were delivered. We concluded that an evidence-based HIV prevention intervention can be implemented at a national level. Prior training of teachers in the intervention curriculum, teacher perception of the importance of the intervention, and fewer years as a teacher are associated with implementation fidelity. Implementation fidelity is associated with improved student outcomes.

Sustained institutional effects of an evidence-based HIV prevention intervention.

Limited data are available as to what happens in institutions involved in behavioral intervention trials after the trial has ended. Specifically, do the trainers continue to administer the behavioral intervention that had been the focus of the trial? To address this question, we examined data in grade six schools before a year-long behavioral intervention had been delivered in some schools (and a year-long control condition in others) and data obtained again 6 and 7 years later in the same two sets of schools. Data were derived from the baseline surveys of two interventions: (1) national implementation of the evidence-based Focus on Youth in the Caribbean (FOYC) intervention in 2011; and (2) the randomized, controlled trial of the FOYC intervention in 2004/2005. Cross-sectional, longitudinal comparisons and random coefficient regression analysis were conducted to evaluate long-term intervention effects. Results indicate that grade six students in 2011 from schools in which the FOYC intervention had been implemented in 2004/2005 had a higher level of HIV/AIDS knowledge, increased reproductive health skills, increased self-efficacy regarding their ability to prevent HIV infection, and greater intention to use protection if they were to have sex compared to their counterparts from schools where no such training took place. We concluded that new cohorts of students benefited from the extensive training and/or experience in teaching the FOYC curriculum received by teachers, guidance counselors and administrators in schools which had delivered the FOYC intervention as part of a randomized trial several years earlier. The findings suggest that teachers who previously were trained to deliver the FOYC intervention may continue to teach at least some portions of the curriculum to subsequent classes of students attending these schools.

Preparing for national implementation of an evidence-based, effective HIV prevention program among bahamian sixth-grade students.

Using data from the preparatory phase prior to national implementation of an effective HIV prevention program (Focus on Youth in the Caribbean; FOYC) in all Bahamian government sixth-grade classes, we describe (1) actual FOYC implementation, (2) factors that influenced implementation, and (3) the relationship of implementation with intervention outcome. Six elementary schools (with 17 grade six classrooms) were selected to participate in the preparatory phase. The 17 teachers were invited to attend a training workshop, coordinate administration of questionnaires to the students, teach the 10 sessions of FOYC and complete self-assessment checklists. A total of 395 students submitted baseline and 311 students submitted year-end questionnaires. Thirteen teachers initiated FOYC; five completed all 10 sessions. Implementation of FOYC was not related to teacher FOYC workshop experience but did cluster by school. There were significant positive correlations between improved student knowledge of HIV/AIDS, protective health skills, perceived parental monitoring and reduced risk behaviours with the number of FOYC sessions delivered. Implementation was impeded by logistics issues, structural issues with the measures, and comfort-level issues, most of which can be addressed for national implementation. Degree of FOYC implementation is correlated with positive student outcomes.

National Implementation of an Evidence-Based HIV Prevention and Reproductive Health Program for Bahamian Youth.

A wide range of behavioral prevention interventions have been demonstrated through longitudinal, randomized controlled trials to reduce sexual risk behaviors. Many of these interventions have been made available at little cost for implementation on a public health scale. However, efforts to utilize such programs typically have been met with a range of problems to be addressed, leading to the recognition that new processes must be identified and integrated into the emerging field of implementation science. A randomized, controlled trial conducted among Bahamian grade six students attending fifteen elementary schools found the sexual risk-reduction intervention "Focus on Youth in the Caribbean (FOYC) and Caribbean Informed Parents and Children Together (CImPACT)" to be effective through three years of follow-up. Based on these results, the Bahamian Ministry of Education decided to implement FOYC-CImPACT throughout all government grade six classes in The Bahamas. This manuscript describes the considerations, approaches, and actions taken regarding national implementation of this evidence-based intervention. The implementation process included active data-gathering, observation and feedback components to inform subsequent intervention phases. This manuscript reviewed the success and challenges to date within this framework and described changes made to enable next stages of the national implementation effort.

Focus on youth in the Caribbean: beyond the numbers.

In the 1990s, an interdisciplinary group including pediatricians, anthropologists, health educators, psychologists, and statisticians developed and evaluated an HIV prevention intervention targeting early adolescents living in public housing developments in the USA. The intervention, "Focus on Kids," (FOK) was effective in reducing risk behaviors, intentions, and perceptions and ultimately was included in the Center for Disease Control's portfolio of effective adolescent programs, "Programs that Work." Learning about FOK and concerned about the need for a structured program to address high rates of teen pregnancy and risk for HIV, professionals from the Ministries of Health of The Bahamas approached the researchers about collaborating to develop a program for Bahamian youth. A partnership developed which has spanned over a decade and led to the development of an intervention program targeting Bahamian children in grade six, a 10-session adolescent HIV prevention program entitled "Focus on Youth in the Caribbean" (FOYC). Two programs including a video and parent discussion were developed for their parents. Caribbean Informed Parents and Children Together (CImPACT) emphasizes the importance of parent-child communication about sexuality and "Goal for It" (GFI) emphasizes the importance of planning ahead. The US-Bahamian team evaluated these interventions through a randomized, controlled 3-celled longitudinal trial (36 months follow-up) involving 15 elementary schools in The Bahamas. The programs have been shown to be effective. This article describes the context in which the epidemic occurred, events leading up to the collaboration and the issues, decisions, processes, and relationships that we have developed that have allowed it to succeed.