Validity Evidence for an Observational Fidelity Measure to Inform Scale-Up of Evidence-Based Interventions.

As evidence-based interventions are scaled, fidelity of implementation, and thus effectiveness, often wanes. Validated fidelity measures can improve researchers' ability to attribute outcomes to the intervention and help practitioners feel more confident in implementing the intervention as intended. We aim to provide a model for the validation of fidelity observation protocols to guide future research studying evidence-based interventions scaled-up under real-world conditions. We describe a process to build evidence of validity for items within the Session Review Form, an observational tool measuring fidelity to interactive drug prevention programs such as the Botvin LifeSkills Training program. Following Kane's (2006) assumptions framework requiring that validity evidence be built across four areas (scoring, generalizability, extrapolation, and decision), confirmatory factor analysis supported the hypothesized two-factor structure measuring quality of delivery (seven items assessing how well the material is implemented) and participant responsiveness (three items evaluating how well the intervention is received), and measurement invariance tests suggested the structure held across grade level and schools serving different student populations. These findings provide some evidence supporting the extrapolation assumption, though additional research is warranted since a more complete overall depiction of the validity argument is needed to evaluate fidelity measures.

Substance use prevention during adolescence: Study protocol for a large-scale cluster randomized trial of Botvin High School LifeSkills Training.

BACKGROUND: Concerns about the rise in adolescent vaping and cannabis use suggest the need for effective substance use prevention programs. Botvin LifeSkills Training (LST) has a strong evidence base at the middle school level for preventing or reducing tobacco use and related problems. A high school (grades 9-10) version of the LST program was also developed and shows promising initial evidence for reducing tobacco use in a single pilot study. However, the high school version of LST has not been sufficiently tested in an experimental trial, despite being widely implemented in high schools across the U.S. This paper outlines the study protocol for a large-scale cluster randomized trial of Botvin High School LST, with objectives of documenting the design of prospective research and promoting transparency. METHODS: A total of 60 high schools in Colorado and Ohio were randomized to the 10-session, teacher-led intervention group (n = 33 schools) or business-as-usual control group (n = 27 schools). Across two cohorts of schools, 9th-grade students complete self-report surveys at pretest, immediate posttest, 1-year follow-up, and 21-month follow-up. Primary outcomes are tobacco (nicotine) use and cannabis use. Secondary outcomes are alcohol use, illicit drug use, psychosocial behaviors (e.g., violence and mental health), and academic achievement. Intent-to-treat analyses will use multilevel modeling to estimate intervention effects across assessment points. CONCLUSION: This independent evaluation will help to determine whether the intervention is appropriate for large-scale adoption. This trial is preregistered with the Open Science Framework (https://osf.io/dnz5q/).

Sustainment of an Evidence-Based, Behavioral Health Curriculum in Schools.

The development of evidence-based interventions (EBIs) for the prevention of behavioral health problems is well supported. However, limited data exist on the sustainability of EBIs once initial support has ceased. The current study assessed, at 2 years after initial start-up support: (1) What percent of schools sustained EBI implementation? (2) To what degree did sustaining schools implement the EBI with fidelity? (3) What were the primary reasons for not sustaining the EBI? (4) What theoretical and contextual factors during initial start-up support predicted sustainment of the EBI? The study used process evaluation data from the dissemination of the Botvin LifeSkills Training (LST) middle school program in 158 school districts (including 419 schools). Fifty-one percent of districts sustained the EBI and most of these sustaining districts reported following key fidelity guidelines. Primary reasons for discontinuing centered around low teacher or administrative support and turnover. Significant predictors of sustaining LST were higher ratings of LST's complexity, benefit, and compatibility by teachers; more positive perceptions of organizational support from administrators; and smaller proportions of Black students. Results show that EBI sustainment and fidelity of implementation post-initial startup support are feasible, though likely not for all schools. Findings also suggest that cultivating support for the EBI among staff during start-up support may be helpful for sustainment and that social determinants of a school have a complex relationship with EBI sustainment. Future research should explore true causes of differences due to race/ethnicity as well as COVID-19 effects.

Influence of Classroom-Level Factors on Implementation Fidelity During Scale-up of Evidence-Based Interventions.

As evidence-based interventions (EBIs) become more widely disseminated, fidelity of implementation (FOI) often wanes. This study explores the association between FOI and malleable variables within classrooms that could be targeted to optimize resources without compromising FOI as school-based EBIs are disseminated across real-world settings. We utilized process evaluation data from a national dissemination project of the Botvin LifeSkills Training (LST) middle school program, a universal prevention intervention shown to reduce substance use. The sample included 1,626 teachers in 371 schools across 14 states. Hierarchical linear models examined the relationship between observational measures of implementation factors and three domains of fidelity (e.g., adherence, student responsiveness, and quality of delivery). Findings suggest that curriculum modifications, student misbehavior, and shortage of time to implement the LST middle school program were factors most associated with lower FOI. Class size, access to program materials, and whether LST was delivered in a traditional classroom setting that is well-suited for instruction (versus in a less structured environment such as the school cafeteria) are less predictive. In scale-up of classroom-based universal interventions targeting behavioral health outcomes, our findings indicate that carefully vetting modifications, supporting classroom management strategies, and ensuring sufficient class time for implementation of highly interactive EBIs such as LST are important considerations. Since changes to EBIs are inevitable, efforts are needed to guide facilitators in making adjustments that improve program fit without compromising the essential intervention activities deemed necessary to produce desired outcomes.

Does Training Modality Predict Fidelity of an Evidence-based Intervention Delivered in Schools?

Training prior to implementing evidence-based interventions (EBIs) is essential to reach high levels of fidelity. However, the time and cost of in-person training are often barriers to implementation. Online learning offers a potential solution, though few studies examine the relationship between online training and fidelity of implementation. This study explored whether teachers trained online have similar levels of adherence, dosage, quality of delivery, and student responsiveness compared to teachers trained in-person on the Botvin LifeSkills Training (LST) middle school program, a universal prevention intervention proven to reduce substance use and violence, as part of a national dissemination project. This study involved a sample of 989 LST teachers across 114 school districts, representing 296 schools in 14 states. All teachers were first trained in LST implementation between 2016 and 2019. Hierarchical linear models were used to assess relationships between training modality and the four fidelity outcomes. Online training was associated with lower ratings of quality of delivery compared to in-person training, but no significant associations existed between online training and adherence to the curriculum, dosage, or student responsiveness. Findings from this study generally indicate that online training builds competencies important for school-based EBI implementation, while also highlighting potential shortcomings related to quality of delivery. Ensuring the inclusion of experiential learning activities (e.g., practice delivering content, receiving feedback on delivery) may be key to quality of delivery as online trainings for facilitators of school-based EBIs evolve.

Individual and Social Network Correlates of Sexual Health Communication Among Youth Experiencing Homelessness.

Despite the potential for sexual health communication to be leveraged for HIV prevention among youth experiencing homelessness (YEH), there has yet to be a rigorous examination of individual and network or relational characteristics associated with sexual health communication in this group of young people. Cross-sectional survey and social network data from 1014 YEH aged 14-25 recruited in Los Angeles, California, were utilized to assess individual and network or relational characteristics associated with communication regarding condom use and HIV testing among YEH. Results suggest that social networks are key to understanding sexual health communication; YEH's engagement in sexual health communication was significantly related to the composition of their networks. To increase testing and decrease new HIV cases, a prudent strategy would be to train existing social network members (e.g., staff members, home-based peers, or partners) as agents of change in naturally occurring social networks of YEH.

Sociodemographic and substance use characteristics associated with typologies and composition of social support networks among youth experiencing homelessness in Los Angeles, USA.

Youth experiencing homelessness are a vulnerable population with increased behavioural health risks. Social networks are a consistent correlate of youths' substance use behaviours. However, less is known about the reciprocal relationships among these constructs. This study classified youth experiencing homelessness according to their social support network type (e.g. instrumental, emotional, service) and composition (e.g. family, peers, service staff) and linked their membership in these social network classes to sociodemographic and substance use characteristics. Four waves of cross-sectional data were collected between October 2011 and June 2013 from youth experiencing homelessness, ages 14-29, at three drop-in centres in Los Angeles, CA (N = 1,046). This study employed latent class analysis to identify subgroups of youth experiencing homelessness according to the type and composition of their social support networks. Multinomial logistic regression analyses were then conducted to identify the sociodemographic and substance use characteristics associated with social support network class membership. Five latent classes of youths' social support networks were identified: (a) high staff emotional and service support; (b) high home-based peer and family emotional, service and instrumental support; (c) moderate street- and home-based peer emotional support; (d) low or no support and (e) high home-based peer and family emotional and instrumental support. Multinomial logistic regression models indicated that race/ethnicity, gender, sexual orientation, literal homelessness, former foster care experience, depression, heroin and marijuana use were significant correlates of social support network class membership. Results indicate distinct classes of social support networks among youth experiencing homelessness, with certain sociodemographic and substance use characteristics implicated in youths' social networks.

"It seems kinda like a different language to us": Homeless youths' attitudes and experiences pertaining to condoms and contraceptives.

Homeless youth become pregnant or involved in pregnancies at high rates. There are many ways by which unintended pregnancies may be prevented, including the use of condoms and other contraceptives. However, there is a dearth of research regarding contraceptive use among this vulnerable youth population, and especially through lenses that consider homeless youths' diverse gender identities, expressions, and sexualities. This study qualitatively explores homeless youths' attitudes and experiences regarding condom and other contraceptive use. Data were obtained from interviews with 30 youth experiencing homelessness, ages 18-21. Youth reported inconsistent use of condoms and other contraceptives, which youth often attributed to their perceptions of contraceptive inaccessibility and exorbitant cost. Most youth also did not know where to obtain contraceptive information and services, and reported transportation barriers and fear of being stigmatized in health care settings, particularly in relation to their gender identities and sexualities. Findings suggest that reproductive and sexual health information and services are urgently needed by all homeless young people, and from low-barrier, non-judgmental, and empathetic sources.

Pregnancy and Childbearing Among Young Adults Who Experienced Foster Care.

This study explores rates of early pregnancy and parenthood among a sample of young adults ( N = 215), ages 18-22, with a history of foster care. The study also compares the educational attainment, financial resources, and homelessness experiences of young adults who became parents to those who did not. By age 21, 49% of the young women became pregnant, and 33% of young men reported getting someone pregnant. Over a quarter of participants experienced parenthood, which was associated with lower educational attainment, less employment, not having a checking or savings account, and a history of homelessness. Gender moderated the association between parenthood and employment such that males who were parents were more likely than female parents to be employed. Given that these young adults were at risk of early pregnancy and parenthood regardless of emancipation status and across several racial/ethnic groups, the results suggest a need for early pregnancy prevention efforts for all youth with child welfare involvement as well as improving resources and support for those who become young parents.

Pregnancy Attitudes and Contraceptive Use among Young Adults with Histories of Foster Care.

INTRODUCTION: This study examined pregnancy attitudes and contraceptive use among young adults with histories of foster care. METHODS: 209 female and male young adults, aged 18-22, with histories of foster care were interviewed about their intentions and feelings towards pregnancy. Respondents were then categorized as having pro-pregnancy (i.e., having positive pregnancy intentions and happy feelings about pregnancy), ambivalent (either intentions or happy feelings), or anti-pregnancy (no intentions and unhappy feelings) attitudes. Participants also reported their past-year contraceptive use, and the relationship between pregnancy attitudes and contraceptive use was subsequently explored for the overall sample, and by sex and history of pregnancy using Chi-square tests. RESULTS: Only 13.4% of participants had positive pregnancy intentions, though 41.9% reported that they would feel happy with a pregnancy. Over half (55.9%) of participants were anti-pregnancy, a third (32.8%) were ambivalent and 11.3% were pro-pregnancy. Compared to females, males were more likely to have positive pregnancy intentions (18.6% vs. 7.8%, p = .03) and to be pro-pregnancy (16.5% vs. 5.6%, p = .04). No differences in pregnancy attitudes were found as a function of pregnancy history. Consistent contraceptive use was significantly associated with pregnancy attitudes; 22.2% of pro-pregnancy participants reported consistent contraceptive use versus 52.9% of ambivalent and 62.2% of anti-pregnancy participants. DISCUSSION: In this exploratory study, few participants held pro-pregnancy attitudes and a high percentage of participants who were anti-pregnancy did not use contraception consistently. Although studies with larger samples examining this topic are needed, professionals should distinguish between young adults' intentions and feelings about pregnancy in an effort to better address contraceptive needs.