Validity Evidence for an Observational Fidelity Measure to Inform Scale-Up of Evidence-Based Interventions.

As evidence-based interventions are scaled, fidelity of implementation, and thus effectiveness, often wanes. Validated fidelity measures can improve researchers' ability to attribute outcomes to the intervention and help practitioners feel more confident in implementing the intervention as intended. We aim to provide a model for the validation of fidelity observation protocols to guide future research studying evidence-based interventions scaled-up under real-world conditions. We describe a process to build evidence of validity for items within the Session Review Form, an observational tool measuring fidelity to interactive drug prevention programs such as the Botvin LifeSkills Training program. Following Kane's (2006) assumptions framework requiring that validity evidence be built across four areas (scoring, generalizability, extrapolation, and decision), confirmatory factor analysis supported the hypothesized two-factor structure measuring quality of delivery (seven items assessing how well the material is implemented) and participant responsiveness (three items evaluating how well the intervention is received), and measurement invariance tests suggested the structure held across grade level and schools serving different student populations. These findings provide some evidence supporting the extrapolation assumption, though additional research is warranted since a more complete overall depiction of the validity argument is needed to evaluate fidelity measures.

A Pilot Phase I Trial of Allogeneic Umbilical Cord Tissue-Derived Mesenchymal Stromal Cells in Neonates With Hypoxic-Ischemic Encephalopathy.

Hypoxic ischemic encephalopathy (HIE) in neonates causes increased mortality and long-term morbidity in surviving babies. Hypothermia (HT) has improved outcomes, however, mortality remains high with ~half of surviving babies developing neurological impairment in their first years. We previously explored the use of autologous cord blood (CB) to determine if CB cells could lessen long-term damage to the brain. However, the feasibility of CB collection from sick neonates limited the utility of this approach. Allogeneic cord tissue mesenchymal stromal cells (hCT-MSC), cryopreserved and readily available, have been shown to ameliorate brain injury in animal models of HIE. We, therefore, conducted a pilot, phase I, clinical trial to test the safety and describe the preliminary efficacy of hCT-MSC in neonates with HIE. The study treated infants with moderate to severe HIE, treated with HT, with 1 or 2 doses of 2 million cells/kg/dose of hCT-MSC given intravenously. The babies were randomized to receive 1 or 2 doses with the first dose during HT and the second dose 2 months later. Babies were followed for survival and development with scoring of Bayley's at 12 postnatal months. Six neonates with moderate (4) or severe (2) HIE were enrolled. All received 1 dose of hCT-MSC during HT and 2 received a 2nd dose, 2 months later. hCT-MSC infusions were well tolerated although 5/6 babies developed low titer anti-HLA antibodies by 1 year of age. All babies survived, with average to low-average developmental assessment standard scores for ages between 12 and 17 postnatal months. Further study is warranted.

Substance use prevention during adolescence: Study protocol for a large-scale cluster randomized trial of Botvin High School LifeSkills Training.

BACKGROUND: Concerns about the rise in adolescent vaping and cannabis use suggest the need for effective substance use prevention programs. Botvin LifeSkills Training (LST) has a strong evidence base at the middle school level for preventing or reducing tobacco use and related problems. A high school (grades 9-10) version of the LST program was also developed and shows promising initial evidence for reducing tobacco use in a single pilot study. However, the high school version of LST has not been sufficiently tested in an experimental trial, despite being widely implemented in high schools across the U.S. This paper outlines the study protocol for a large-scale cluster randomized trial of Botvin High School LST, with objectives of documenting the design of prospective research and promoting transparency. METHODS: A total of 60 high schools in Colorado and Ohio were randomized to the 10-session, teacher-led intervention group (n = 33 schools) or business-as-usual control group (n = 27 schools). Across two cohorts of schools, 9th-grade students complete self-report surveys at pretest, immediate posttest, 1-year follow-up, and 21-month follow-up. Primary outcomes are tobacco (nicotine) use and cannabis use. Secondary outcomes are alcohol use, illicit drug use, psychosocial behaviors (e.g., violence and mental health), and academic achievement. Intent-to-treat analyses will use multilevel modeling to estimate intervention effects across assessment points. CONCLUSION: This independent evaluation will help to determine whether the intervention is appropriate for large-scale adoption. This trial is preregistered with the Open Science Framework (https://osf.io/dnz5q/).

Sustainment of an Evidence-Based, Behavioral Health Curriculum in Schools.

The development of evidence-based interventions (EBIs) for the prevention of behavioral health problems is well supported. However, limited data exist on the sustainability of EBIs once initial support has ceased. The current study assessed, at 2 years after initial start-up support: (1) What percent of schools sustained EBI implementation? (2) To what degree did sustaining schools implement the EBI with fidelity? (3) What were the primary reasons for not sustaining the EBI? (4) What theoretical and contextual factors during initial start-up support predicted sustainment of the EBI? The study used process evaluation data from the dissemination of the Botvin LifeSkills Training (LST) middle school program in 158 school districts (including 419 schools). Fifty-one percent of districts sustained the EBI and most of these sustaining districts reported following key fidelity guidelines. Primary reasons for discontinuing centered around low teacher or administrative support and turnover. Significant predictors of sustaining LST were higher ratings of LST's complexity, benefit, and compatibility by teachers; more positive perceptions of organizational support from administrators; and smaller proportions of Black students. Results show that EBI sustainment and fidelity of implementation post-initial startup support are feasible, though likely not for all schools. Findings also suggest that cultivating support for the EBI among staff during start-up support may be helpful for sustainment and that social determinants of a school have a complex relationship with EBI sustainment. Future research should explore true causes of differences due to race/ethnicity as well as COVID-19 effects.

Influence of Classroom-Level Factors on Implementation Fidelity During Scale-up of Evidence-Based Interventions.

As evidence-based interventions (EBIs) become more widely disseminated, fidelity of implementation (FOI) often wanes. This study explores the association between FOI and malleable variables within classrooms that could be targeted to optimize resources without compromising FOI as school-based EBIs are disseminated across real-world settings. We utilized process evaluation data from a national dissemination project of the Botvin LifeSkills Training (LST) middle school program, a universal prevention intervention shown to reduce substance use. The sample included 1,626 teachers in 371 schools across 14 states. Hierarchical linear models examined the relationship between observational measures of implementation factors and three domains of fidelity (e.g., adherence, student responsiveness, and quality of delivery). Findings suggest that curriculum modifications, student misbehavior, and shortage of time to implement the LST middle school program were factors most associated with lower FOI. Class size, access to program materials, and whether LST was delivered in a traditional classroom setting that is well-suited for instruction (versus in a less structured environment such as the school cafeteria) are less predictive. In scale-up of classroom-based universal interventions targeting behavioral health outcomes, our findings indicate that carefully vetting modifications, supporting classroom management strategies, and ensuring sufficient class time for implementation of highly interactive EBIs such as LST are important considerations. Since changes to EBIs are inevitable, efforts are needed to guide facilitators in making adjustments that improve program fit without compromising the essential intervention activities deemed necessary to produce desired outcomes.

Does Training Modality Predict Fidelity of an Evidence-based Intervention Delivered in Schools?

Training prior to implementing evidence-based interventions (EBIs) is essential to reach high levels of fidelity. However, the time and cost of in-person training are often barriers to implementation. Online learning offers a potential solution, though few studies examine the relationship between online training and fidelity of implementation. This study explored whether teachers trained online have similar levels of adherence, dosage, quality of delivery, and student responsiveness compared to teachers trained in-person on the Botvin LifeSkills Training (LST) middle school program, a universal prevention intervention proven to reduce substance use and violence, as part of a national dissemination project. This study involved a sample of 989 LST teachers across 114 school districts, representing 296 schools in 14 states. All teachers were first trained in LST implementation between 2016 and 2019. Hierarchical linear models were used to assess relationships between training modality and the four fidelity outcomes. Online training was associated with lower ratings of quality of delivery compared to in-person training, but no significant associations existed between online training and adherence to the curriculum, dosage, or student responsiveness. Findings from this study generally indicate that online training builds competencies important for school-based EBI implementation, while also highlighting potential shortcomings related to quality of delivery. Ensuring the inclusion of experiential learning activities (e.g., practice delivering content, receiving feedback on delivery) may be key to quality of delivery as online trainings for facilitators of school-based EBIs evolve.

Arabidopsis mRNA polyadenylation machinery: comprehensive analysis of protein-protein interactions and gene expression profiling.

BACKGROUND: The polyadenylation of mRNA is one of the critical processing steps during expression of almost all eukaryotic genes. It is tightly integrated with transcription, particularly its termination, as well as other RNA processing events, i.e. capping and splicing. The poly(A) tail protects the mRNA from unregulated degradation, and it is required for nuclear export and translation initiation. In recent years, it has been demonstrated that the polyadenylation process is also involved in the regulation of gene expression. The polyadenylation process requires two components, the cis-elements on the mRNA and a group of protein factors that recognize the cis-elements and produce the poly(A) tail. Here we report a comprehensive pairwise protein-protein interaction mapping and gene expression profiling of the mRNA polyadenylation protein machinery in Arabidopsis. RESULTS: By protein sequence homology search using human and yeast polyadenylation factors, we identified 28 proteins that may be components of Arabidopsis polyadenylation machinery. To elucidate the protein network and their functions, we first tested their protein-protein interaction profiles. Out of 320 pair-wise protein-protein interaction assays done using the yeast two-hybrid system, 56 (approximately 17%) showed positive interactions. 15 of these interactions were further tested, and all were confirmed by co-immunoprecipitation and/or in vitro co-purification. These interactions organize into three distinct hubs involving the Arabidopsis polyadenylation factors. These hubs are centered around AtCPSF100, AtCLPS, and AtFIPS. The first two are similar to complexes seen in mammals, while the third one stands out as unique to plants. When comparing the gene expression profiles extracted from publicly available microarray datasets, some of the polyadenylation related genes showed tissue-specific expression, suggestive of potential different polyadenylation complex configurations. CONCLUSION: An extensive protein network was revealed for plant polyadenylation machinery, in which all predicted proteins were found to be connecting to the complex. The gene expression profiles are indicative that specialized sub-complexes may be formed to carry out targeted processing of mRNA in different developmental stages and tissue types. These results offer a roadmap for further functional characterizations of the protein factors, and for building models when testing the genetic contributions of these genes in plant growth and development.