The Role of Implementation Climate in Moderating Educator Use of Evidence-Based Practices and Outcomes for Autistic Students.

Ensuring effective use of evidence-based practice (EBP) for autism in schools is imperative due to the significantly increasing number of autistic students receiving school services each year. High-quality EBP use has proven challenging in schools. Research indicates implementation climate, or how EBP are supported, rewarded, and valued, and EBP resources are related to successful implementation. However, limited understanding of system-level contextual factors that impact EBP implementation for school-based providers makes development of appropriate implementation supports challenging. Understanding these factors is crucial for selecting and tailoring implementation strategies to support EBP scale up. In this observational study, California school-based providers (n = 1084) completed surveys related to implementation climate, leadership, autism experience and EBP implementation (use, competence, knowledge). Student outcomes included state level academic and behavioral indicators. Using an implementation science framework (Aarons et al., in Administration and Policy in Mental Health and Mental Health Services Research 38:4-23, 2011) and multilevel modeling, we examined the relationship between EBP Implementation and student outcomes and the moderation effects of provider and district level factors. Higher implementation climate predicted better EBP implementation outcomes, and proved more impactful when provider hands-on autism experience was low. Greater EBP resources predicted a higher percentage of students who met math standards only when district poverty level was high. Our findings suggested moderating effects on EBP implementation from both provider and system level factors. Implementation climate and resources may be especially key in addressing equity issues related to high poverty schools in which teachers often have less autism experience.

Implementation readiness for evidence-based autism practices in school systems.

Background: The increase in the number of autistic children being identified has led to increased demand on public schools to provide high-quality services. Effectively scaling up evidence-based practice (EBP) use for autistic students is challenging, given the complicated organization of special education. Teachers have significant challenges implementing autism EBP with fidelity. Factors such as implementation leadership and climate and attitudes toward EBP are linked to successful EBP use and may vary at different levels of the education system. Examining mechanisms of successful implementation is a critical step to support scale-up. Method: In this observational study, conducted from September 2018 to March 2020, California school personnel (n = 2273) at multiple levels of the system completed surveys related to implementation climate, leadership, and attitudes toward EBP. Data were collected throughout California at the Special Education Local Plan Areas, County Office of Education, and district and school levels from educators and administrators working in public schools supporting autistic students. Multi-level modeling was conducted to characterize implementation readiness. Results: Overall, implementation climate and leadership scores are low across levels with regional levels rated more positively than districts or schools. Attitudes toward EBP were moderate, with those working in schools having the poorest ratings and specialists/trainers and related service providers (e.g., speech-language pathologists) having the highest ratings. Conclusions: Outcomes provide a unique opportunity to compare implementation factors across organizational levels with a large, statewide sample. These data provide guidance for developing implementation interventions at multiple levels of the education system to increase readiness for effective scale-up of autism EBP in schools. Personnel and leaders at different organizational levels may need differentiated training targeting improved implementation climate and leadership. Personnel within districts and schools may experience a particular benefit from leadership support for EBP implementation.

The increase in the number of autistic children being identified in schools is increasing. To address this, schools are trying to do a better job of using high-quality practices based on research. However, teachers have had difficulty using research-based strategies for autistic students the way the manuals indicate they should be used. This might be due to the complexity of the strategies or limited support from special education leadership and infrastructure. Research shows that leaders can be very important in helping teachers use effective strategies. Over 2200 school personnel in California, including administrators, professional development providers, teachers, and paraprofessionals completed surveys asking about how their leaders, schools, districts, and regions supported the use of research practices for autistic students. Overall, limited support is provided in special education, with regional agencies providing more support than districts or schools. These data suggest that school and district leaders need training in how to support educators in using autism-specific strategies.

Practice-driven research for statewide scale up: Implementation outcomes of the California Autism Professional Training and Information Network.

LAY ABSTRACT: Supporting use of evidence-based practice in public service programs for autistic individuals is critical. The California Autism Professional Training and Information Network (CAPTAIN) brings together best practices from intervention and implementation research to support scale up of autism services. The current study was designed to evaluate the impact of CAPTAIN on provider-level outcomes including attitude toward, knowledge, fidelity, and use of autism EBPs and overall classroom quality. Overall, results indicated variability across measures, with some significant differences between CAPTAIN-trained and non-CAPTAIN-trained providers. These preliminary findings show promise for the efficacy of the CAPTAIN model to increase dissemination and implementation of EBP at the classroom level.

Statewide Interagency Collaboration to Support Evidence-based Practice Scale Up: The California Autism Professional Training and Information Network (CAPTAIN).

The California Autism Professional Training and Information Network (CAPTAIN) is a statewide interagency collaboration with the goal of scaling up use of evidence-based practices (EBPs) for individuals with autism spectrum disorder (ASD). CAPTAIN began as a clinical initiative then further developed under the influence of implementation science methodology. The Exploration, Preparation, Implementation and Sustainment framework (EPIS) has impacted targeted strategy use for this statewide scale up of EBPs by informing the development of key partnerships, implementation goals, and collaborative processes within CAPTAIN. Currently, CAPTAIN has over 407 members representing 140 school and community agencies who provide training and coaching in EBP and meet regularly with regional teams. Outcome data indicate 51.9% of members provide training and coaching to more than three direct service providers/programs per year. Primary barriers to implementation of EBP were time for training (25.6%), lack of substitute teachers (16.5%), and staff lacking foundational skills (11.5%). Facilitators of implementation and sustainment of the CAPTAIN model include active participation in effective dissemination practices, creative funding and leveraging of local resources, development of the regional collaboratives with active membership, member commitment to EBP for ASD, and use of implementation science to identify and overcome barriers. The purpose of this paper is to highlight CAPTAIN as a model for statewide scale up of EBP in schools as well as other community agencies. Although these efforts have focused on EBP for ASD, the concepts, partnerships and procedures will likely be transferable to other focal issues and may be generalized across service sectors.

Correction to: Exploring multi-level system factors facilitating educator training and implementation of evidence-based practices (EBP): a study protocol.

After publication of the original article [1] it was brought to our attention that author Elizabeth McGhee Hassrick was erroneously included as Elizabeth McGee Hassrick. The correct spelling of the author's name is included in the author list of this erratum.

Exploring multi-level system factors facilitating educator training and implementation of evidence-based practices (EBP): a study protocol.

BACKGROUND: This study examines how system-wide (i.e., region, district, and school) mechanisms such as leadership support, training requirements, structure, collaboration, and education affect the use of evidence-based practices (EBPs) in schools and how this affects the outcomes for students with autism spectrum disorder (ASD). Despite growing evidence for the positive effects of EBPs for ASD, these practices are not consistently or effectively used in schools. Although special education programs are mandated to use EBPs, there are very few evidence-based methods for selecting, implementing, and sustaining EBPs. Research focuses primarily on teacher training, without attention to contextual factors (e.g., implementation climate, attitudes toward EBPs, resource allocation, and social networks) that may impact outcomes. Using an implementation science framework, this project will prospectively examine relations between system-wide factors and teachers' use of EBPs and student education outcomes. METHODS/DESIGN: Survey data will be collected from approximately 85 regional special education directors, 170 regional program specialists, 265 district special education directors, 265 behavior specialists, 925 school principals, 3538 special education teachers, and 2700 paraprofessionals. Administrative data for the students with ASD served by participating teachers will be examined. A total of 79 regional-, district-, and school-level personnel will also participate in social network interviews. Mixed methods, including surveys, administrative data, and observational checklists, will be used to gather in-depth information about system-wide malleable factors that relate to positive teacher implementation of EBPs and student outcomes. Multi-level modeling will be used to assess system-wide malleable factors related to EBP implementation which will be linked to the trainer, teacher, and student outcomes and examined based on moderators (e.g., district size, Special Education Local Plan Area structure, teachers' ASD experience). Finally, a dynamic social network approach will be used to map EBP-related connectivity across all levels of the system for selected regions. Dynamic network analysis will be used to gauge the degree to which and ways that EBP trainings, resources, and interventions are shared (or not shared) among school staff. DISCUSSION: Results are expected to inform the development of system-wide interventions to improve the school-based implementation of EBPs for students with ASD.