Process Evaluation of a Scaled-Up School-Based Physical Activity Program for Adolescents: Physical Activity 4 Everyone.

BACKGROUND: Physical Activity 4 Everyone (PA4E1) is a whole-school physical activity program, with demonstrated efficacy (2012-2014). PA4E1 was adapted (scaled-up) and tested in a scale-up trial (2017-2020). This process evaluation study of the scale-up trial had 2 aims. First, to describe the acceptability, appropriateness, and feasibility of PA4E1 in the scale-up trial, from the perspective of school staff involved in the program management and delivery. Second, to generate themes that may explain school staff assessments of acceptability, appropriateness, and feasibility. METHODS: Data were collected at various time points throughout the 2-year implementation phase. Online surveys were collected from In-School Champions, Head Physical Education teachers, Principals, and Physical Education teachers (quantitative data). Focus groups and interviews were conducted with In-School Champions, Principals, and Physical Education teachers (qualitative data). Existing published data on website engagement, adaptations, modifications, and the scale-up trial primary outcome (implementation of physical activity practices) were triangulated with the quantitative and qualitative during analysis, to generate themes. RESULTS: School staff delivering PA4E1 reported it was highly acceptable, appropriate, and feasible. Seven themes were generated relating to acceptability, appropriateness, and feasibility. The themes related to how the program was funded, the delivery modes of implementation support, the identification of easy-wins, the recruitment of the right in-school champion, facilitating principal buy-in, mitigating the impact of school staff turnover, and engaging the whole school. CONCLUSIONS: Recommendations are made to inform future adaptations for PA4E1 and potentially school-based physical activity programs more generally. The findings may inform future scalability assessments of the suitability of programs for scale-up.

Scale-up of the Physical Activity 4 Everyone (PA4E1) intervention in secondary schools: 24-month implementation and cost outcomes from a cluster randomised controlled trial.

BACKGROUND: Physical Activity 4 Everyone (PA4E1) is an evidence-based program effective at increasing adolescent physical activity (PA) and improving weight status. This study aimed to determine a) the effectiveness of an adapted implementation intervention to scale-up PA4E1 at 24-month follow-up, b) fidelity and reach, and c) the cost and cost-effectiveness of the implementation support intervention. METHODS: A cluster randomised controlled trial using a type III hybrid implementation-effectiveness design in 49 lower socio-economic secondary schools, randomised to a program (n = 24) or control group (n = 25). An adapted implementation intervention consisting of seven strategies was developed to support schools to implement PA4E1 over 24-months. The primary outcome was the proportion of schools implementing at least four of the 7 PA practices, assessed via computer assisted telephone interviews (CATI) with Head Physical Education Teachers. Secondary outcomes included the mean number of PA practices implemented, fidelity and reach, cost and cost-effectiveness. Logistic regression models assessed program effects. RESULTS: At baseline, no schools implemented four of the 7 PA practices. At 24-months, significantly more schools in the program group (16/23, 69.6%) implemented at least four of the 7 PA practices than the control group (0/25, 0%) (p < 0.001). At 24-months, program schools were implementing an average of 3.6 more practices than control schools (4.1 (1.7) vs. 0.5 (0.8), respectively) (P < 0.001). Fidelity and reach of the implementation intervention were high (> 75%). The total cost of the program was $415,112 AUD (2018) ($17,296 per school; $117.30 per student). CONCLUSIONS: The adapted implementation intervention provides policy makers and researchers with an effective and potentially cost-effective model for scaling-up the delivery of PA4E1 in secondary schools. Further assessment of sustainability is warranted. TRIAL REGISTRATION: Australian New Zealand Clinical Trials Registry ACTRN12617000681358 prospectively registered 12th May 2017.

Evaluating Digital Program Support for the Physical Activity 4 Everyone (PA4E1) School Program: Mixed Methods Study.

BACKGROUND: Effectively scaled-up physical activity interventions are urgently needed to address the high prevalence of physical inactivity. To facilitate scale-up of an efficacious school-based physical activity program (Physical Activity 4 Everyone [PA4E1]), provision of implementation support to physical education (PE) teachers was adapted from face-to-face and paper-based delivery modes to partial delivery via a website. A lack of engagement (usage and subjective experience) with digital delivery modes, including websites, may in part explain the typical reduction in effectiveness of scaled-up interventions that use digital delivery modes. A process evaluation focused on the PA4E1 website was undertaken. OBJECTIVE: The 2 objectives were to (1) describe the usage of the PA4E1 program website by in-school champions (PE teachers leading the program within their schools) and PE teachers using quantitative methods; (2) examine the usage, subjective experience, and usability of the PA4E1 program website from the perspective of in-school champions using mixed methods. METHODS: The first objective used website usage data collected across all users (n=273) throughout the 9 school terms of the PA4E1 implementation support. The 4 usage measures were sessions, page views, average session duration, and downloads. Descriptive statistics were calculated and explored across the duration of the 26-month program. The second objective used mixed methods, triangulating data from the first objective with data from a think-aloud survey and usability test completed by in-school champions (n=13) at 12 months. Qualitative data were analyzed thematically alongside descriptive statistics from the quantitative data in a triangulation matrix, generating cross-cutting themes using the "following a thread" approach. RESULTS: For the first objective, in-school champions averaged 48.0 sessions per user, PE teachers 5.8 sessions. PE teacher sessions were of longer duration (10.5 vs 7.6 minutes) and included more page views (5.4 vs 3.4). The results from the mixed methods analysis for the second objective found 9 themes and 2 meta-themes. The first meta-theme indicated that the website was an acceptable and appropriate delivery mode, and usability of the website was high. The second meta-theme found that the website content was acceptable and appropriate, and identified specific suggestions for improvement. CONCLUSIONS: Digital health interventions targeting physical activity often experience issues of lack of user engagement. By contrast, the findings from both the quantitative and mixed methods analyses indicate high usage and overall acceptability and appropriateness of the PA4E1 website to school teachers. The findings support the value of the website within a multidelivery mode implementation intervention to support schools to implement physical activity promoting practices. The analysis identified suggested intervention refinements, which may be adopted for future iterations and further scale-up of the PA4E1 program. TRIAL REGISTRATION: Australian New Zealand Clinical Trials Registry ACTRN12617000681358; https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=372870.

Extent, Type and Reasons for Adaptation and Modification When Scaling-Up an Effective Physical Activity Program: Physical Activity 4 Everyone (PA4E1).

Background: Few studies have described the extent, type and reasons for making changes to a program prior to and during its delivery using a consistent taxonomy. Physical Activity 4 Everyone (PA4E1) is a secondary school physical activity program that was scaled-up for delivery to a greater number of schools. We aimed to describe the extent, type and reasons for changes to the PA4E1 program (the evidence-based physical activity practices, implementation support strategies and evaluation methods) made before its delivery at scale (adaptations) and during its delivery in a scale-up trial (modifications). Methods: The Framework for Reporting Adaptations and Modifications-Enhanced (FRAME) was used to describe adaptations (planned and made prior to the scale-up trial) and modifications (made during the conduct of the trial). A list of adaptations was generated from a comparison of the efficacy and scale-up trials via published PA4E1 protocols, trial registrations and information provided by trial investigators. Monthly trial team meetings tracked and coded modifications in "real-time" during the conduct of the scale-up trial. The extent, type and reasons for both adaptations and modifications were summarized descriptively. Results: In total, 20 adaptations and 20 modifications were identified, these were to physical activity practices (n = 8; n = 3), implementation support strategies (n = 6; n = 16) and evaluation methods (n = 6, n = 1), respectively. Few adaptations were "fidelity inconsistent" (n = 2), made "unsystematically" (n = 1) and proposed to have a "negative" impact on the effectiveness of the program (n = 1). Reasons for the adaptations varied. Of the 20 modifications, all were "fidelity consistent" and the majority were made "proactively" (n = 12), though most were "unsystematic" (n = 18). Fifteen of the modifications were thought to have a "positive" impact on program effectiveness. The main reason for modification was the "available resources" (n = 14) of the PA4E1 Implementation Team. Conclusions: Adaptations and modifications to public health programs are common. Modifications have the potential to impact the implementation and effectiveness of programs. Our findings underscore the importance of comprehensive reporting of the extent, type and reasons for modifications as part of process evaluations, as this data may be important to the interpretation of trial findings. Clinical Trial Registration: https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=372870, Identifier ACTRN12617000681358.

Scale-up of the Physical Activity 4 Everyone (PA4E1) intervention in secondary schools: 12-month implementation outcomes from a cluster randomized controlled trial.

BACKGROUND: 'Physical Activity 4 Everyone' (PA4E1) was an efficacious multi-component school-based physical activity (PA) program targeting adolescents. PA4E1 has seven PA practices. It is essential to scale-up, evaluate effectiveness and assess implementation of such programs. Therefore, the aim is to assess the impact of implementation support on school practice uptake of the PA4E1 program at 12 and 24 months. METHODS: A cluster randomised controlled trial, utilising a type III hybrid implementation-effectiveness design, was conducted in 49 randomly selected disadvantaged Australian Government and Catholic secondary schools. A blinded statistician randomly allocated schools to a usual practice control (n = 25) or the PA4E1 program group (n = 24), with the latter receiving seven implementation support strategies to support school PA practice uptake of the seven practices retained from the efficacy trial. The primary outcome was the proportion of schools adopting at least four of the seven practices, assessed via telephone surveys with Head Physical Education Teachers and analysed using exact logistic regression modelling. This paper reports the 12-month outcomes. RESULTS: Schools were recruited from May to November 2017. At baseline, no schools implemented four of the seven practices. At 12 months significantly more schools in the program group had implemented four of the seven practices (16/24, 66.7%) than the control group (1/25, 4%) (OR = 33.0[4.15-1556.4], p < 0.001). The program group implemented on average 3.2 (2.5-3.9) more practices than the control group (p < 0.001, mean 3.9 (SD 1.5) vs 0.7 (1.0)). Fidelity and reach of the implementation support intervention were high (both > 80%). CONCLUSIONS: Through the application of multiple implementation support strategies, secondary schools were able to overcome commonly known barriers to implement evidence based school PA practices. As such practices have been shown to result in an increase in adolescent PA and improvements in weight status, policy makers and practitioners responsible for advocating PA in schools should consider this implementation approach more broadly when working with schools. Follow-up is required to determine whether practice implementation is sustained. TRIAL REGISTRATION: Australian New Zealand Clinical Trials Registry ACTRN12617000681358 registered 12th May 2017.

Association of staff behaviors and afterschool program features to physical activity: findings from Movin' After School.

BACKGROUND: Children's achievement of recommendations for moderate-to-vigorous physical activity (MVPA) in afterschool programs (ASP) is complex. It is unclear what elements of the ASP environment influence children's physical activity (PA). The purpose of this study was to determine the relationship of staff behaviors and ASP features (eg, organized activity, recreational equipment) to MVPA participation in youth attending ASPs. METHODS: Data were collected in 12 ASPs in the Midwest. Staff behavior and child PA was measured using the System for Observing Play and Leisure Activity in Youth. The percentage of children's MVPA was examined in relation to staff behaviors and ASP features. RESULTS: Increases in MVPA were observed when staff were directly engaged in PA, verbally promoted MVPA, and when PA was organized and equipment was present. When 3 or more of these characteristics were present, the proportion of children engaged in MVPA increased by 25%-30%. Conversely, MVPA levels decreased when these characteristics were absent and when staff were attending to other ASP duties or were supervising. CONCLUSION: This study provides evidence about the specific staff behaviors that may influence higher proportions of youth being active during ASP and implies specific skills that need to be incorporated into ASP staff training.