Five-year fidelity assessment of an evidence-based parenting program (GenerationPMTO): inter-rater reliability following international implementation.

BACKGROUND: Implementing evidence-based programs in community service settings introduces the challenge of ensuring sustained fidelity to the original program. We employ a fidelity measure based on direct observation of practitioners' competence and adherence to the evidence-based parenting program (EBPP) GenerationPMTO following installation in national and international sites. Fidelity monitoring is crucial, especially when the program purveyor transfers administration of the program to the community as was done in this case. In previous studies, the Fidelity of Implementation rating system (FIMP) was used to evaluate practitioners' fidelity to the GenerationPMTO intervention in six countries following implementation showing high levels of adherence up to 17 years post certification. Other studies showed FIMP to have predictive validity. The present study provides inter-rater reliability data for this fidelity tool across teams of the purveyor, Implementation Sciences International, Inc./ISII, and national and international sites over a five-year period. METHODS: Data assess inter-rater reliability in terms of percent agreement and intraclass correlation (ICC) for the purveyor's two fidelity teams and the fidelity teams in seven implementation sites. RESULTS: Results report stable good to excellent levels of inter-rater reliability and ICCs as well as good attendance at fidelity meetings for all fidelity teams. CONCLUSIONS: This observational method of assessing fidelity post implementation is a promising approach to enable EBPPs to be transferred safely from purveyors to communities while maintaining reliable fidelity to the intervention.

Investigating active ingredients of the GenerationPMTO intervention: Predictors of postintervention change trajectories in parenting practices.

Empirically determining the active ingredients of evidence-based parenting interventions is a promising means for strengthening interventions and enhancing their public health impact. This study aimed to determine which distinct ingredients of the GenerationPMTO (GenPMTO) intervention were associated with subsequent changes in parenting practices. Using a sample of 153 participants randomly assigned to the GenPMTO condition, we employed multilevel modeling to identify intervention ingredients empirically linked with change trajectories in parenting practices observed across the 2 years following intervention exposure. Coercive parenting and positive parenting outcomes were examined. Study results indicated that emotion regulation, effective communication, problem solving, and monitoring each demonstrated a significant pattern of findings for coercive parenting. Differential exposure to each of these ingredients significantly predicted the level of coercive parenting immediately postintervention and/or trajectories of change in coercive parenting across the subsequent 2-year period, controlling for coercive parenting at baseline. No significant predictors were found for positive parenting trajectories. Our findings suggest four components as active ingredients of the GenPMTO intervention for coercive parenting. Identification of these active ingredients may lead to strengthening future iterations of GenPMTO by expanding the set of core components specified in the model, which may further improve public health benefits. Implications for further understanding change stemming from evidence-based parenting interventions are also discussed. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Development and Evaluation of a Component Level Implementation Fidelity Rating System for the GenerationPMTO Intervention.

Empirically determining the components of evidence-based interventions contributing to positive change is a crucial, yet understudied area of research. In support of this aim, we describe the development and evaluation of an observational rating system for measuring fidelity to specific components of the evidence-based GenerationPMTO parenting intervention. A five-step process was employed to systematically develop the rating system, which included consultation with the intervention developer and input from additional GenerationPMTO experts. The rating system was then tested using 247 h of video data from 184 parenting group intervention sessions. Study findings support the psychometric properties of the new measure with regard to item performance, reliability (i.e., inter-rater reliability of items, dimensionality of components, internal consistency of component scales), and validity (i.e., content validity, convergent validity, discriminant validity, and predictive validity of the component scales) for seven of the eight scales evaluated. The seven components include clear directions, skill encouragement, emotion regulation, limit setting, effective communication, problem solving, and monitoring. Data did not support the psychometric properties of the positive involvement scale. Overall, the ability to assess component-specific fidelity allows for a more nuanced examination of change processes, with meaningful implications for research and practice.

Behavioral Observations in Northern UGANDA: Development of a Coding System to Assess Mother-Child Interactions in a Post-war Society.

There is growing interest in causes and consequences of disruptions in parent-child relationships in post-war environments. Recent studies mainly relied on self-reports to gain information about family dynamics following war exposure. Considering the limitations of self-report measures, we see the need for an in-depth examination of post-conflict parenting based on observational and quantitative data. The aim of the present study was the development of a coding system for a culturally bound description of parent-child interactions in northern Uganda, where virtually the entire population has been severely affected by 20 years of civil war. Interactions of 101 mothers and their 6- to 12-year-old children were observed during a structured interaction task (problem solving discussion). Foundation for the development of the coding system was the Family and Peer Process Code (FPP code). The cultural adaptation of the FPP code was based on in-depth qualitative analyses of the problem solving task, including a combination of inductive and deductive latent content analyses of textual data and videotapes, member checking and consultations of experts in the field of behavioral observations. The final coding system consists of 35 exhaustive and mutually exclusive content codes including codes for verbal, vocal, and compliance behavior as well as 14 affect codes. Findings indicate that the assessment of behavioral observations in post-conflict settings provides unique insights into culture- and context-specific interaction patterns and may be critical for the development and evaluation of parenting interventions.

Scaling up an Empirically Supported Intervention with Long-Term Outcomes: the Nationwide Implementation of GenerationPMTO in Norway.

Effective implementation outcomes are necessary preconditions for effective service and positive treatment outcomes for children with behavioral problems. The aim of this study is to assess outcomes of the transfer of the empirically supported intervention GenerationPMTO from the developer in the USA to a nationwide implementation in Norway. Adoption, sustainability, reach, and fidelity are tested across seven generations of therapists in Norway. Participants in the study were 521 therapists who began training in the program. The developer's team trained the first generation (G1) and the Norwegian team trained the next six generations (G2-G7). The mean rate of certification was 94.2% (n = 491). Intervention fidelity was assessed from 1964 video recordings of intervention sessions submitted for certification evaluation using the observation-based measure Fidelity of Implementation Rating System (FIMP). A small but significant drop in fidelity scores was previously observed from G1 to G2; however, fidelity scores recovered at or above G1 scores for G3 Forgatch and DeGarmo (Prevention Science 12, 235-246, 2011). Twenty years since the inception of implementation, 314 certified therapists practice the model today, a retention rate of 64%. The outcomes show sustained fidelity scores across seven generations, increasing heterogeneity among therapists trained, and a shift of focus in the target population from clinical to primary services. The present study contributes to the field with the systematic evaluation of outcomes for the full transfer implementation approach with continuing adoption and sustainability, increasing reach and sustained intervention fidelity across several generations of practitioners.

Teaching GenerationPMTO, an evidence-based parent intervention, in a university setting using a blended learning strategy.

BACKGROUND: Despite the large number of evidence-based practices (EBPs) ready for implementation, they are the exception in usual care, especially for ethnic minority patients, who may not have access to trained health professionals. Providing EBP training as part of a graduate curriculum could help build the pipeline of professionals to provide quality care. METHODS: We conducted a before-after study to determine whether we could implement a blended learning strategy (BL; i.e., in vivo and online training) to teach an EBP in university settings. Feasibility in this pilot was operationalized as knowledge acquisition, satisfaction, fidelity, acceptability, and usability. Using GenerationPMTO as the EBP, our aim was to train graduate students enrolled in Psychology, Social Work, and Family Therapy programs in the EBP in one academic year. Two therapists from a community agency were also students in this pilot. A total of 13 students from five universities were trained in the intervention. Adaptations were made to the intervention and training strategy to optimize training fidelity. Focus groups were conducted with the students to capture their perspective about the training. RESULTS: Students demonstrated significant knowledge acquisition from baseline (Mean = 61.79, SD = 11.18) to training completion (Mean = 85.27, SD = 5.08, mean difference = - 23.48, 95% CI = - 29.62, - 17.34). They also reported satisfaction with the BL format, as measured by teaching evaluations at the end of the course. Instructors received acceptable fidelity scores (range of 7-9 in a 9-point scale). Qualitative findings from focus groups showed support for acceptability and usability of BL training. CONCLUSIONS: BL training in university settings can be conducted with fidelity when provided by appropriately trained instructors. BL that integrates EBP and adaptations may be uniquely applicable for training providers in low-resource and ethnically diverse settings. The BL enhanced knowledge of GenerationPMTO was acceptable and usable to students, and was delivered with high instructor fidelity to the training model.

Implementing an Evidence-Based Intervention for Children in Europe: Evaluating the Full-Transfer Approach.

This study evaluated the implementation outcomes of GenerationPMTO, an evidence-based parenting intervention for child and adolescent behavior problems, in three European countries. The implementation approach was full transfer, in which purveyors train a first generation (G1) of practitioners; adopting sites assume oversight, training, certification, and fidelity assessment for subsequent generations (Forgatch & DeGarmo, 2011; Forgatch & Gewirtz, 2017). Three hundred therapists participated in trainings in GenerationPMTO in Iceland, Denmark, and the Netherlands. Data are from the implementation's initiation in each country through 2016, resulting in 6 generations in Iceland, 8 in Denmark, and 4 in the Netherlands. Therapist fidelity was measured at certification with an observation-based tool, the Fidelity of Implementation Rating System (Knutson, Forgatch, Rains, & Sigmarsdóttir, 2009). Candidates in all generations achieved fidelity scores at or above the required standard. Certification fidelity scores were evaluated for G1 candidates, who were trained by the purveyor, and subsequent generations trained by the adopting implementation site. In each country, certification fidelity scores declined for G2 candidates compared with G1 and recovered to G1 levels for subsequent generations, partially replicating findings from a previous Norwegian study (Forgatch & DeGarmo, 2011). Recovery to G1 levels of fidelity scores was obtained in Iceland and the Netherlands by G3; in Denmark, the recovery was obtained by G5. The mean percentage of certification in each country was more than 80%; approximately 70% of certified therapists remained active in 2017. Findings support full transfer as an effective implementation approach with long-term sustainability and fidelity.

Feasibility of Implementation of a Parenting Intervention with Karen Refugees Resettled from Burma.

Parents and children exposed to war and relocation have high rates of negative relational and mental health outcomes. This study tested the feasibility of implementing an adapted evidence-based parenting intervention for contexts of trauma and relocation stress. Eleven Karen refugee caregivers from Burma participated in the intervention. Participants and a focal child completed ethnographic interviews as well as structured assessments at baseline and follow-up. Caregivers reported changes in their teaching, directions, emotional regulation, discipline, and child compliance. Children reported changes in these areas and in positive parent involvement. Caregivers reported higher mental health distress immediately after the intervention, potentially due to increased awareness. Researchers made personalized referrals for counseling services as needed. Children reported a decrease in mental health symptoms.

Preparing the Field for Feasibility Testing of a Parenting Intervention for War-Affected Mothers in Northern Uganda.

In this article, we discuss the successful implementation of an adapted evidence-based parenting intervention for families affected by two decades of war in Northern Uganda. The adaptation and adoption of such interventions to support mental health and family functioning is widely endorsed by prevention scientists and considered a priority in global mental health. The preparation and early adoption phases of engaging with a highly vulnerable community affected by war trauma are documented in this paper along with a discussion of the steps taken to adapt a parenting intervention for cultural and contextual fit. This study is a component of an overall program of research aimed at reducing the long-term negative effects of war on parenting practices and childhood outcomes, which have considerable implications for preventing mental, neurological, and substance-use disorders. The processes described here cover a 4-year period culminating in the implementation of the nine-session Enhancing Family Connection intervention piloted with a group of 14 mothers. The lessons in cultural adaptation have been valuable and the feasibility results promising for further testing the intervention.

The Oregon Model of Behavior Family Therapy: From Intervention Design to Promoting Large-Scale System Change.

This paper reviews the evolution of the Oregon model of family behavior therapy over the past four decades. Inspired by basic research on family interaction and innovation in behavior change theory, a set of intervention strategies were developed that were effective for reducing multiple forms of problem behavior in children (e.g., Patterson, Chamberlain, & Reid, 1982). Over the ensuing decades, the behavior family therapy principles were applied and adapted to promote children's adjustment to address family formation and adaptation (Family Check-Up model), family disruption and maladaptation (Parent Management Training-Oregon model), and family attenuation and dissolution (Treatment Foster Care-Oregon model). We provide a brief overview of each intervention model and summarize randomized trials of intervention effectiveness. We review evidence on the viability of effective implementation, as well as barriers and solutions to adopting these evidence-based practices. We conclude by proposing an integrated family support system for the three models applied to the goal of reducing the prevalence of severe problem behavior, addiction, and mental problems for children and families, as well as reducing the need for costly and largely ineffective residential placements.

Resurrecting the chimera: Progressions in parenting and peer processes.

This report uses 6-year outcomes of the Oregon Divorce Study to examine the processes by which parenting practices affect deviant peer association during two developmental stages: early to middle childhood and late childhood to early adolescence. The participants were 238 newly divorced mothers and their 5- to 8-year-old sons who were randomly assigned to Parent Management Training-Oregon Model (PMTO®) or to a no-treatment control group. Parenting practices, child delinquent behavior, and deviant peer association were repeatedly assessed from baseline to 6 years after baseline using multiple methods and informants. PMTO had a beneficial effect on parenting practices relative to the control group. Two stage models linking changes in parenting generated by PMTO to children's growth in deviant peer association were supported. During the early to middle childhood stage, the relationship of improved parenting practices on deviant peer association was moderated by family socioeconomic status (SES); effective parenting was particularly important in mitigating deviant peer association for lower SES families whose children experience higher densities of deviant peers in schools and neighborhoods. During late childhood and early adolescence, the relationship of improved parenting to youths' growth in deviant peer association was mediated by reductions in the growth of delinquency during childhood; higher levels of early delinquency are likely to promote deviant peer association through processes of selective affiliation and reciprocal deviancy training. The results are discussed in terms of multilevel developmental progressions of diminished parenting, child involvement in deviancy producing processes in peer groups, and increased variety and severity of antisocial behavior, all exacerbated by ecological risks associated with low family SES.

Parent Management Training-Oregon Model: Adapting Intervention with Rigorous Research.

Parent Management Training-Oregon Model (PMTO(®) ) is a set of theory-based parenting programs with status as evidence-based treatments. PMTO has been rigorously tested in efficacy and effectiveness trials in different contexts, cultures, and formats. Parents, the presumed agents of change, learn core parenting practices, specifically skill encouragement, limit setting, monitoring/supervision, interpersonal problem solving, and positive involvement. The intervention effectively prevents and ameliorates children's behavior problems by replacing coercive interactions with positive parenting practices. Delivery format includes sessions with individual families in agencies or families' homes, parent groups, and web-based and telehealth communication. Mediational models have tested parenting practices as mechanisms of change for children's behavior and found support for the theory underlying PMTO programs. Moderating effects include children's age, maternal depression, and social disadvantage. The Norwegian PMTO implementation is presented as an example of how PMTO has been tailored to reach diverse populations as delivered by multiple systems of care throughout the nation. An implementation and research center in Oslo provides infrastructure and promotes collaboration between practitioners and researchers to conduct rigorous intervention research. Although evidence-based and tested within a wide array of contexts and populations, PMTO must continue to adapt to an ever-changing world.

Implementation and evaluation of linked parenting models in a large urban child welfare system.

During the past decade, there have been increased efforts to implement evidence-based practices into child welfare systems to improve outcomes for children in foster care and their families. In this paper, the implementation and evaluation of a policy-driven large system-initiated reform is described. Over 250 caseworkers and supervisors were trained and supported to implement two evidence-based parent focused interventions in five private agencies serving over 2,000 children and families. At the request of child welfare system leaders, a third intervention was developed and implemented to train the social work workforce to use evidence-based principles in everyday interactions with caregivers (including foster, relative, adoptive, and biological parents). In this paper, we describe the policy context and the targeted outcomes of the reform. We discuss the theory of the interventions and the logistics of how they were linked to create consistency and synergy. Training and ongoing consultation strategies used are described as are some of the barriers and opportunities that arose during the implementation. The strategy for creating a path to sustainability is also discussed. The reform effort was evaluated using both qualitative and quantitative methods; the evaluation design, research questions and preliminary results are provided.

Observational measures of implementer fidelity for a school-based preventive intervention: development, reliability, and validity.

Current measures of implementer fidelity often fail to adequately measure core constructs of adherence and competence, and their relationship to outcomes can be mixed. To address these limitations, we used observational methods to assess these constructs and their relationships to proximal outcomes in a randomized trial of a school-based preventive intervention (Rochester Resilience Project) designed to strengthen emotion self-regulation skills in first-third graders with elevated aggressive-disruptive behaviors. Within the intervention group (n = 203), a subsample (n = 76) of students was selected to reflect the overall sample. Implementers were 10 paraprofessionals. Videotaped observations of three lessons from year 1 of the intervention (14 lessons) were coded for each implementer-child dyad on adherence (content) and competence (quality). Using multilevel modeling, we examined how much of the variance in the fidelity measures was attributed to implementer and to the child within implementer. Both measures had large and significant variance accounted for by implementer (competence, 68 %; adherence, 41 %); child within implementer did not account for significant variance indicating that ratings reflected stable qualities of the implementer rather than the child. Raw adherence and competence scores shared 46 % of variance (r = .68). Controlling for baseline differences and age, the amount (adherence) and quality (competence) of program delivered predicted children's enhanced response to the intervention on both child and parent reports after 6 months, but not on teacher report of externalizing behavior. Our findings support the use of multiple observations for measuring fidelity and that adherence and competence are important components of fidelity which could be assessed by many programs using these methods.

Examining the process of change in an evidence-based parent training intervention: a qualitative study grounded in the experiences of participants.

While strong research evidence demonstrates that parent training interventions are capable of preventing child behavioral problems, much less is known about how the participants in these programs experience the change process. The purpose of this study was to provide a better understanding of how parents' experiences in an evidence-based parent training intervention led to change in their parenting practices, based on the first-person accounts of program participants. Qualitative data were collected through in-depth, individual interviews with parents who had completed the intervention known as Parent Management Training-the Oregon Model (PMTO™). Data were analyzed according to principles of the grounded theory approach, using the constant comparative method and a sequential process of open, axial, and selective coding. Study findings suggest that parents make active and intentional efforts to attempt, appraise, and apply the intervention material within their various life contexts, contributing to change in their parenting practices. Aspects of intervention content, method of delivery, and the role of the interventionist were also found to be important. This study can guide further prevention research into the mechanisms of change operating in parent training interventions and has the potential to inform continued efforts to adapt and implement evidence-based parent training interventions.

Parent Management Training-Oregon Model (PMTO™) in Mexico City: Integrating Cultural Adaptation Activities in an Implementation Model.

This article describes the process of cultural adaptation at the start of the implementation of the Parent Management Training intervention-Oregon model (PMTO) in Mexico City. The implementation process was guided by the model, and the cultural adaptation of PMTO was theoretically guided by the cultural adaptation process (CAP) model. During the process of the adaptation, we uncovered the potential for the CAP to be embedded in the implementation process, taking into account broader training and economic challenges and opportunities. We discuss how cultural adaptation and implementation processes are inextricably linked and iterative and how maintaining a collaborative relationship with the treatment developer has guided our work and has helped expand our research efforts, and how building human capital to implement PMTO in Mexico supported the implementation efforts of PMTO in other places in the United States.

Duration and mutual entrainment of changes in parenting practices engendered by behavioral parent training targeting recently separated mothers.

Parent management training (PMT) has beneficial effects on child and parent adjustment that last for 5 to 10 years. Short-term changes in parenting practices have been shown to mediate these effects, but the manner in which changes in specific components of parenting are sequenced and become reciprocally reinforcing (or mutually entrained) to engender and sustain the cascade of long-term beneficial effects resulting from PMT has received modest empirical attention. Long-term changes in parenting resulting from the Oregon model of PMT (PMTO) over a 2-year period were examined using data from the Oregon Divorce Study-II in which 238 recently separated mothers and their 6- to 10-year-old sons were randomly assigned to PMTO or a no treatment control (NTC) group. Multiple indicators of observed parenting practices were used to define constructs for positive parenting, monitoring and discipline at baseline, and at 6-, 12-, 18- and 30-months postbaseline. PMTO relative to NTC resulted in increased positive parenting and prevented deterioration in discipline and monitoring over the 30-month period. There were reliable sequential, transactional relationships among parenting practices; positive parenting supported better subsequent monitoring, and positive parenting and better monitoring supported subsequent effective discipline. Small improvements in parenting resulting from PMTO and small deteriorations in parenting in the NTC group may be sustained and amplified by mutually entrained relationships among parenting practices. These data about the change processes engendered by PMTO may provide information needed to enhance the power, effectiveness, and efficiency of behavioral parent training interventions.

Looking Forward: The Promise of Widespread Implementation of Parent Training Programs.

Over the past quarter century, researchers have developed a body of parent training programs that have proven effective in reducing child behavior problems, but few of these have made their way into routine practice. This article describes the long and winding road of implementation as applied to children's mental health. Adopting Rogers' (1995) diffusion framework and Fixsen and colleagues' implementation framework (Fixsen, Naoom, Blase, Friedman, & Wallace, 2005), we review more than a decade of research on the implementation of Parent Management Training-Oregon Model (PMTO). Data from U.S. and international PMTO implementations are used to illustrate the payoffs and the challenges of making empirically supported interventions routine practice in the community. Technological advances that break down barriers to communication across distances, the availability of efficacious programs suitable for implementation, and the urgent need for high quality mental health care provide strong rationales for prioritizing implementation. Over the next quarter of a century, the challenge is to reduce the prevalence of children's psychopathology by creating science-based delivery systems to reach families in need, everywhere.

Treatment effectiveness of PMTO for children's behavior problems in Iceland: assessing parenting practices in a randomized controlled trial.

Findings are presented from an Icelandic randomized control trial (RCT) evaluating parent management training - Oregon model (PMTO™), a parent training intervention designed to improve parenting practices and reduce child behavior problems. In a prior report from this effectiveness study that focused on child outcomes, children in the PMTO condition showed greater reductions in reported child adjustment problems relative to the comparison group. The present report focuses on observed parenting practices as the targeted outcome, with risk by treatment moderators also tested. It was hypothesized that mothers assigned to the PMTO condition would show greater gains in pre-post parenting practices relative to controls. The sample was recruited from five municipalities throughout Iceland and included 102 participating families of children with behavior problems. Cases were referred by community professionals and randomly assigned to either PMTO (n = 51) or community services usually offered (n = 51). Child age ranged from 5 to 12 years; 73% were boys. Contrary to expectations, findings showed no main effects for changes in maternal parenting. However, evaluation of risk by treatment moderators showed greater gains in parenting practices for mothers who increased in depressed mood within the PMTO group relative to their counterparts in the comparison group. This finding suggests that PMTO prevented the expected damaging effects of depression on maternal parenting. Failure to find hypothesized main effects may indicate that there were some unobserved factors regarding the measurement and a need to further adapt the global observational procedures to Icelandic culture.