Expert-generated standard practice elements for evidence-based home visiting programs using a Delphi process.

BACKGROUND: States, territories, non-profits, and tribes are eligible to obtain federal funding to implement federally endorsed evidence-based home visiting programs. This represents a massive success in translational science, with $400 million a year allocated to these implementation efforts. This legislation also requires that 3% of this annual funding be allocated to tribal entities implementing home visiting in their communities. However, implementing stakeholders face challenges with selecting which program is best for their desired outcomes and context. Moreover, recent reviews have indicated that when implemented in practice and delivered at scale, many evidence-based home visiting programs fail to replicate the retention rates and effects achieved during clinical trials. To inform program implementers and better identify the active ingredients in home visiting programs that drive significant impacts, we aimed to develop an expert derived consensus taxonomy on the elements used in home visiting practice that are essential to priority outcome domains. METHODS: We convened a panel of 16 experts representing researchers, model representatives, and program implementers using a Delphi approach. We first elicited standard practice elements (SPEs) using open-ended inquiry, then compared these elements to behavior change techniques (BCTs) given their general importance in the field of home visiting; and finally rated their importance to 10 outcome domains. RESULTS: Our process identified 48 SPEs derived from the panel, with 83 additional BCTs added based on the literature. Six SPEs, mostly related to home visitor characteristics and skills, were rated essential across all outcome domains. Fifty-three of the 83 BCTs were rated unnecessary across all outcome domains. CONCLUSIONS: This work represents the first step in a consensus-grounded taxonomy of techniques and strategies necessary for home visiting programs and provides a framework for future hypothesis testing and replication studies.

Transparent, Open, and Reproducible Prevention Science.

The field of prevention science aims to understand societal problems, identify effective interventions, and translate scientific evidence into policy and practice. There is growing interest among prevention scientists in the potential for transparency, openness, and reproducibility to facilitate this mission by providing opportunities to align scientific practice with scientific ideals, accelerate scientific discovery, and broaden access to scientific knowledge. The overarching goal of this manuscript is to serve as a primer introducing and providing an overview of open science for prevention researchers. In this paper, we discuss factors motivating interest in transparency and reproducibility, research practices associated with open science, and stakeholders engaged in and impacted by open science reform efforts. In addition, we discuss how and why different types of prevention research could incorporate open science practices, as well as ways that prevention science tools and methods could be leveraged to advance the wider open science movement. To promote further discussion, we conclude with potential reservations and challenges for the field of prevention science to address as it transitions to greater transparency, openness, and reproducibility. Throughout, we identify activities that aim to strengthen the reliability and efficiency of prevention science, facilitate access to its products and outputs, and promote collaborative and inclusive participation in research activities. By embracing principles of transparency, openness, and reproducibility, prevention science can better achieve its mission to advance evidence-based solutions to promote individual and collective well-being.

Clearinghouse Standards of Evidence on the Transparency, Openness, and Reproducibility of Intervention Evaluations.

Clearinghouses are influential repositories of information on the effectiveness of social interventions. To identify which interventions are "evidence-based," clearinghouses review intervention evaluations using published standards of evidence that focus primarily on internal validity and causal inferences. Open science practices can improve trust in evidence from evaluations on the effectiveness of social interventions. Including open science practices in clearinghouse standards of evidence is one of many efforts that could increase confidence in designations of interventions as "evidence-based." In this study, we examined the policies, procedures, and practices of 10 federal evidence clearinghouses that review preventive interventions-an important and influential subset of all evidence clearinghouses. We found that seven consider at least one open science practice when evaluating interventions: replication (6 of 10 clearinghouses), public availability of results (6), investigator conflicts of interest (3), design and analysis transparency (3), study registration (2), and protocol sharing (1). We did not identify any policies, procedures, or practices related to analysis plan registration, data sharing, code sharing, material sharing, and citation standards. We provide a framework with specific recommendations to help federal and other evidence clearinghouses implement the Transparency and Openness Promotion (TOP) Guidelines. Our proposed "TOP Guidelines for Clearinghouses" includes reporting whether evaluations used open science practices, incorporating open science practices in their standards for receiving "evidence-based" designations, and verifying that evaluations used open science practices. Doing so could increase the trustworthiness of evidence used for policy making and support improvements throughout the evidence ecosystem.

The Role of Open Science Practices in Scaling Evidence-Based Prevention Programs.

The goal of creating evidence-based programs is to scale them at sufficient breadth to support population-level improvements in critical outcomes. However, this promise is challenging to fulfill. One of the biggest issues for the field is the reduction in effect sizes seen when a program is taken to scale. This paper discusses an economic perspective that identifies the underlying incentives in the research process that lead to scale up problems and to deliver potential solutions to strengthen outcomes at scale. The principles of open science are well aligned with this goal. One prevention program that has begun to scale across the USA is early childhood home visiting. While there is substantial impact research on home visiting, overall average effect size is .10 and a recent national randomized trial found attenuated effect sizes in programs implemented under real-world conditions. The paper concludes with a case study of the relevance of the economic model and open science in developing and scaling evidence-based home visiting. The case study considers how the traditional approach for testing interventions has influenced home visiting's evolution to date and how open science practices could have supported efforts to maintain impacts while scaling home visiting. It concludes by considering how open science can accelerate the refinement and scaling of home visiting interventions going forward, through accelerated translation of research into policy and practice.

Innovative Research Methods to Advance Precision in Home Visiting for More Efficient and Effective Programs.

Home visiting during early childhood can improve a range of outcomes for children and families. As evidence-based models are implemented across the nation, two questions have emerged. First, can home visiting improve outcomes more efficiently? Second, can overall effects be strengthened for specific subgroups of families? For the past several decades, research focused on testing the average effects of home visiting models on short- to long-term outcomes has found small impacts. These effects are not the same for all families. The field needs new evidence produced in new ways to overcome these challenges. In this article, we provide an overview of the evidence in this field, including what works and for whom. Next, we explain precision approaches to various fields and how this approach could be used in home visiting programs. Research on precision home visiting focuses on the ingredients of home visiting models, collaborating with practitioners to identify the ingredients and testing them on near-term outcomes, and using innovative study designs to learn more quickly what works best for which families. We conclude by proposing four pillars of research that will help achieve precision home visiting services.

Scaling up Evidence-Based Interventions in US Public Systems to Prevent Behavioral Health Problems: Challenges and Opportunities.

A number of programs, policies, and practices have been tested using rigorous scientific methods and shown to prevent behavioral health problems (Catalano et al., Lancet 379:1653-1664, 2012; National Research Council and Institute of Medicine, 2009). Yet these evidence-based interventions (EBIs) are not widely used in public systems, and they have limited reach (Glasgow et al., American Journal of Public Health 102:1274-1281, 2012; National Research Council and Institute of Medicine 2009; Prinz and Sanders, Clinical Psychology Review 27:739-749, 2007). To address this challenge and improve public health and well-being at a population level, the Society for Prevention Research (SPR) formed the Mapping Advances in Prevention Science (MAPS) IV Translation Research Task Force, which considered ways to scale up EBIs in five public systems: behavioral health, child welfare, education, juvenile justice, and public health. After reviewing other efforts to scale up EBIs in public systems, a common set of factors were identified as affecting scale-up in all five systems. The most important factor was the degree to which these systems enacted public policies (i.e., statutes, regulations, and guidance) requiring or recommending EBIs and provided public funds for EBIs. Across systems, other facilitators of scale-up were creating EBIs that are ready for scale-up, public awareness of and support for EBIs, community engagement and capacity to implement EBIs, leadership support for EBIs, a skilled workforce capable of delivering EBIs, and data monitoring and evaluation capacity. It was concluded that the following actions are needed to significantly increase EBI scale-up in public systems: (1) provide more public policies and funding to support the creation, testing, and scaling up of EBIs; (2) develop and evaluate specific frameworks that address systems level barriers impeding EBI scale-up; and (3) promote public support for EBIs, community capacity to implement EBIs at scale, and partnerships between community stakeholders, policy makers, practitioners, and scientists within and across systems.

Principles of Precision Prevention Science for Improving Recruitment and Retention of Participants.

Precision medicine and precision public health focus on identifying and providing the right intervention to the right population at the right time. Expanding on the concept, precision prevention science could allow the field to examine prevention programs to identify ways to make them more efficient and effective at scale, including addressing issues related to engagement and retention of participants. Research to date on engagement and retention has often focused on demographics and risk factors. The current paper proposes using McCurdy and Daro (Family Relations, 50, 113-121, 2001) model that posits a complex mixture of individual, provider, program, and community-level factors synergistically affect enrollment, engagement, and retention. The paper concludes recommending the use of research-practice partnerships and innovative, rapid cycle methods to design and improve prevention programs related to participant engagement and retention at scale.

The Intersection Between Prevention Science and Evidence-Based Policy: How the SPR Evidence Standards Support Human Services Prevention Programs.

The revised Society for Prevention Research (SPR) standards of evidence are an exciting advance in the field of prevention science. We appreciate the committee's vision that the standards represent goals to aspire to rather than a set of benchmarks for where prevention science is currently. The discussion about the standards highlights how much has changed in the field over the last 10 years and as knowledge, theory, and methods continue to advance, the new standards push the field toward increasing rigor and relevance. This commentary discusses how the revised standards support work of translating high-quality evaluations to support evidence-based policy and work supporting evidence-based programs' ability to implement at scale. The commentary ends by raising two areas, generating evidence at scale and transparency of research, as additional areas for consideration in future standards.

Effectiveness of home visiting in improving child health and reducing child maltreatment.

BACKGROUND AND OBJECTIVE: The Patient Protection and Affordable Care Act established the Maternal, Infant, and Early Childhood Home Visiting Program, which provides $1.5 billion to states over 5 years for home visiting program models serving at-risk pregnant women and children from birth to age 5. The act stipulates that 75% of the funds must be used for programs with evidence of effectiveness based on rigorous evaluation research. Home Visiting Evidence of Effectiveness reviewed the home visiting research literature and provided an assessment of the evidence of effectiveness for program models that serve families with pregnant women and children from birth to age 5. METHODS: Home Visiting Evidence of Effectiveness included a systematic search and screening process, a review of the research quality, and an assessment of program effectiveness. Reviewers rated studies' capacity to provide unbiased estimates of program impacts and determined whether a program met the Department of Health and Human Services' criteria for an evidence-based model. RESULTS: As of July 2012, 32 models were reviewed, of which 12 met the Department of Health and Human Services criteria. Most of these models were shown to have favorable effects on child development. Other common favorable effects included health care usage and reductions in child maltreatment. Less common were favorable effects on birth outcomes. CONCLUSIONS: Home visiting is a promising way to serve families who may be difficult to engage in supportive services. Existing rigorous research indicates that home visiting has the potential for positive results among high-risk families, particularly on health care usage and child development.

Directions in implementation research methods for behavioral and social science.

There is a growing interest, by researchers, policymakers, and practitioners, in evidence-based policy and practice. As a result, more dollars are being invested in program evaluation in order to establish "what works," and in some cases, funding is specifically tied to those programs found to be effective. However, reproducing positive effects found in research requires more than simply adopting an evidence-based program. Implementation research can provide guidance on which components of an intervention matter most for program impacts and how implementation components can best be implemented. However, while the body of rigorous research on effective practices continues to grow, research on implementation lags behind. To address these issues, the Administration for Children and Families and federal partners convened a roundtable meeting entitled, Improving Implementation Research Methods for Behavioral and Social Science, in the fall of 2010. This special section of the Journal of Behavioral Health Services & Research includes papers from the roundtable and highlights the role implementation science can play in shedding light on the difficult task of taking evidence-based practices to scale.

Preschool boys' development of emotional self-regulation strategies in a sample at risk for behavior problems.

Little longitudinal research has been conducted on changes in children's emotional self-regulation strategy (SRS) use after infancy, particularly for children at risk. In this study, the authors examined changes in boys' emotional SRS from toddlerhood through preschool. Repeated observational assessments using delay of gratification tasks at ages 2, 3, and 4 years were examined with both variable- and person-oriented analyses in a low-income sample of boys (N = 117) at risk for early problem behavior. Results were consistent with theory on emotional SRS development in young children. Children initially used more emotion-focused SRS (e.g., comfort seeking) and transitioned to greater use of planful SRS (e.g., distraction) by 4 years of age. Person-oriented analysis using trajectory analysis found similar patterns from 2 to 4 years, with small groups of boys showing delayed movement away from emotion-focused strategies or delay in the onset of regular use of distraction. The results provide a foundation for future researchers to examine the development of SRS in low-income young children.

Emotion regulation strategies and later externalizing behavior among European American and African American children.

Children's early emotion regulation strategies (ERS) have been related to externalizing problems; however, most studies have included predominantly European American, middle-class children. The current study explores whether ERS use may have differential outcomes as a function of the mother's ethnic culture. The study utilizes two diverse samples of low-income male toddlers to examine observed ERS during a delay of gratification task in relation to maternal and teacher reports of children's externalizing behavior 2 to 6 years later. Although the frequencies of ERS were comparable between ethnic groups in both samples, the use of physical comfort seeking and self-soothing was positively related to African American children's later externalizing behavior but negatively related to externalizing behavior for European American children in Sample 1. Data from Sample 2 appear to support this pattern for self-soothing in maternal, but not teacher, report of externalizing behavior. Within group differences by income were examined as a possible explanatory factor accounting for the ethnic differences, but it was not supported. Alternative explanations are discussed to explain the pattern of findings.

Physical Environmental Adversity and the Protective Role of Maternal Monitoring in Relation to Early Child Conduct Problems.

Research on the development of externalizing behaviors during early childhood has focused on child and parenting factors. Fewer studies have investigated effects of aversive features of the micro-level physical environment, such as overcrowding and chaos in the home, and the macro-level environment, such as neighborhood quality. This study extends research on physical environmental factors by examining their association with children's early externalizing behaviors, and exploring how maternal monitoring may serve as a protective factor in such contexts. 120 male toddlers at high risk for developing early externalizing behaviors were followed from ages 2 to 5 years. Direct longitudinal associations were found for micro-level environmental factors beginning at age 2 and for neighborhood risk beginning at age 3. Maternal monitoring served as a protective factor for child externalizing behaviors in the context of neighborhood risk. Implications for prevention research and the development of early externalizing behaviors are discussed.

The correlates of dyadic synchrony in high-risk, low-income toddler boys.

Synchrony has been broadly conceptualized as the quality of the parent-child dyadic relationship. Parenting, factors that compromise caregiving quality, and child characteristics have all been theoretically linked to synchrony, but little research has been conducted to validate such associations. The present study examined correlates of synchrony including parenting, maternal psychological resources and child attributes, among a sample of 120 mother-son dyads who were participating in a treatment study for children identified as being at risk for developing early conduct problems. There families participated in an at-home assessment, which included a series of mother-son interactions. Synchrony was associated with aspects of parenting and child attributes, including maternal nurturance, and child emotional negativity and language skills. The findings are discussed in terms of parent and child contributions to the development of synchrony.