An Equity-Focused Assessment of Evidence-Based Parenting Intervention Research.

Evidence-based parenting interventions (EBPI) support children and families to promote resilience, address emotional and behavioral concerns, and prevent or address issues related to child maltreatment. Critiques of EBPIs include concerns about their relevance and effectiveness for diverse populations when they are implemented at population scale. Research methods that center racial equity and include community-based participatory approaches have the potential to address some of these concerns. The purpose of the present review was to document the extent to which methods associated with promoting racial equity in research have been used in studies that contribute to the evidence base for programs that meet evidentiary standards for a clearinghouse that was developed to support the Family First Prevention Services Act in the United States. We developed a coding system largely based on the Culturally Responsive Evaluation model. A sample of 47 papers that are part of the evidence base for ten in-home parent skill-based programs were reviewed and coded. Only three of 28 possible codes were observed to occur in over half of the studies (including race/ethnicity demographic characteristics, conducting measure reliability for the study sample, and including information on socioeconomic status). Although the overall presence of equity-informed methods was low, a positive trend was observed over time. This review highlights ways in which rigorous research can incorporate racial equity into the planning, design, execution, and interpretation and dissemination of programs of study. We posit that doing so improves the external validity of studies while maintaining high-quality research that can contribute to an evidence base.

Culturally responsive evaluation: A scoping review of the evaluation literature.

Evaluators have become increasingly aware of the influence of culture in evaluation, leading to new evaluation approaches that account for the cultural considerations in which evaluations are situated. This scoping review sought to explore how evaluators understand culturally responsive evaluation and identify promising practices. A search of nine evaluation journals yielded 52 articles that were included in this review. Nearly two-thirds of the articles stated that community involvement was essential to culturally responsive evaluation. Power differentials were discussed in almost half of the articles, and the majority used participatory or collaborative approaches to community engagement. Findings from this review suggest that in culturally responsive evaluation, evaluators prioritize community involvement and have an awareness and attentiveness to power differentials. Yet, gaps exist in how culture and evaluation are defined and interpreted, and consequently, inconsistency in how culturally responsive evaluation is practiced.

Integrating the Consolidated Framework for Implementation Research (CFIR) into a Culturally Responsive Evaluation (CRE) Approach to Conduct Mixed-Method Evaluations of Diabetes Prevention and Management Programs Reaching Underresourced Populations and Communities.

Diabetes is a significant population health threat. Evidence-based interventions, such as the Centers for Disease Control and Prevention's National Diabetes Prevention Program and diabetes self-management education and support programs, can help prevent, delay, or manage the disease. However, participation is suboptimal, especially among populations who are at an increased risk of developing diabetes. Evaluations of programs reaching populations who are medically underserved or people with lower incomes can help elucidate how best to tailor evidence-based interventions, but it is also important for evaluations to account for cultural and contextual factors. Culturally responsive evaluation (CRE) is a framework for centering an evaluation in the culture of the programs being evaluated. We integrated CRE with implementation and outcome constructs from the Adapted Consolidated Framework for Implementation Research (CFIR) to ensure that the evaluation produced useful evidence for putting evidence-based diabetes interventions to use in real-world settings, reaching populations who are at an increased risk of developing diabetes. The paper provides an overview of how we integrated CRE and CFIR approaches to conduct mixed-methods evaluations of evidence-based diabetes interventions.

Empowering Indigenous Communities Through a Participatory, Culturally Responsive Evaluation of a Federal Program for Older Americans.

This article describes our experience of conducting a 5-year, culturally responsive evaluation of a federal program with Indigenous communities. It describes how we adapted tenets from "participatory evaluation models" to ensure cultural relevance and empowerment. We provide recommendations for evaluators engaged in similar efforts. The evaluation included stakeholder engagement through a Steering Committee and an Evaluation Working Group in designing and implementing the evaluation. That engagement facilitated attention to Indigenous cultural values in developing a program logic model and medicine wheel and in gathering local perspectives through storytelling to facilitate understanding of community traditions. Our ongoing assessment of program grantees' needs shaped our approach to evaluation capacity building and development of a diverse array of experiential learning opportunities and user-friendly tools and resources. We present practical strategies from lessons learned during the evaluation design and implementation phases of our project that might be useful for other evaluators.

Patient involvement in basic rheumatology research at Nijmegen: a three year's responsive evaluation of added value, pitfalls and conditions for success.

BACKGROUND: Empirical evidence for effective patient-researcher collaboration in basic research is lacking. This study aims to explore good working models and impact of patient involvement in basic rheumatology research and to identify barriers and facilitators. METHOD: A responsive evaluation of a three years' participatory research project in a basic and translational laboratory research setting. Several working models for patient involvement were piloted and adapted if considered necessary. The study comprised surveys, interviews, training days, meeting reports, Q-sort exercises and field notes, and regular reflective team sessions with participant involvement. A qualitative analysis using thematic coding focused on impact, barriers and facilitators. RESULTS: Thirteen patient research partners (PRPs) and fifteen basic researchers participated. PRPs experienced basic research as fascinating though complex to understand. Their initial role was mostly listening and asking questions. After several meetings equal and more meaningful relationships emerged. Researchers' motivation increased by listening to patient stories. They learned about disease impact on daily life and to speak in understandable language. This enabled PRPs to learn about research and the pathogenesis of their disease. It inspired them to stay involved over a longer period. After three years, both parties preferred 1:1 contacts over collaboration in team meetings. A common language and respectful communication were important facilitators. Limitations were the complexity of disease processes for patients and the time commitment for researchers. Impact was reported as a sincere dialogue with multiple advantages for patients and researchers, and to a lesser extent than expected on the research process and outcomes. CONCLUSION: Patient involvement contributes to motivating young scientists in performing basic research projects. Patients and researchers valued the benefits of long-term one-on-one collaboration. These benefits outweigh the lack of direct impact on basic research goals and performance. A plain language summary of the abstract is available (as) online Additional file 1.

Conceptualizing and Fostering the Quality of CES Through a Dutch National Network on CES (NEON).

The prevalence of Clinical ethics support (CES) services is increasing. Yet, questions about what quality of CES entails and how to foster the quality of CES remain. This paper describes the development of a national network (NEON), which aimed to conceptualize and foster the quality of CES in the Netherlands simultaneously. Our methodology was inspired by a responsive evaluation approach which shares some of our key theoretical presuppositions of CES. A responsive evaluation methodology engages stakeholders in developing quality standards of a certain practice, instead of evaluating a practice by predefined standards. In this paper, we describe the relationship between our theoretical viewpoint on CES and a responsive evaluation methodology. Then we describe the development of the network (NEON) and focus on three activities that exemplify our approach. In the discussion, we reflect on the similarities and differences between our approach and other international initiatives focusing on the quality of CES.

Using the science of positive psychology in the formative evaluation of social justice interventions: A case example.

The pursuit of culturally responsive approaches for designing and evaluating programs to promote social justice has become of the utmost importance to the evaluation community in the past decade. A strengths-focused evaluation approach has great promise for empowering individuals, groups, communities, and organizations, and identifying program strengths to build upon in addition to illuminating program deficits. However, there is a dearth of literature on using a strengths approach to evaluate interventions and programs to promote social justice. Drawing from the two disciplines of positive psychology and evaluation, this article illustrates a strengths-focused approach to formative evaluation using a case example of a halfway house for previously incarcerated women. The findings exemplify the positive psychological phenomena that emerge as a result of focusing the evaluation on program strengths. The case demonstrates that the application of a strengths-focused approach to evaluating social justice interventions can be empowering for institutions and the communities they serve.

Theory and practice of integrative clinical ethics support: a joint experience within gender affirmative care.

BACKGROUND: Clinical ethics support (CES) aims to support health care professionals in dealing with ethical issues in clinical practice. Although the prevalence of CES is increasing, it does meet challenges and pressing questions regarding implementation and organization. In this paper we present a specific way of organizing CES, which we have called integrative CES, and argue that this approach meets some of the challenges regarding implementation and organization. METHODS: This integrative approach was developed in an iterative process, combining actual experiences in a case study in which we offered CES to a team that provides transgender health care and reflecting on the theoretical underpinnings of our work stemming from pragmatism, hermeneutics and organizational and educational sciences. RESULTS: In this paper we describe five key characteristics of an integrative approach to CES; 1. Positioning CES more within care practices, 2. Involving new perspectives, 3. Creating co-ownership of CES, 4. Paying attention to follow up, and 5. Developing innovative CES activities through an emerging design. CONCLUSIONS: In the discussion we compare this approach to the integrated approach to CES developed in the US and the hub and spokes strategy developed in Canada. Furthermore, we reflect on how an integrative approach to CES can help to handle some of the challenges of current CES.

Responsive evaluation of stakeholder dialogue as a worksite health promotion intervention to contribute to the reduction of SEP related health inequalities: a study protocol.

BACKGROUND: Large health inequalities exist in the Netherlands among individuals with a high compared to a low socioeconomic position. Worksite health promotion interventions are considered promising to reduce these inequalities, however, current interventions seem not to have the desired effects. This study proposes 'moral case deliberation', a form of stakeholder dialogue on moral dilemmas, as an integrated and inclusive intervention for worksite health promotion. This intervention takes into account three factors that are considered possible underlying causes of low effectiveness of current interventions, namely the lack of deliberate attention to: 1) the diverging values and interests of stakeholders in worksite health promotion, 2) the ethical issues of worksite health promotion, and 3) the connection with the lived experience (lifeworld) of lower SEP employees. Moral case deliberation will help to gain insight in the conflicting values in worksite health promotion, which contributes to the development of a vision for worksite health promotion that is supported by all parties. METHODS: The intervention will be evaluated through Responsive Evaluation, a form of participatory research. Key to Responsive Evaluation is that stakeholders are consulted to determine relevant changes as a result of the intervention. The intervention will be evaluated yearly at both fixed moments (baseline and annual evaluation(s)) and continuously. Mixed methods will be used, including interviews, participatory observations, analyses of HRM-data and short questionnaires. In addition, the intervention will be evaluated economically, on both monetary and non-monetary outcomes. DISCUSSION: This protocol proposes an innovative intervention and a novel participatory evaluation in the context of worksite health promotion. The study aims to gain understanding in how dialogue on moral dilemmas on health and health promotion can contribute to heightened personal and mutual understanding among stakeholders and practice improvements in the work context. By evaluating the intervention in more than one setting, findings of this study will provide knowledge about how MCD can be adapted to specific work settings and what changes it may lead to in these settings. TRIAL REGISTRATION: Netherlands Trial Register (NRT): NL8051. Registration date: 28/09/2019, retrospectively registered. https://www.trialregister.nl/.

Lessons learned from implementing a responsive quality assessment of clinical ethics support.

BACKGROUND: Various forms of Clinical Ethics Support (CES) have been developed in health care organizations. Over the past years, increasing attention has been paid to the question of how to foster the quality of ethics support. In the Netherlands, a CES quality assessment project based on a responsive evaluation design has been implemented. CES practitioners themselves reflected upon the quality of ethics support within each other's health care organizations. This study presents a qualitative evaluation of this Responsive Quality Assessment (RQA) project. METHODS: CES practitioners' experiences with and perspectives on the RQA project were collected by means of ten semi-structured interviews. Both the data collection and the qualitative data analysis followed a stepwise approach, including continuous peer review and careful documentation of the decisions. RESULTS: The main findings illustrate the relevance of the RQA with regard to fostering the quality of CES by connecting to context specific issues, such as gaining support from upper management and to solidify CES services within health care organizations. Based on their participation in the RQA, CES practitioners perceived a number of changes regarding CES in Dutch health care organizations after the RQA: acknowledgement of the relevance of CES for the quality of care; CES practices being more formalized; inspiration for developing new CES-related activities and more self-reflection on existing CES practices. CONCLUSIONS: The evaluation of the RQA shows that this method facilitates an open learning process by actively involving CES practitioners and their concrete practices. Lessons learned include that "servant leadership" and more intensive guidance of RQA participants may help to further enhance both the critical dimension and the learning process within RQA.

Evaluating Family Group Conferencing: Towards a meaningful research methodology.

There is discussion on the most appropriate research methodology to examine the efficacy of Family Group Conferencing (FGC). Randomised controlled trials (RCTs), despite their pitfalls, are considered by many to be the 'golden standard', but the argument is not compelling. In this paper, the theory on programme evaluation is discussed which offers an alternative methodology to study FGC. It is argued that reaching a comprehensive image of truth in the social sciences is never within reach. A RCT is an abstraction of reality, it only provides a partial image of the complex reality of families and the impact that FGC has on safety issues and the quality of their lives. Moreover, the rigour of a study depends heavily on the researcher's interpretative skills. In studying the efficacy of a complex intervention, such as FGC, it is a challenge to provide a valid and reliable picture of its impact. The context of such a conference, where the lifeworld of families constantly interacts with the system world of professionals, is characterised by multiplicity, polyvalence and interference. The methodology used to examine the efficacy of FGC should meet this 'interplexitiy'.

Ethical Challenges Inherent in the Evaluation of an American Indian/Alaskan Native Circles of Care Project.

This article provides first-person accounts of ethical issues inherent in an evaluation of the Native American Indian Center of Central Ohio (NAICCO) Circles of Care project. Circles of Care is a three-year, infrastructure development program funded through the Substance Abuse and Mental Health Services Administration (SAMHSA) which is part of the federal Department of Health and Human Services (DHHS). The grant program is for American Indian and Alaskan Native (AI/AN) tribes and urban Indian communities and includes a strong emphasis on community engagement and community ownership. The Native American Indian Center of Central Ohio received a Circles of Care grant in the fifth cohort of the program. The first author (Project Evaluator) presents views that typically represent a western approach to evaluation, while the second author (Project Director) presents a Native perspective. Ethical issues are defined as well as the authors' efforts to address these concerns.

Work-related change in residential elderly care: Trust, space and connectedness.

Increasing care needs and a declining workforce put pressure on the quality and continuity of long-term elderly care. The need to attract and retain a solid workforce is increasingly acknowledged. This study reports about a change initiative that aimed to improve the quality of care and working life in residential elderly care. The research focus is on understanding the process of workforce change and development, by retrospectively exploring the experiences of care professionals. A responsive evaluation was conducted at a nursing home department in the Netherlands one year after participating in the change program. Data were gathered by participant observations, interviews and a focus and dialogue group. A thematic analysis was conducted. Care professionals reported changes in workplace climate and interpersonal interactions. We identified trust, space and connectedness as important concepts to understand perceived change. Findings suggest that the interplay between trust and space fostered interpersonal connectedness. Connectedness improved the quality of relationships, contributing to the well-being of the workforce. We consider the nature and contradictions within the process of change, and discuss how gained insights help to improve quality of working life in residential elderly care and how this may reflect in the quality of care provision.

Evaluating Clinical Ethics Support: A Participatory Approach.

The current process towards formalization within evaluation research, in particular the use of pre-set standards and the focus on predefined outcomes, implies a shift of ownership from the people who are actually involved in real clinical ethics support services (CESS) in a specific context to external stakeholders who increasingly gain a say in what 'good CESS' should look like. The question is whether this does justice to the insights and needs of those who are directly involved in actual CESS practices, be it as receivers or providers. We maintain that those actually involved in concrete CESS practices should also be involved in its evaluation, not only as respondents, but also in setting the agenda of the evaluation process and in articulating the criteria by which CESS is evaluated. Therefore, we propose a participatory approach to CESS evaluation. It focuses on (1) the concrete contexts in which CESS takes place, (2) reflective and dialogical learning processes, and (3) how to be democratic and inclusive. In particular, this approach to CESS evaluation is akin to realist evaluation, dialogical evaluation, and responsive evaluation. An example of a participatory approach to evaluating CESS is presented and some critical issues concerning this approach are discussed.

Politics in evaluation: Politically responsive evaluation in high stakes environments.

The role of politics has often been discussed in evaluation theory and practice. The political influence of the situation can have major effects on the evaluation design, approach and methods. Politics also has the potential to influence the decisions made from the evaluation findings. The current study focuses on the influence of the political context on stakeholder decision making. Utilizing a simulation scenario, this study compares stakeholder decision making in high and low stakes evaluation contexts. Findings suggest that high stakes political environments are more likely than low stakes environments to lead to reduced reliance on technically appropriate measures and increased dependence on measures better reflect the broader political environment.

Managers' views on and experiences with moral case deliberation in nursing teams.

AIMS: Providing management insights regarding moral case deliberation (MCD) from the experiential perspective of nursing managers. BACKGROUND: MCD concerns systematic group-wise reflection on ethical issues. Attention to implementing MCD in health care is increasing, and managers' experiences regarding facilitating MCD's implementation have not yet been studied. METHOD: As part of an empirical qualitative study on implementing MCD in mental health care, a responsive evaluation design was used. Using former research findings (iterative procedures), a managers' focus group was organised. RESULTS: Managers appreciated MCD, fostering nurses' empowerment and critical reflection - according to managers, professional core competences. Managers found MCD a challenging intervention, resulting in dilemmas due to MCD's confidential and egalitarian nature. Managers value MCD's process-related outcomes, yet these are difficult to control/regulate. CONCLUSIONS: MCD urges managers to reflect on their role and (hierarchical) position both within MCD and in the nursing team. IMPLICATIONS FOR NURSING MANAGEMENT: MCD is in line with transformative and participatory management, fostering dialogical interaction between management and nursing team.

Patient involvement in research programming and implementation: A responsive evaluation of the Dialogue Model for research agenda setting.

BACKGROUND: The Dialogue Model for research agenda-setting, involving multiple stakeholders including patients, was developed and validated in the Netherlands. However, there is little insight into whether and how patient involvement is sustained during the programming and implementation of research agendas. AIM: To understand how the Dialogue Model can be optimised by focusing on programming and implementation, in order to stimulate the inclusion of (the perspectives of) patients in research. METHODS: A responsive evaluation of the programming and implementation phases of nine agenda-setting projects that had used the Dialogue Model for agenda-setting was conducted. Fifty-four semi-structured interviews were held with different stakeholders (patients, researchers, funding agencies). Three focus groups with patients, funding agencies and researchers (16 participants) were organized to validate the findings. RESULTS: Patient involvement in programming and implementation of the research agendas was limited. This was partly related to poor programming and implementation, partly to pitfalls in earlier phases of the agenda-setting. Optimization of the Dialogue Model is possible by attending to the nature of the agenda and its intended use in earlier phases. Attention should also be given to the ambassadors and intended users of agenda topics. Support is needed during programming and implementation to organize patient involvement and adapt organizational structures like review procedures. In all phases the attitude to patient involvement, stakeholder participation, especially of researchers, and formal and informal relationships between parties need to be addressed to build a strong relationship with a shared goal. CONCLUSION: Patient involvement in agenda-setting is not automatically followed by patient involvement in programming and implementation. More attention should be paid, in earlier stages, to the attitude and engagement of researchers and funding agencies.

Crossing cultures: health promotion for senior migrants in the Netherlands.

A health promotion programme focusing on the meaning of everyday activities was implemented and evaluated to test its usefulness for community-dwelling seniors in the Netherlands. To evaluate how senior migrants with a Surinamese-Hindustani background and professionals received the programme, and how it could be contextualized and improved in line with their values and expectations. A responsive evaluation methodology was followed to foster reflexive learning in and among stakeholders as the basis for programme contextualization. The evaluation consisted of three phases. Outcomes of former phases served as input for subsequent phases. Methods included interviews and focus groups with seniors and professionals. Open and selective coding techniques were used to analyse the interactively derived data. A. small group of women was interested and followed the programme. It was not individual concerns or daily life problems that dominated, but the wish to become well informed, to maintain functional capacities and to continue their roles in the family and community. Striking differences in perspectives between professionals and migrants related to conflict between the underlying Western values of the programme (independence, personal control and autonomy) and the values of the migrants (interdependence, predestination, rebirth and destiny). Awareness among professionals of their own cultural background and the values of the migrant seniors was enhanced, but adapting the programme to the local context and values appeared far more complicated than originally expected. Adaptation requires intensive collaboration with participants and cultural brokers in the community.

Reflections on conducting evaluations for rural development interventions in China.

An appropriate evaluation methodology is critical in collecting valid data in complex development intervention contexts. This paper explores this issue by putting forward an appropriate evaluation methodology for development interventions in rural China. It draws on the experience of an impact evaluation of a sustainable agricultural biodiversity management project conducted in Hainan, China in 2010. The authors propose that evaluation be culturally responsive and the evaluation design be rooted in the particular cultural context where an evaluation is conducted. The appropriate use of the participatory rural appraisal (PRA) approach and methods helps generate data that are relevant and meaningful for evaluation purposes in rural China.

Stimulating program implementation via a Community of Practice: a responsive evaluation of care programs for frail older people in the Netherlands.

Evaluation is often used as a vehicle to improve program implementation. To evaluate the implementation of programs that provide care for frail older people a Community of Practice (CoP) was developed in the Netherlands. The purpose of this paper is to describe and reflect on the role of a CoP in the implementation of these programs. Using a responsive evaluation approach this study was based on interviews with participating stakeholders and transcripts of the CoP meetings with 13 professionals, project managers and patient representatives. Findings showed that CoP members had unanticipated concerns regarding the pro-active approach of the programs and older people not being open to receiving care. The heterogenic composition was appreciated and fostered learning. A social infrastructure was created for active learning inside and outside the CoP. We conclude that a CoP is a useful strategy as part of an evaluation aimed at improving program implementation. Lessons learned include the importance of creating ownership among CoP members by sharing responsibilities and paying attention to the heterogenic group composition and professional language spoken to involve all members.