Using intervention mapping to facilitate and sustain return-to work after breast cancer: protocol for the FASTRACS multicentre randomized controlled trial.

BACKGROUND: Women with breast cancer face many barriers to return to work (RTW) after their cancer. The main objective of the FASTRACS-RCT is to evaluate the impact of the FASTRACS (Facilitate and Sustain Return to Work after Breast Cancer) intervention on the sustainable RTW of breast cancer patients, 12 months after the end of active treatment. METHODS: FASTRACS-RCT is a prospective, national, multicentre, randomized, controlled and open-label study. A total of 420 patients with early breast cancer scheduled for surgery and (neo)adjuvant chemotherapy, will be randomly assigned (1:1 ratio) to: (i) the intervention arm comprising four steps over 6 months : Handing over the intervention tools; transitional medical consultation with the general practitioner (GP); pre-RTW visit with the company's occupational physician (OP); catch-up visit with a hospital-based RTW expert (if sick leave > 10 months) (ii) the control arm to receive usual care. The design of the FASTRACS intervention was informed by intervention mapping for complex interventions in health promotion planning, and involved patients and representatives of relevant stakeholders. Specific tools were developed to bridge the gap between the hospital, the GP, the OP and the workplace: a toolkit for breast cancer patients comprising a theory-based guide; specific checklists for the GP and the OP, respectively; and a theory-based guide for workplace actors (employer, manager, colleagues). The primary endpoint will associate sustainable RTW (full-time or part-time work at 50% or more of working time, for at least 28 consecutive days) and days off work. It will be assessed at 4, 8 and 12 months after the end of active oncological treatment. Secondary endpoints will include quality of life, anxiety, depression, RTW self-efficacy, physical activity, social support, job accommodations, work productivity, job status, and the usefulness and acceptability of the intervention's tools. DISCUSSION: FASTRACS-RCT will be supplemented by a realist evaluation approach aimed at understanding the influence of context in activating the intervention's mechanisms and effects. If the expected impact of the intervention is confirmed, the intervention will be adapted and scaled-up for other cancers and chronic diseases to better integrate healthcare and work disability prevention. TRIAL REGISTRATION: NCT04846972 ; April 15, 2021.

Building a programme theory of a specialist paediatric palliative and hospice care programme: development process and methodological reflection.

BACKGROUND: Paediatric palliative and hospice care aims to improve the quality of life of children with life-limiting and life-threatening conditions and their families. The number of these patients has risen significantly in recent years, resulting in an increased need for palliative care for this population. Although the need for paediatric palliative and hospice care is growing, meaningful outcome evaluation to demonstrate its effectiveness as a complex healthcare intervention is in its early stages. For complex interventions (programmes), theory-based evaluations have grown in prominence in recent years. They seek to understand how and why an intervention works by uncovering its underlying mechanisms by means of programme theory. To support both outcome evaluation in paediatric palliative care and a reflective practice of programme theorizing, we aimed to describe the construction of a programme theory for a specialist paediatric palliative and hospice care programme in Austria and to offer a reflective account of its development process. METHODS: We drew on a combination of theory-based evaluation frameworks to construct a programme theory consisting of an action and a change component. Through multiple iterations, incorporating different stakeholders' perspectives and drawing on different sources of knowledge and theory, we theorized how and why the programme likely achieves its intended outcomes. RESULTS: The programme theory outlines the proposed chains of events, causal mechanisms and outcomes of a specialist paediatric palliative and hospice care programme for children and families in several areas corresponding to its main conceptual tenets. Through a range of activities and interventions, the programme triggers coping and adaptation mechanisms that ultimately contribute to family and child wellbeing in physical, psychological, social, and spiritual dimensions. Established trust and partnership between children/families and healthcare professionals as well as a person-centered and family-centered approach were identified as enabling factors. CONCLUSIONS: Our findings provide insights into how a specialized paediatric palliative and hospice care programme works to achieve its intended outcomes for children and families. This helps demonstrate its impact, contributing to meaningful outcome evaluation and service improvement.

Using theory knitting to conceptualize social phenomena in the design and evaluation of dance programs for people living with Parkinson's disease.

Dance programs for people living with Parkinson's disease (PwPD) offer participants an opportunity to exercise, engage in artistic self-expression, and form new relationships. While it is understood that the social dimension of dance programs for PwPD contributes to dancer satisfaction and program sustainability, the social mechanisms instrumental to program success are under-examined. Engaging with theory from wider disciplines, or "theory knitting" can help program designers and evaluators examine the mechanisms and contextual factors that make classes socially meaningful with greater detail and specificity. This study identified and examined three theoretical frameworks that program planners and evaluators could use to conceptualize social engagement in dance for PwPD contexts and inform practice. Each theory was assessed for fit using the T-CaST theory comparison and selection tool developed by Birken et al. (2018). As an example, we used anthropologist Victor Turner's (1970; 1977) theory of liminality and communitas to identify five key areas for fostering a sense of social connection in dance for PwPD contexts: (1) selecting a meaningful dance space (2) creating a joyous atmosphere (3) marking entrance into the liminal time and space with rituals (4) embodying liminality and anti-structure and (5) inverting power relations and embracing fluid roles.

Doing better in theory-based evaluation, from A to Z.

This paper presents an alphabetical, select listing of concepts related to program theory-informed evaluation. Taken together, the concepts are relevant for understanding the fundamentals of program theory-based evaluation and especially for achieving a more beneficial future practice of such evaluations. The paper is offered with the hope of contributing to and helping stimulate further discussion of ways to improve theory-informed evaluation practice.

Simple but not simplistic: Findings from a theory-driven retrospective evaluation of a small projects program.

BACKGROUND AND PURPOSE: From 2010-2019, the United States Peace Corps Volunteers in Georgia implemented 270 small projects as part of the US Peace Corps/Georgia Small Projects Assistance (SPA) Program. In early 2020, the US Peace Corps/Georgia office commissioned a retrospective evaluation of these projects. The key evaluation questions were: 1) To what degree were SPA Program projects successful in achieving the SPA Program objectives over the ten years, 2) To what extent can the achieved outcomes be attributed to the SPA Program's interventions, and 3) How can the SPA Program be improved to increase likelihood of success of future projects. METHODS: Three theory-driven methods were used to answer the evaluation questions. First, a performance rubric was collaboratively developed with SPA Program staff to clearly identify which small projects had achieved intended outcomes and satisfied the SPA Program's criteria for successful projects. Second, qualitative comparative analysis was used to understand the conidtions that led to successful and unsuccessful projects and obtain a causal package of conditions that was conducive to a successful outcome. Third, causal process tracing was used to unpack how and why the conjunction of conditions identified through qualitative comparative analysis were sufficient for a successful outcome. FINDINGS: Based on the performance rubric, thirty-one percent (82) of small projects were categorized as successful. Using Boolean minimization of a truth table based on cross case analysis of successful projects, a causal package of five conditions was sufficient to produce the likelihood of a successful outcome. Of the five conditions in the causal package, the productive relationship of two conditions was sequential whereas for the remaining three conditions it was simultaneous. Distinctive characteristics explained the remaining successful projects that had only several of the five conditions present from the causal package. A causal package, comprised of the conjunction of two conditions, was sufficient to produce the likelihood of an unsuccessful project. CONCLUSIONS: Despite having modest grant amounts, short implementation periods, and a relatively straightforward intervention logic, success in the SPA Program was uncommon over the ten years because a complex combination of conditions was necessary to achieve success. In contrast, project failure was more frequent and uncomplicated. However, by focusing on the causal package of five conditions during project design and implementation, the success of small projects can be increased.

Towards an evolutionary approach to learning from assumptions: Lessons from the evaluation of Dancing with Parkinson's.

This paper highlights how learnings from exploring assumptions can be strengthened by taking an evolutionary approach to theory building and analysis. We discuss theory-driven evaluation applied to a community-based intervention implemented by Dancing With Parkinson's in Toronto, Canada, targeting Parkinson's disease (PD), a neurodegenerative condition affecting movement. A major gap in the literature is understanding the mechanisms by which dance might make a difference in the daily lives of people living with PD. This study was an early exploratory evaluation to better understand mechanisms and short-term outcomes. Conventional thinking generally favors "permanent" over "transitory" changes, and "long-term" over "short-term" effects. Yet, for people living with degenerative conditions (and also people experiencing chronic pain and other chronic symptoms), transitory and short-term changes may be highly valued and welcomed relief. In order to study and link multiple longitudinal events to explore key linkages in the theory of change, we piloted the use of diaries, with brief entries filled out daily by participants. The aim was to better understand the short-term experiences of participants using their daily routines as a means of learning about potential mechanisms, what matters to participants, and to see if small effects could be observed on days when participants danced versus days when they did not dance and also longitudinally over several months. Our initial theoretical stance began with a view of dance as exercise and the well-established benefits of exercise; yet, we explored through the diary data collected, as well as client interviews and literature review, potential other mechanisms of dancing (such as group interaction, touch, stimulation by the music, and esthetics including "feeling lovely"). This paper does not develop a full, comprehensive theory of dance but moves towards a more comprehensive view that locates dance within the routine activities of participants' daily lives. We argue that given the challenges of evaluating complex interventions comprising multiple, interacting components, there is a need for an evolutionary learning process to understand heterogeneities in mechanisms -- what works for whom -- when faced with knowledge incompleteness in the theory of change.

Learning from experiences of evaluators implementing theory-driven evaluations in diverse settings: Building on the contributions of John Mayne.

The papers in this volume grapple with various issues related to the use of theories of change in program evaluation. This introductory paper reviews some of the critical challenges that arise in developing and learning from theory-driven evaluations. These challenges include the relationship between theories of change and ecologies of evidence, the need for epistemic fluency in learning, and coming to terms with the initial incompleteness of knowledge in program mechanisms. The nine papers that follow, which represent a geographically diverse set of evaluations and evaluators (including Scotland, India, Canada, USA), help develop these and other themes. This volume of papers also serves as a celebration of the work of John Mayne, one of the foremost theory-driven evaluators of the last few decades. John passed away in December of 2020. This volume is intended to honor his legacy while also identifying challenging issues that call for further development.

Examining holistically the experiences of mentors in school-based programs: A logic analysis.

The article presents a Logic Analysis of the Scottish MCR ("Motivation, Commitment and Resilience") Pathways school-based mentoring scheme. MCR Pathways provides vulnerable secondary school students with one-to-one support, helping them realize their full potential through education. The perceptions of 12 mentors were explored through interviews, thematically analyzed and mapped to derive the program's Theory of Change as regards the volunteers themselves. This model was then assessed against the evidence base yielded from prior studies. The evaluation highlighted a mismatch between mentors' outcome expectations and what they actually gained from the experience. Furthermore, some themes (e.g., being driven by community concerns) turned out to be more prominent in the context of this specific scheme than in the wider literature, as opposed to other ones (e.g., developing friendships). The study generates insights into the ways to attract and retain growing numbers of volunteers, as well as to advance scientific knowledge.

Methods in realist evaluation: A mapping review.

Realist evaluation is becoming increasingly popular as an evaluation methodology. Its main objective is to uncover the mechanisms that lead to observed outcomes following an intervention and the contextual conditions that enabled this. The focus is on explaining why, for whom and in what circumstances an intervention works. It is a theory-driven approach and is explicitly method neutral, meaning that both quantitative and qualitative data collection methods can be used to unearth the underlying mechanisms that cause the intervention outcomes. In this review, we aim to map the methods used in realist evaluation studies, to draw lessons from the findings and to reflect on ways forward. We found that qualitative methods and interviews specifically are most commonly used in realist evaluations; that theory is often absent behind the methods and sampling techniques used; and that more innovative methods remain underexplored. We conclude the review by proposing four ways forward: (1) developing realist surveys, (2) exploring the relevance of innovative methods, (3) increasing the attention paid to sampling procedures and (4) strengthening the theory-driven nature of method. We believe that these four action points can strengthen the practice of realist evaluation and its outcomes.

Evaluating corporate-startup collaboration: A government perspective.

Government authorities are seeking new, cost-effective ways to stimulate growth and job creation by increasingly favoring the development of innovation ecosystems. Corporate-startup collaboration as a phenomenon has therefore progressively been supported by government authorities as a way of simultaneously helping startups to scale and providing opportunities for incumbents to renew themselves. However, evaluating the results of such collaborations is complex, which means that it can be a challenge for any government agency to evaluate its related interventions. This paper is explorative and based on empirical data from a study of the Swedish innovation agency, Vinnova. The purpose is to improve our understanding on how governments today evaluate government funded corporate-startup collaboration initiatives and how evaluations can be improved by utilizing methods based on theory-driven and realist approaches. An evaluation framework for policy makers based on a theory-driven and learning cycle approach is presented in this paper.

Conducting focus groups in realist evaluation.

Focus groups are valuable tools for evaluators to help stakeholders to clarify programme theories. In 1987, R.K. Merton, often attributed with the birth of focus groups, wrote about how these were 'being mercilessly misused'. In the 1940s, his team had conceived focus groups as tools for developing middle-range theory, but through their astonishing success focus groups have metamorphosed and are often an 'unchallenged' choice in many evaluation approaches, while their practice seems to provide a philosophically diverse picture. This article examines what knowledge focus group data generate, and how they support theory development. It starts with an overview of the history of focus groups, establishing a relationship between their emergence as a data collection method and the evaluation profession. Practical lessons for conducting groups in realist evaluation are suggested, while exploring how qualitative data can support programme and middle-range theory development using the example of realist evaluation.

Mechanisms of impact and contextual aspects of a dementia special care unit in long-term care: a process evaluation.

BACKGROUND: In long-term care, persons with dementia are often cared for in specialised facilities, which are rather heterogeneous in regard to care concepts. Little information is available on how these facilities and care concepts bring about changes in the targeted outcomes. Such knowledge is needed to understand the effects of care concepts and to consciously shape further developments. This study aimed to explore the mechanisms of impact of a specific care concept from a dementia special care unit and the contextual aspects that influence its implementation or outcomes. METHODS: Using a qualitative approach to process evaluation of complex interventions, we conducted participating observations and focus groups with nurses and single interviews with ward and nursing home managers. Data were collected from two identical dementia special care units to enhance the contrasts in the analysis of two non-specialised nursing homes. We analysed the data thematically. We conducted 16 observations, three group interviews and eleven individual interviews. RESULTS: We identified seven themes in three domains related to mechanisms that lead to outcomes regarding residents' and nurses' behaviour and well-being. The themes include the development of nurses' skills and knowledge, the promotion of a positive work climate, adjusted spatial structures, adjusted personnel deployment strategy "dedicated time for activities", promotion of relaxation, of engagement in activities and of engagement in social interaction of residents. The implementation and outcomes of the care concept are influenced by contextual aspects relating to the (target) population and cultural, organisational and financial features. CONCLUSIONS: The study found expected and unexpected mechanisms of impact and contextual aspects. The care concept of the dementia special care unit results in higher levels of relaxation, activities, and social interaction of residents. Its implementation highly depends on the shared understanding of nursing and the skills of the nursing team. Changes in residents' characteristics result in altered effects of the concept. TRIAL REGISTRATION: DRKS00011513 .

Tailoring Process and Impact Evaluation of a "Cash-Plus" Program: The Value of Using a Participatory Program Impact Pathway Analysis.

BACKGROUND: Evaluations are often limited to affirming what impact health and nutrition interventions have, without providing enough insights into "how/why" impacts are achieved. OBJECTIVES: This article describes how a Program Impact Pathway (PIP) analysis was used to tailor theory-driven impact and process evaluation of a "Cash-Plus" program combining unconditional cash transfers with behavior change communication (BCC) activities, which was implemented to improve children's growth in Togo. METHODS: A theoretical PIP diagram was developed using existing literature, program documentation, and interviews with the program's stakeholders at the central level. Next, the PIP diagram was refined through 2 regional participatory workshops, 6 mo after the program began. Workshop participants were multilevel field implementers and were asked to 1) discuss their vision of the program's objectives; 2) describe the "inputs-process-outputs-outcomes-impacts" flow; 3) reflect on modifiers that may arise along the PIP; and 4) report bottlenecks in the program's conception or implementation and suggest corrective actions. The PIP was used to determine research questions that should be investigated during impact or process evaluation and guided the choice of data collection methods and tools. RESULTS: The PIP analysis identified 3 impact pathways, all based on the synergy between cash and raised women's knowledge. Along these pathways, the motivation and workload of frontline workers, along with issues in cash flow, were identified as factors that may affect the delivery of activities, whereas women's control over resources, time availability, support from relatives, and the presence of markets and health and school services were recognized as factors that may influence the uptake of activities. Improved communication between stakeholders and increased involvement of husbands were suggested for better impact achievement. CONCLUSIONS: The participatory PIP analysis helped implementers and evaluators to share a common vision of the program's objective and logic, encouraged communication across sectors, and facilitated course-adjustments of the program.

The NASSS-CAT Tools for Understanding, Guiding, Monitoring, and Researching Technology Implementation Projects in Health and Social Care: Protocol for an Evaluation Study in Real-World Settings.

BACKGROUND: Projects to implement health care and social care innovations involving technologies are typically ambitious and complex. Many projects fail. Greenhalgh et al's nonadoption, abandonment, scale-up, spread, and sustainability (NASSS) framework was developed to analyze the varied outcomes of such projects. OBJECTIVE: We sought to extend the NASSS framework to produce practical tools for understanding, guiding, monitoring, and researching technology projects in health care or social care settings. METHODS: Building on NASSS and a complexity assessment tool (CAT), the NASSS-CAT tools were developed (in various formats) in seven co-design workshops involving 50 stakeholders (industry executives, technical designers, policymakers, managers, clinicians, and patients). Using action research, they were and are being tested prospectively on a sample of case studies selected for variety in conditions, technologies, settings, scope and scale, policy context, and project goals. RESULTS: The co-design process resulted in four tools, available as free downloads. NASSS-CAT SHORT is a taster to introduce the instrument and gauge interest. NASSS-CAT LONG is intended to support reflection, due diligence, and preliminary planning. It maps complexity through stakeholder discussion across six domains, using free-text open questions (designed to generate a rich narrative and surface uncertainties and interdependencies) and a closed-question checklist; this version includes an action planning section. NASSS-CAT PROJECT is a 35-item instrument for monitoring how subjective complexity in a technology implementation project changes over time. NASSS-CAT INTERVIEW is a set of prompts for conducting semistructured research or evaluation interviews. Preliminary data from empirical case studies suggest that the NASSS-CAT tools can potentially identify, but cannot always help reconcile, contradictions and conflicts that block projects' progress. CONCLUSIONS: The NASSS-CAT tools are a useful addition to existing implementation tools and frameworks. Further support of the implementation projects is ongoing. We are currently producing digital versions of the tools, and plan (subject to further funding) to establish an online community of practice for people interested in using and improving the tools, and hold workshops for building cross-project collaborations. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/16861.

'Our village is dependent on us. That's why we can't leave our work'. Characterizing mechanisms of motivation to perform among Accredited Social Health Activists (ASHA) in Bihar.

Community health workers (CHWs) play major roles in delivering primary healthcare services, linking communities to the formal health system and addressing the social determinants of health. Available evidence suggests that the performance of CHW programmes in low- and middle-income countries can be influenced by context-dependent causal mechanisms such as motivation to perform. There are gaps regarding what these mechanisms are, and what their contribution is to CHW performance. We used a theory-driven case study to characterize motivational mechanisms among Accredited Social Health Activists (ASHAs) in Bihar, India. Data were collected through semi-structured interviews with CHWs and focus group discussions with beneficiary women. Data were coded using a combined deductive and inductive approach. We found that ASHAs were motivated by a sense of autonomy and self-empowerment; a sense of competence, connection and community service; satisfaction of basic financial needs; social recognition; and feedback and answerability. Findings highlight the potential of ASHAs' intrinsic motivation to increase their commitment to communities and identification with the health system and of programme implementation and management challenges as sources of work dissatisfaction. Efforts to nurture and sustain ASHAs' intrinsic motivation while addressing these challenges are necessary for improving the performance of Bihar's ASHA programme. Further research is needed to characterize the dynamic interactions between ASHAs' motivation, commitment, job satisfaction and overall performance; also, to understand how work motivation is sustained or lost through time. This can inform policy and managerial reforms to improve ASHA programme's performance.

Implementing a toolkit for the prevention, management and control of carbapenemase-producing Enterobacteriaceae in English acute hospitals trusts: a qualitative evaluation.

BACKGROUND: Antimicrobial resistance is an increasing problem in hospitals world-wide. Following other countries, English hospitals experienced outbreaks of carbapenemase-producing Enterobacteriaceae (CPE), a bacterial infection commonly resistant to last resort antibiotics. One way to improve CPE prevention, management and control is the production of guidelines, such as the CPE toolkit published by Public Health England in December 2013. The aim of this research was to investigate the implementation of the CPE toolkit and to identify barriers and facilitators to inform future policies. METHODS: Acute hospital trusts (N = 12) were purposively sampled based on their self-assessed CPE colonisation rates and time point of introducing local CPE action plans. Following maximum variation sampling, 44 interviews with hospital staff were conducted between April and August 2017 using a semi-structured topic guide based on the Capability, Opportunity, Motivation and Behaviour Model and the Theoretical Domains Framework, covering areas of influences on behaviour. Interviews were audio-recorded, transcribed verbatim and analysed using thematic analysis. RESULTS: The national CPE toolkit was widely disseminated within infection prevention and control teams (IPCT), but awareness was rare among other hospital staff. Local plans, developed by IPCTs referring to the CPE toolkit while considering local circumstances, were in place in all hospitals. Implementation barriers included: shortage of isolation facilities for CPE patients, time pressures, and competing demands. Facilitators were within hospital and across-hospital collaborations and knowledge sharing, availability of dedicated IPCTs, leadership support and prioritisation of CPE as an important concern. Participants using the CPE toolkit had mixed views, appreciating its readability and clarity about patient management, but voicing concerns about the lack of transparency on the level of evidence and the practicality of implementation. They recommended regular updates, additional clarifications, tailored information and implementation guidance. CONCLUSIONS: There were problems with the awareness and implementation of the CPE toolkit and frontline staff saw room for improvement, identifying implementation barriers and facilitators. An updated CPE toolkit version should provide comprehensive and instructive guidance on evidence-based CPE prevention, management and control procedures and their implementation in a modular format with sections tailored to hospitals' CPE status and to different staff groups.

Assessment of the monitoring and evaluation system of a population-based program: Theory-driven evaluation approach.

The purpose of this study was to assess the performance of a monitoring and evaluation system (M&E) for a population-based program and to identify factors influencing its performance. The state health department provided a grant, guide, and training for health districts to implement the program and report quarterly and annually to the M&E. Some health districts did not meet this requirement. We also found that the M&E did not adequately capture the actual work of the districts in communities as related to the work plan. We identified several factors that negatively affected the performances of the M&E. For example, the state and health districts did not have a shared vision on how to develop and implement the program, and what to report to the M&E. We proposed strategies that could improve the timeliness and content of reports to the M&E, as well as its utilization in the future.

Implementation, Mechanisms of Effect and Context of an Integrated Care Intervention for Vulnerable Families in Central Sydney Australia: A Research and Evaluation Protocol.

INTRODUCTION: In March 2014, the New South Wales (NSW) Government (Australia) announced the NSW Integrated Care Strategy. In response, a family-centred, population-based, integrated care initiative for vulnerable families and their children in Sydney, Australia was developed. The initiative was called Healthy Homes and Neighbourhoods. A realist translational social epidemiology programme of research and collaborative design is at the foundation of its evaluation. THEORY AND METHOD: The UK Medical Research Council (MRC) Framework for evaluating complex health interventions was adapted. This has four components, namely 1) development, 2) feasibility/piloting, 3) evaluation and 4) implementation. We adapted the Framework to include: critical realist, theory driven, and continuous improvement approaches. The modified Framework underpins this research and evaluation protocol for Healthy Homes and Neighbourhoods. DISCUSSION: The NSW Health Monitoring and Evaluation Framework did not make provisions for assessment of the programme layers of context, or the effect of programme mechanism at each level. We therefore developed a multilevel approach that uses mixed-method research to examine not only outcomes, but also what is working for whom and why.

The NASSS framework for ex post theorisation of technology-supported change in healthcare: worked example of the TORPEDO programme.

BACKGROUND: Evaluation of health technology programmes should be theoretically informed, interdisciplinary, and generate in-depth explanations. The NASSS (non-adoption, abandonment, scale-up, spread, sustainability) framework was developed to study unfolding technology programmes in real time-and in particular to identify and manage their emergent uncertainties and interdependencies. In this paper, we offer a worked example of how NASSS can also inform ex post (i.e. retrospective) evaluation. METHODS: We studied the TORPEDO (Treatment of Cardiovascular Risk in Primary Care using Electronic Decision Support) research programme, a multi-faceted computerised quality improvement intervention for cardiovascular disease prevention in Australian general practice. The technology (HealthTracker) had shown promise in a cluster randomised controlled trial (RCT), but its uptake and sustainability in a real-world implementation phase was patchy. To explain this variation, we used NASSS to undertake secondary analysis of the multi-modal TORPEDO dataset (results and process evaluation of the RCT, survey responses, in-depth professional interviews, videotaped consultations) as well as a sample of new, in-depth narrative interviews with TORPEDO researchers. RESULTS: Ex post analysis revealed multiple areas of complexity whose influence and interdependencies helped explain the wide variation in uptake and sustained use of the HealthTracker technology: the nature of cardiovascular risk in different populations, the material properties and functionality of the technology, how value (financial and non-financial) was distributed across stakeholders in the system, clinicians' experiences and concerns, organisational preconditions and challenges, extra-organisational influences (e.g. policy incentives), and how interactions between all these influences unfolded over time. CONCLUSION: The NASSS framework can be applied retrospectively to generate a rich, contextualised narrative of technology-supported change efforts and the numerous interacting influences that help explain its successes, failures, and unexpected events. A NASSS-informed ex post analysis can supplement earlier, contemporaneous evaluations to uncover factors that were not apparent or predictable at the time but dynamic and emergent.

Qualitative impact evaluation of a social marketing campaign for conservation.

Social marketing campaigns use marketing techniques to influence human behavior for the greater social good. In the conservation sector, social marketing campaigns have been used to influence behavior for the benefit of biodiversity as well as society. However, there are few evaluations of their effectiveness. We devised an approach for evaluating the influences of social marketing campaigns on human behavior and conservation outcomes. We used general elimination methodology, a theory-driven qualitative evaluation method, to assess the long-term impacts of a 1998 Rare Pride campaign on the island of Bonaire that was designed to increase the population of the Lora (Amazona barbadensis), a threatened parrot. We interviewed stakeholder groups to determine their perceptions of the drivers of the changes in the Lora population over time. We used these data to develop an overall theory of change to explain changes in the Lora population by looking at the overlap in hypotheses within and between stakeholder groups. We then triangulated that theory of change with evidence from government reports, peer-reviewed literature, and newspapers. The increase in the Lora population was largely attributed to a decrease in illegal poaching of Loras and an associated decrease in local demand for pet Loras. Decreases in poaching and demand were likely driven by a combination of law enforcement, social marketing (including the Rare campaign), and environmental education in schools. General elimination methodology helped show how multiple interventions influenced a conservation outcome over time. There is a need for evidence-based evaluations of social marketing interventions to ensure that limited resources are spent wisely.

Evaluación del Impacto Cualitativo de una Campaña de Mercadotecnia Social para la Conservación Resumen Las campañas de mercadotecnia social usan técnicas de mercadotecnia para influenciar al comportamiento humano para el mayor beneficio social. En el sector de conservación, las campañas sociales se han usado para influenciar al comportamiento para el beneficio de la biodiversidad y de la sociedad. Sin embargo, existen pocas evaluaciones sobre su efectividad. Diseñamos una estrategia para evaluar la influencia de las campañas de mercadotecnia social sobre el comportamiento humano y los resultados de conservación. Usamos la metodología de eliminación general, un método de evaluación cualitativa llevada por la teoría, para evaluar los impactos a largo plazo de una campaña de Rare Pride de 1998 en la isla de Bonaire, la cual fue diseñada para incrementar la población de la lora (Amazona barbadensis), un psitácido amenazado. Entrevistamos a grupos de accionistas para determinar sus percepciones de los causantes del cambio en la población de loras con el tiempo. Usamos estos datos para desarrollar una teoría general de cambio para explicar los cambios en la población de loras al observar el traslape en las hipótesis dentro y entre los grupos de accionistas. Después triangulamos esa teoría de cambio con evidencia tomada de los reportes gubernamentales, la literatura revisada por pares, y los periódicos. El incremento poblacional de loras se atribuyó en su mayoría a una disminución en la demanda local de las loras como mascota. La disminución de la caza furtiva y de la demanda estuvo probablemente causada por una combinación de la aplicación de la ley, la mercadotecnia social (incluyendo a la campaña de Rare), y la educación ambiental en las escuelas. La metodología de eliminación general ayudó a mostrar cómo las múltiples intervenciones influyeron sobre un resultado de conservación con el tiempo. Existe una necesidad por tener evaluaciones basadas en evidencias de las intervenciones de mercadotecnia social para asegurar que los recursos limitados se usen racionalmente.