Multi-method evaluation of a national clinical fellowship programme to build leadership capacity for quality improvement.

BACKGROUND: This paper reports the results of the evaluation of the Scottish Quality and Safety Fellowship (SQSF)-a 10-month, lead-level international educational programme established in 2008 with the overarching aim of developing clinicians with advanced quality improvement knowledge, technical ability and essential leadership skills. The evaluation explores four levels of educational and practice outcomes associated with (1) the reaction of fellows to SQSF participation, (2) learning gained, (3) subsequent behaviour changes and (4) the overall impact on national and international level capability and capacity building. METHODS: A theory-informed multi-method design was applied using (1) a search and review of the SQSF organisational database to tabulate personal, professional and demographic characteristics; (2) semi-structured telephone interviews with 16 participants using purposive and self-selected sampling; and (3) a cross-sectional online evaluation survey across all 10 cohorts involving 222 fellows RESULTS: SQSF was positively perceived as a high-quality learning experience containing a well-balanced mix of theory and practice, with a majority of respondents reporting career changing benefits. Most participants reported improved social, behavioural and emotional skills, knowledge and attitudes and, with sustained support of their host organisations, were able to apply and share learning in their workplace. The impact of the SQSF on a wider national and international level capability and capacity was both mediated and moderated by a wide range of interrelated contextual factors. CONCLUSIONS: This multi-method evaluation demonstrates that SQSF has achieved significant positive outcomes for the great majority of participants. Some tentative recommendations are provided with the aim of further enhancing fellowship content, delivery, transfer and future evaluations of wider impacts at regional, national and international health system levels.

Qualitative evaluation of the Safety and Improvement in Primary Care (SIPC) pilot collaborative in Scotland: perceptions and experiences of participating care teams.

OBJECTIVES: To explore general practitioner (GP) team perceptions and experiences of participating in a large-scale safety and improvement pilot programme to develop and test a range of interventions that were largely new to this setting. DESIGN: Qualitative study using semistructured interviews. Data were analysed thematically. SUBJECTS AND SETTING: Purposive sample of multiprofessional study participants from 11 GP teams based in 3 Scottish National Health Service (NHS) Boards. RESULTS: 27 participants were interviewed. 3 themes were generated: (1) programme experiences and benefits, for example, a majority of participants referred to gaining new theoretical and experiential safety knowledge (such as how unreliable evidence-based care can be) and skills (such as how to search electronic records for undetected risks) related to the programme interventions; (2) improvements to patient care systems, for example, improvements in care systems reliability using care bundles were reported by many, but this was an evolving process strongly dependent on closer working arrangements between clinical and administrative staff; (3) the utility of the programme improvement interventions, for example, mixed views and experiences of participating in the safety climate survey and meeting to reflect on the feedback report provided were apparent. Initial theories on the utilisation and potential impact of some interventions were refined based on evidence. CONCLUSIONS: The pilot was positively received with many practices reporting improvements in safety systems, team working and communications with colleagues and patients. Barriers and facilitators were identified related to how interventions were used as the programme evolved, while other challenges around spreading implementation beyond this pilot were highlighted.

Measuring the tail of the dog that doesn't bark in the night: the case of the national evaluation of Choose Life (the national strategy and action plan to prevent suicide in Scotland).

BACKGROUND: Learning about the impact of public health policy presents significant challenges for evaluators. These include the nebulous and organic nature of interventions ensuing from policy directives, the tension between long-term goals and short-term interventions, the appropriateness of establishing control groups, and the problems of providing an economic perspective. An example of contemporary policy that has recently been subject to evaluation is the first phase of the innovative Scottish strategy for suicide prevention (Choose Life). DISCUSSION AND SUMMARY: This paper discusses how challenges, such as those above, were made manifest within this programme. After a brief summary of the overarching approach taken to evaluating the first phase of Choose Life, this paper then offers a set of recommendations for policymakers and evaluators on how learning from a second phase might be augmented. These recommendations are likely to have general resonance across a range of policy evaluations as they move from early planning and implementation to more mature phases.

Changing the individual to promote health-enhancing physical activity: the difficulties of producing evidence and translating it into practice.

This paper presents conclusions from recent systematic reviews and highlights individually targeted interventions that are effective at increasing physical activity. It discusses the limitations of currently available evidence, considers what factors lead to these limitations and what barriers exist in terms of implementing the evidence as part of local and national policy and practice. Barriers present themselves in terms of getting evidence into practice and in terms of ensuring that practice informs the evidence base. These barriers include difficulties in conducting systematic reviews, disaggregating knowledge from complex interventions, making local adaptations to existing evidence, the lack of an evaluation culture, ethical and pragmatic difficulties in designing interventions, selecting appropriate outcome measures, poor designs and implementation of evidence and, finally, a recognition that policy making is not only based on the available evidence. New and more integrated approaches to evaluation and to practice are needed.