Evaluating quality improvement at scale: A pilot study on routine reporting for executive board governance in a UK National Health Service organisation.

BACKGROUND: Quality improvement (QI) in healthcare is a cultural transformation process. We explored how routine reporting could be developed to aid visibility of the process for QI governance. METHOD: A retrospective evaluation of QI projects in a large healthcare organisation was conducted. We used an online survey so that the data accrual process resembled routine reporting to help identify implementation challenges. A purposive sample of QI projects was targeted to maximise contrast between projects that were or were not successful as determined by the resident QI team. To hone strategic focus in what should be reported, we also compared factors that might affect project outcomes. RESULTS: Out of 52 QI projects, 10 led to a change in routine practice ('adoption'). Details of project outcomes were limited. Project team outcomes, indicative of capacity building, were not systematically documented. Service user involvement, quality of measurement plan, fidelity of plan-do-study-act (PDSA) cycles had a major impact on adoption. CONCLUSION: Designing a routine reporting framework requires an iterative process to navigate data accrual demands. A retrospective evaluation, as in this study, can yield empirical insights to support development of QI governance, thereby honing the implementation science of QI in a healthcare organisation.

The development of the concept of return-on-investment from large-scale quality improvement programmes in healthcare: an integrative systematic literature review.

BACKGROUND: Return on Investment (ROI) is increasingly being used to evaluate financial benefits from healthcare Quality Improvement (QI). ROI is traditionally used to evaluate investment performance in the commercial field. Little is known about ROI in healthcare. The aim of this systematic review was to analyse and develop ROI as a concept and develop a ROI conceptual framework for large-scale healthcare QI programmes. METHODS: We searched Medline, Embase, Global health, PsycInfo, EconLit, NHS EED, Web of Science, Google Scholar using ROI or returns-on-investment concepts (e.g., cost-benefit, cost-effectiveness, value). We combined this terms with healthcare and QI. Included articles discussed at least three organisational QI benefits, including financial or patient benefits. We synthesised the different ways in which ROI or return-on-investment concepts were used and discussed by the QI literature; first the economically focused, then the non-economically focused QI literature. We then integrated these literatures to summarise their combined views. RESULTS: We retrieved 10 428 articles. One hundred and two (102) articles were selected for full text screening. Of these 34 were excluded and 68 included. The included articles were QI economic, effectiveness, process, and impact evaluations as well as reports and conceptual literature. Fifteen of 68 articles were directly focused on QI programme economic outcomes. Of these, only four focused on ROI. ROI related concepts in this group included cost-effectiveness, cost-benefit, ROI, cost-saving, cost-reduction, and cost-avoidance. The remaining articles mainly mentioned efficiency, productivity, value, or benefits. Financial outcomes were not the main goal of QI programmes. We found that the ROI concept in healthcare QI aligned with the concepts of value and benefit, both monetary and non-monetary. CONCLUSION: Our analysis of the reviewed literature indicates that ROI in QI is conceptualised as value or benefit as demonstrated through a combination of significant outcomes for one or more stakeholders in healthcare organisations. As such, organisations at different developmental stages can deduce benefits that are relevant and legitimate as per their contextual needs. TRIAL REGISTRATION: Review registration: PROSPERO; CRD42021236948.

Identifying and understanding benefits associated with return-on-investment from large-scale healthcare Quality Improvement programmes: an integrative systematic literature review.

BACKGROUND: We previously developed a Quality Improvement (QI) Return-on-Investment (ROI) conceptual framework for large-scale healthcare QI programmes. We defined ROI as any monetary or non-monetary value or benefit derived from QI. We called the framework the QI-ROI conceptual framework. The current study describes the different categories of benefits covered by this framework and explores the relationships between these benefits. METHODS: We searched Medline, Embase, Global health, PsycInfo, EconLit, NHS EED, Web of Science, Google Scholar, organisational journals, and citations, using ROI or returns-on-investment concepts (e.g., cost-benefit, cost-effectiveness, value) combined with healthcare and QI. Our analysis was informed by Complexity Theory in view of the complexity of large QI programmes. We used Framework analysis to analyse the data using a preliminary ROI conceptual framework that was based on organisational obligations towards its stakeholders. Included articles discussed at least three organisational benefits towards these obligations, with at least one financial or patient benefit. We synthesized the different QI benefits discussed. RESULTS: We retrieved 10 428 articles. One hundred and two (102) articles were selected for full text screening. Of these 34 were excluded and 68 included. Included articles were QI economic, effectiveness, process, and impact evaluations as well as conceptual literature. Based on these literatures, we reviewed and updated our QI-ROI conceptual framework from our first study. Our QI-ROI conceptual framework consists of four categories: 1) organisational performance, 2) organisational development, 3) external outcomes, and 4) unintended outcomes (positive and negative). We found that QI benefits are interlinked, and that ROI in large-scale QI is not merely an end-outcome; there are earlier benefits that matter to organisations that contribute to overall ROI. Organisations also found positive aspects of negative unintended consequences, such as learning from failed QI. DISCUSSION AND CONCLUSION: Our analysis indicated that the QI-ROI conceptual framework is made-up of multi-faceted and interconnected benefits from large-scale QI programmes. One or more of these may be desirable depending on each organisation's goals and objectives, as well as stage of development. As such, it is possible for organisations to deduce incremental benefits or returns-on-investments throughout a programme lifecycle that are relevant and legitimate.

'Walk this way': results from a pilot randomised controlled trial of a health coaching intervention to reduce sedentary behaviour and increase physical activity in people with serious mental illness.

BACKGROUND: Cardiovascular disease (CVD) is the leading cause of premature death among people with serious mental illness (SMI). Sedentary behaviour (SB) is an independent risk factor for CVD and mortality and people with SMI are highly sedentary. We developed a health coaching intervention called 'Walk this Way' to reduce SB and increase physical activity (PA) in people with SMI and conducted a pilot randomised controlled trial (RCT) to test its feasibility and acceptability. METHODS: We randomised people with SMI from three community mental health teams into either the WTW intervention or treatment as usual. The WTW intervention lasted 17 weeks and included an initial education session, fortnightly coaching, provision of pedometers and access to a weekly walking group. Objective SB and PA were measured with accelerometers. Cardiometabolic risk factors and wellbeing measures were collected. RESULTS: We recruited 40 people of whom 33 (82.5%) were followed up. 13/20 (65%) of participants allocated to the coaching intervention completed it. In the intervention group SB decreased by 56 min and total PA increased by 32 min per day on average which was sustained 6 months later. There was no change in PA or SB in the control group. When interviewed, participants in the intervention found the intervention helpful and acceptable. No adverse events were reported from the intervention. CONCLUSIONS: The intervention was feasible and acceptable to participants. Preliminary results were encouraging with improvement seen in both SB and PA. A larger study is needed to assess the effectiveness of the intervention and address any implementation challenges. TRIAL REGISTRATION: ISRCTN Registry identifier: ISRCTN37724980 , retrospectively registered 25 September 2015.

Development of the REFOCUS intervention to increase mental health team support for personal recovery.

BACKGROUND: There is an emerging evidence base about best practice in supporting recovery. This is usually framed in relation to general principles, and specific pro-recovery interventions are lacking. AIMS: To develop a theoretically based and empirically defensible new pro-recovery manualised intervention--called the REFOCUS intervention. METHOD: Seven systematic and two narrative reviews were undertaken. Identified evidence gaps were addressed in three qualitative studies. The findings were synthesised to produce the REFOCUS intervention, manual and model. RESULTS: The REFOCUS intervention comprises two components: recovery-promoting relationships and working practices. Approaches to supporting relationships comprise coaching skills training for staff, developing a shared team understanding of recovery, exploring staff values, a Partnership Project with people who use the service and raising patient expectations. Working practices comprise the following: understanding values and treatment preferences; assessing strengths; and supporting goal-striving. The REFOCUS model describes the causal pathway from the REFOCUS intervention to improved recovery. CONCLUSIONS: The REFOCUS intervention is an empirically supported pro-recovery intervention for use in mental health services. It will be evaluated in a multisite cluster randomised controlled trial (ISRCTN02507940).