ABSTRACT
While health services are expected to have public involvement in service (re)design, there is a dearth of evaluation of outcomes to inform policy and practice. There are major gaps in understanding why outcome evaluation is under-utilised. The aims of this interpretive descriptive study were to explore researcher participants' experiences of and/or attitudes towards evaluating health service outcomes from public involvement in health service design in high-income countries. Additionally, the aims were to explore barriers and enablers of evaluation, and reasons for the use of evaluation tools or frameworks. Semi-structured interviews (n = 13) were conducted with researchers of published studies where the public was involved in designing health services. Using framework analysis, four themes were developed that captured participants' experiences: Public involvement is hard - evaluation is harder; power, a diversity of agendas, and the invisible public; practical and methodological challenges; and genuineness and authenticity matter. Evaluation is driven by stakeholder requirements, including decision-makers, funding bodies, researchers, and academics, and evaluation tools are rarely used. The public is largely absent from the outcome evaluation agenda. There is a lack of commitment and clarity of purpose of public involvement and its evaluation. Outcome evaluation must be multi-layered and localised and reflect the purpose of public involvement, what constitutes success (and to whom), and use the most appropriate methods. Multi-level supports should include increased resources, such as funding, time, and expertise. Without improved evaluation, outcomes of investment in public involvement in health service design/redesign remain unknown.
Subject(s)
Health Services , Policy , Humans , Research PersonnelABSTRACT
BACKGROUND: Internationally, it is expected that health services will involve the public in health service design. Evaluation of public involvement has typically focused on the process and experiences for participants. Less is known about outcomes for health services. The aim of this systematic review was to a) identify and synthesise what is known about health service outcomes of public involvement and b) document how outcomes were evaluated. METHODS: Searches were undertaken in MEDLINE, EMBASE, The Cochrane Library, PsycINFO, Web of Science, and CINAHL for studies that reported health service outcomes from public involvement in health service design. The review was limited to high-income countries and studies in English. Study quality was assessed using the Mixed Methods Appraisal Tool and critical appraisal guidelines for assessing the quality and impact of user involvement in health research. Content analysis was used to determine the outcomes of public involvement in health service design and how outcomes were evaluated. RESULTS: A total of 93 articles were included. The majority were published in the last 5 years, were qualitative, and were located in the United Kingdom. A range of health service outcomes (discrete products, improvements to health services and system/policy level changes) were reported at various levels (service level, across services, and across organisations). However, evaluations of outcomes were reported in less than half of studies. In studies where outcomes were evaluated, a range of methods were used; most frequent were mixed methods. The quality of study design and reporting was inconsistent. CONCLUSION: When reporting public involvement in health service design authors outline a range of outcomes for health services, but it is challenging to determine the extent of outcomes due to inadequate descriptions of study design and poor reporting. There is an urgent need for evaluations, including longitudinal study designs and cost-benefit analyses, to fully understand outcomes from public involvement in health service design.
Subject(s)
Health Services , Income , Developed Countries , Humans , Longitudinal Studies , United KingdomABSTRACT
PURPOSE: To investigate whether the installation of electronic patient journey boards in an inpatient adult rehabilitation centre in Victoria, Australia, is associated with shorter lengths of stay for admitted adult rehabilitation patients. METHODS: A retrospective before-after analysis of 3 259 adult inpatient rehabilitation episodes from 2013 to 2018 was performed, analysing case-mix adjusted lengths of stay. RESULTS: A reduction in case-mix adjusted length of stay of 4.1 days per episode (95 % confidence interval: 2.0-6.4 days) was found. The corresponding reduction in hospital costs was estimated to be $3 738 per episode (95 % confidence interval $2 398-$4 983). CONCLUSIONS: Installation of electronic patient journey boards was associated with shorter lengths of stay in an inpatient adult rehabilitation centre. Additional research is needed to 1) provide further evidence of the causal effect of the boards on length of stay, and 2) investigate the mechanisms by which they reduce lengths of stay (e.g., increased currency of information, changes to procedures, remote viewing) in rehabilitation settings.
