Rapid evaluation of COVID-19 related service and practice changes in health and human services using tailored methods.

The COVID-19 pandemic required substantive delivery and practice changes for government services under tight timeframes and high public scrutiny. These urgently implemented service changes provided the opportunity for evaluators to support decision-makers to understand the impact of adaptations for those delivering and receiving health and human services. Tailored rapid evaluation methods (REM) provide a pragmatic approach to generating timely information for evidence-based policy and decision-making under these conditions. Drawing from features of a range of existing rapid evaluation models, as well as developmental and utilization-focussed evaluation theory, this article outlines the design and implementation of a novel REM approach and considers the benefits of both tailoring and standardizing rapid evaluation approaches to meet end-user needs. The tailored REM approach and mixed methods are contextualized and compared to other documented rapid evaluation models to demonstrate the purpose and value of customization. This article builds on previous descriptions of the implementation of a novel REM approach to provide a comparative account of tailored rapid evaluation methods. The article outlines the drivers that led to the selected tailoring of the REM approach, and shares lessons learned in the context of the COVID-19 pandemic by a large internal government evaluation unit (Department of Health and Human Services) in Victoria, Australia. The customized features of REM ensure that it can consider the experiences of those delivering and receiving services, and inform near-term decision-making on programme and policy design in emergency and fast-paced contexts. The article shares a case study of a rapid evaluation of telehealth in pediatric care to demonstrate insights from tailoring the REM approach in practice. The REM method was utilized with the aim of delivering findings in a time-sensitive manner to rapidly inform decision making for policy-makers. Key enablers for the tailored REM protocol include the use of multi-disciplinary teams, flexible evaluation design, and a participatory approach that facilitates stakeholder involvement throughout delivery. Insights from the case study and methods presented seek to inform practice for evaluators who intend to or may want to tailor their own rapid evaluation model in resource and time-limited settings.

Using rapid evaluation methods to assess service delivery changes: Lessons learned for evaluation practice during the COVID-19 pandemic.

The COVID-19 pandemic required large-scale service delivery changes for government, and provided the opportunity for evaluators to step up and support decision makers to understand the impact of these changes. Rapid evaluation methods (REM) provide a pragmatic approach for generating timely information for evidence-based policy and decision-making. Grounded in developmental and utilisation-focused evaluation theory, REM incorporates a team-based, mixed methods design, executed over a 6-8-week period. Customised rubrics were used to rigorously assess effectiveness and scalability of practice changes to inform COVID-19 response planning. REM is an alternative approach to full-scale evaluation models frequently implemented to assess policies and programs. Adapted use of REM suggests that meaningful insights can be gained through use of smaller scale evaluations. This article shares lessons learned from a novel rapid evaluation method applied in the context of the COVID-19 pandemic. The rapid evaluation approach was implemented to provide real-time insights and evaluative conclusions for 15 program and practice adaptations across Victorian health and human service settings. The article shares insights about the practical applicability of balancing rigour and timeliness when implementing a rapid evaluation, and strengths and limitations of working within a fast-paced evaluation framework. Findings can inform evaluative practice in resource and time-limited settings.

Trans-disciplinary advanced allied health practitioners for acute hospital inpatients: a feasibility study.

OBJECTIVE: To explore cost-efficiency, safety and acceptability of trans-disciplinary advanced allied health (AH) practitioners for acute adult general medicine inpatients. DESIGN: Quasi-experimental feasibility study. SETTING: Three acute general medical units in an Australian urban hospital. PARTICIPANTS: Two hundred and fifty-six acute hospital inpatients. MAIN OUTCOME MEASURES: Cost-efficiency measures included AH service utilization and length of stay (LOS). Patient outcomes were functional independence, discharge destination, adverse events, unplanned admissions within 28 days, patient satisfaction and quality of life data on admission, and 30 days post-discharge. Ward staff were surveyed regarding satisfaction with the service model, and advanced health practitioners (AHPs) rated their confidence in their own ability to meet the performance standards of the role. RESULTS: Patients allocated to AHPs (n = 172) received 0.91 less hours of AH intervention (adjusted for LOS) (95% confidence intervals (CI): -1.68 to -0.14; P = 0.02) and had 1.76 days shorter LOS relative to expected (95%CI: 0.18-3.34; P = 0.03) compared with patients receiving standard AH (n = 84). There were no differences in patient outcomes or satisfaction. AHPs demonstrated growth in job satisfaction and skill confidence. CONCLUSIONS: Trans-disciplinary advanced AH roles may be feasible and cost-efficient compared with traditional roles for acute general medical inpatients. Further development of competency frameworks is recommended.

Advanced musculoskeletal physiotherapy: Barriers and enablers to multi-site implementation.

OBJECTIVES: Advanced musculoskeletal physiotherapy (AMP) services are a safe, effective model of care, but without broad-scale healthcare implementation to date. The aim of the present study was to identify the barriers and enablers to implementation of 12 AMP services from the perspective of clinical staff. METHODS: In a qualitative study, 12 participants (physiotherapists), from 12 different healthcare networks (seven metropolitan, three regional, two rural), were included. Their departments implemented AMP services (orthopaedic postoperative joint replacement review, n = 10; general orthopaedic, n = 1; emergency, n = 1; and neurosurgery n = 1) over a 12-month period. Participants completed a structured survey specifically designed for the study. Thematic analysis was used, with themes mapped to the validated Theoretical Domains Framework. RESULTS: Nine major themes emerged from the data regarding barriers and enablers to the implementation of the AMP services from the perspective of clinical staff. These were: demand/capacity; model of care; the organization; stakeholders; communication; planning and processes; evaluation; workforce; and learning and assessment framework. Important enablers included engagement and buy-in from key stakeholders and medical staff, and well-established AMP learning frameworks for training and operational frameworks. Barriers included competitive funding environment, and issues that hindered effective communication. The knowledge, skills, availability, motivation and experience of the advanced musculoskeletal physiotherapists had a large impact on the implementation. CONCLUSIONS: The study identified a number of factors that should be considered for successful implementation of AMP services across healthcare services or wider healthcare networks.

Advanced musculoskeletal physiotherapists in post arthroplasty review clinics: a state wide implementation program evaluation.

OBJECTIVE: To evaluate outcomes following a state-wide implementation of post arthroplasty review (PAR) clinics for patients following total hip and knee arthroplasty, led by advanced musculoskeletal physiotherapists in collaboration with orthopaedic specialists. DESIGN AND SETTING: A prospective observational study analysed data collected by 10 implementation sites (five metropolitan and five regional/rural centres) between September 2014 and June 2015. MAIN OUTCOME MEASURES: The Victorian Innovation and Reform Impact Assessment Framework was used to assess efficiency, effectiveness (access to care, safety and quality, workforce capacity, utilisation of skill sets, patient and workforce satisfaction) and sustainability (stakeholder engagement, succession planning and availability of ongoing funding). RESULTS: 2362 planned occasions of service (OOS) were provided for 2057 patients. Reduced patient wait times from referral to appointment were recorded and no adverse events occurred. Average cost savings across 10 sites was AUD$38 per OOS (Baseline $63, PAR clinic $35), representing a reduced pathway cost of 44%. Average annual predicted total value of increased orthopaedic specialist capacity was $11,950 per PAR clinic (range $6149 to $23,400). The Australian Orthopaedic Association review guidelines were met (8/10 sites, 80%) and patient-reported outcome measures were introduced as routine clinical care. High workforce and patient satisfaction were expressed. Eighteen physiotherapists were trained creating a sustainable workforce. Eight sites secured ongoing funding. CONCLUSIONS: The PAR clinics delivered a safe, cost-efficient model of care that improved patient access and quality of care compared to traditional specialist-led workforce models.

Exploration of an allied health workforce redesign model: quantifying the work of allied health assistants in a community workforce.

The Victorian Assistant Workforce Model (VAWM) enables a systematic approach for the identification and quantification of work that can be delegated from allied health professionals (AHPs) to allied health assistants (AHAs). The aim of the present study was to explore the effect of implementation of VAWM in the community and ambulatory health care setting. Data captured using mixed methods from allied health professionals working across the participating health services enabled the measurement of opportunity for workforce redesign in the community and ambulatory allied health workforce. A total of 1112 AHPs and 135 AHAs from the 27 participating organisations took part in the present study. AHPs identified that 24% of their time was spent undertaking tasks that could safely be delegated to an appropriately qualified and supervised AHA. This equates to 6837h that could be redirected to advanced and expanded AHP practice roles or expanded patient-centred service models. The VAWM demonstrates potential for more efficient implementation of assistant workforce roles across allied health. Data outputs from implementation of the VAWM are vital in informing strategic planning and sustainability of workforce change. A more efficient and effective workforce promotes service delivery by the right person, in the right place, at the right time.

Building allied health workforce capacity: a strategic approach to workforce innovation.

OBJECTIVE: The aim of the present study was to identify areas where allied health assistants (AHAs) are not working to their full scope of practice in order to improve the effectiveness of the allied health workforce. METHODS: Qualitative data collected via focus groups identified suitable AHA tasks and a quantitative survey with allied health professionals (AHPs) measured the magnitude of work the current AHP workforce spends undertaking these tasks. RESULTS: Quantification survey results indicate that Victoria's AHP workforce spends up to 17% of time undertaking tasks that could be delegated to an AHA who has relevant training and adequate supervision. Over half this time is spent on clinical tasks. CONCLUSIONS: The skills of AHAs are not being optimally utilised. Significant opportunity exists to reform the current allied health workforce. Such reform should result in increased capacity of the workforce to meet future demands.