RESUMO
The COVID-19 pandemic required substantive delivery and practice changes for government services under tight timeframes and high public scrutiny. These urgently implemented service changes provided the opportunity for evaluators to support decision-makers to understand the impact of adaptations for those delivering and receiving health and human services. Tailored rapid evaluation methods (REM) provide a pragmatic approach to generating timely information for evidence-based policy and decision-making under these conditions. Drawing from features of a range of existing rapid evaluation models, as well as developmental and utilization-focussed evaluation theory, this article outlines the design and implementation of a novel REM approach and considers the benefits of both tailoring and standardizing rapid evaluation approaches to meet end-user needs. The tailored REM approach and mixed methods are contextualized and compared to other documented rapid evaluation models to demonstrate the purpose and value of customization. This article builds on previous descriptions of the implementation of a novel REM approach to provide a comparative account of tailored rapid evaluation methods. The article outlines the drivers that led to the selected tailoring of the REM approach, and shares lessons learned in the context of the COVID-19 pandemic by a large internal government evaluation unit (Department of Health and Human Services) in Victoria, Australia. The customized features of REM ensure that it can consider the experiences of those delivering and receiving services, and inform near-term decision-making on programme and policy design in emergency and fast-paced contexts. The article shares a case study of a rapid evaluation of telehealth in pediatric care to demonstrate insights from tailoring the REM approach in practice. The REM method was utilized with the aim of delivering findings in a time-sensitive manner to rapidly inform decision making for policy-makers. Key enablers for the tailored REM protocol include the use of multi-disciplinary teams, flexible evaluation design, and a participatory approach that facilitates stakeholder involvement throughout delivery. Insights from the case study and methods presented seek to inform practice for evaluators who intend to or may want to tailor their own rapid evaluation model in resource and time-limited settings.
RESUMO
The COVID-19 pandemic required large-scale service delivery changes for government, and provided the opportunity for evaluators to step up and support decision makers to understand the impact of these changes. Rapid evaluation methods (REM) provide a pragmatic approach for generating timely information for evidence-based policy and decision-making. Grounded in developmental and utilisation-focused evaluation theory, REM incorporates a team-based, mixed methods design, executed over a 6-8-week period. Customised rubrics were used to rigorously assess effectiveness and scalability of practice changes to inform COVID-19 response planning. REM is an alternative approach to full-scale evaluation models frequently implemented to assess policies and programs. Adapted use of REM suggests that meaningful insights can be gained through use of smaller scale evaluations. This article shares lessons learned from a novel rapid evaluation method applied in the context of the COVID-19 pandemic. The rapid evaluation approach was implemented to provide real-time insights and evaluative conclusions for 15 program and practice adaptations across Victorian health and human service settings. The article shares insights about the practical applicability of balancing rigour and timeliness when implementing a rapid evaluation, and strengths and limitations of working within a fast-paced evaluation framework. Findings can inform evaluative practice in resource and time-limited settings.
