Tools and methods for evaluating the change to health service delivery due to pandemics or other similar emergencies: A rapid evidence review.

The novel coronavirus, SARS-CoV-2 and its associated disease COVID-19, were declared a pandemic in March 2020. Countries developed rapid response activities within their health services to prevent spread of the virus and protect their populations. Evaluating health service delivery change is vital to assess how adapted practices worked, particularly during times of crisis. This review examined tools and methods that are used to evaluate health service delivery change during pandemics and similar emergencies. Five databases were searched, including PubMed, CENTRAL, Embase, CINAHL, and PsycINFO. The SPIDER tool informed the inclusion criteria for the articles. Articles in English and published from 2002 to 2020 were included. Risk of bias was assessed using the Mixed-Methods Appraisal Tool (MMAT). A narrative synthesis approach was used to analyse the studies. Eleven articles met the inclusion criteria. Many evaluation tools, methods, and frameworks were identified in the literature. Only one established tool was specific to a particular disease outbreak. Others, including rapid-cycle improvement and PDSA cycles were implemented across various disease outbreaks. Novel evaluation strategies were common across the literature and included checklists, QI frameworks, questionnaires, and surveys. Adherence practices, experience with telehealth, patient/healthcare staff safety, and clinical competencies were some areas evaluated by the tools and methods. Several domains, including patient/practitioner safety and patient/practitioner experience with telemedicine were also identified in the studies.

Quality improvement in a crisis: a qualitative study of experiences and lessons learned from the Irish National Ambulance Service response to the COVID-19 pandemic.

OBJECTIVES: The COVID-19 pandemic has produced radical changes in international health services. In Ireland, the National Ambulance Service established a novel home and community testing service that was central to the national COVID-19 screening programme. This service was overseen by a multidisciplinary response room. This research examined the response room service, particularly areas that performed well and areas requiring improvement, using a quality improvement (QI) framework. DESIGN: This was a qualitative study comprising semi-structured, individual interviews. Maximum variation sampling was used. The data were analysed using an established thematic analysis procedure. The analysis was guided by the framework, which comprised six QI drivers. SETTING: Response room employees, including clinicians, dispatchers and administrators, were interviewed via telephone. RESULTS: Leadership for quality: participants valued person-oriented leadership, including regular, open communication and consultation with staff. Person/family engagement: participants endeavoured to provide patient-centred care. Formal patient feedback mechanisms and shared decision-making could be beneficial in the future. Staff engagement: working in a response room could affect well-being, though it also provided networking and learning opportunities. Staff require support and teambuilding. Use of improvement methods: improvements were made in a relatively informal, ad hoc manner. The use of robust methods based on improvement science was not reported. Measurement for quality: data were collected to improve efficiency and accuracy. More rigorous measurement would be beneficial, especially formally collecting stakeholder feedback. Governance for quality: close alignment with collaborators and clear communication with staff are essential. Information and communications technology for quality: this seventh driver was added because the importance of information technology specially designed for pandemics was frequently highlighted. CONCLUSIONS: The study provides insights on what worked well and what required improvement in a pandemic response room. It can inform health services, particularly emergency services, in their preparation for additional COVID-19 waves, as well as future crises.

Rapid response and learning for later: establishing high quality information networks and evaluation frameworks for the National Ambulance Service response to COVID-19 - the ENCORE COVID Project Protocol.

Background: The National Ambulance Service (NAS) is at the forefront of Ireland's response to the COVID-19 pandemic. As directed in Ireland's National Action Plan, NAS significantly expanded prehospital services, including provision of a novel COVID-19 testing service. Additionally, other health services rely on NAS's capacity to assess, transport and/or treat COVID-19 patients. In a climate of innovation and adaptation, NAS needs to learn from international ambulance services and share experience. Evaluation of the NAS response to COVID-19 is required to facilitate evidence-based planning for subsequent waves or future pandemics, and to identify innovative practice for mainstreaming into routine service provision. Aims: This project aims to test the utility of novel information networks and develop a tool that is tailored to evaluating pandemic-imposed change in an emergency medical service. Methods: The first aim will be to introduce and measure the impact of ambulance-specific research and information updates for NAS. Secondly, the usefulness to members of an international network of senior ambulance and research personnel ('AMBULANCE+COVID19' network), and the clarity and feasibility of a short-survey instrument, the Emergency Medical Services Five Question Survey (EMS-5QS), will be assessed. Finally, an evaluation framework for assessing pandemic-imposed change will be developed to enable NAS determine innovations: (1) for reactivation in another wave or new pandemic; (2) to be sustained as part of routine service. The framework will be developed in collaboration with NAS and the National Quality Improvement Team. The Research Team includes expertise from academia, ambulance services and the National Public Health Emergency Team. Conclusions: This project will facilitate the prompt introduction of information sharing processes to an emergency medical service and assess the impact of those processes. By developing a process for evaluating pandemic-imposed change in NAS, this project will add to the toolbox for future pandemic planning in emergency medical services internationally.