Development and validation of a data dictionary for a feasibility analysis of emergency department key performance indicators.

OBJECTIVES: The primary study objective was to describe the development of a data dictionary for a feasibility analysis of 11 emergency department (ED) key performance indicators (KPIs). The secondary objective was to internally validate the data dictionary by measuring the inter-observer agreement between data abstractors at participating study sites. METHODS: A list of data variables based on the minimum data set elements relevant to the KPIs was developed by a panel of emergency medicine (EM) specialists and from the EM literature. A summit involving the relevant stakeholders, including ED frontline staff, a health economist, an ED clinical data manager and a health care informatician, was convened. For the feasibility analysis project, each data abstractor was furnished with a copy of the data dictionary and attended a one-hour training session prior to commencing data abstraction. Data was independently abstracted for each KPI by two abstractors at each of 12 participating EDs. Inter-rater agreement between abstractors was calculated using Cohen's kappa and results were reported using the Landis and Koch criteria. RESULTS: A data dictionary was developed by creating clear definitions and establishing abstraction instructions for each variable. A total of 43 data variables were included in the study data dictionary: 4 on patient demographics; 19 time variables; 5 outcome variables; 8 ED service and staffing units and 7 medical definitions. A clear definition and a set of data abstraction instructions including data sources were developed for each variable to aid data abstraction during the feasibility analysis. Overall 9,276 ED patient records were used for data abstraction to internally validate the data dictionary. The median Cohen kappa score ranged between 0.56 to 0.81. CONCLUSION: There is a continued need to standardize definitions of KPIs for the purpose of comparing ED performance and for research purposes. This is a necessary first step in the implementation of valid and reliable ED performance measures. This study successfully developed an internally valid data dictionary that can be used for day-to-day ED operations and for research purposes.

Capturing data for emergency department performance monitoring purposes.

Background: Good-quality data is required for valid and reliable key performance indicators. Little is known of the facilitators and barriers of capturing the required data for emergency department key performance indicators. This study aimed to explore and understand how current emergency department data collection systems relevant to emergency department key performance indicators are integrated into routine service delivery, and to identify the resources required to capture these data elements. Methods: Following pilot testing, we conducted two focus groups with a multi-disciplinary panel of 14 emergency department stakeholders drawn from urban and rural emergency departments, respectively. Focus groups were analyzed using Attride-Stirling's framework for thematic network analysis. Results: The global theme "Understanding facilitators and barriers for emergency department data collection systems" emerged from three organizing themes: "understanding current emergency department data collection systems"; "achieving the ideal emergency department data capture system for the implementation of emergency department key performance indicators"; and "emergency department data capture systems for performance monitoring purposes within the wider context". Conclusion: The pathways to improving emergency department data capture systems for emergency department key performance indicators include upgrading emergency department information systems and investment in hardware technology and data managers. Educating stakeholders outside the emergency department regarding the importance of emergency department key performance indicators as hospital-wide performance indicators underpins the successful implementation of valid and reliable emergency department key performance indicators.