Inter-rater Reliability of the Modified Emergency Severity Index as a Triage Tool

ABSTRACT

PURPOSE: Triage in the emergency departmen (ED) is the preliminary clinical sorting process before full disclosure of patients' problems so that patients with the highest acuity are treated first in the setting of resource constraints. To overcome the inter-rater variability of existing triage tools, the Emergency Severity Index (ESI) was developed and was shown to be both valid and reliable in practice. Because of the disparity in practice patterns and some inappropriate criteria in the original ESI, the authors have modified the ESI and determined its inter-rater reliability. METHODS: We applied the modified ESI to a convenient sample of adults who visited an urban academic ED between July 24, 2001, and August 5, 2001. After completion of a short, 4-hour training course on the modified ESI, an intern and emergency medicine resident pair triaged the patients independently. The inter-rater reliability was measured using a weighted kappa analysis and was categorized as excellent (>or=0.8), good (0.60-0.79), or fair (RESULTS: Five hundred forty-two patients were enrolled. The overall weighted kappa between the intern and the resident was 0.82 (95% CI 0.78-0.86). Among the 542 patients, 469/542 (87%) pairs agreed exactly, 67 (12%) pairs disagreed by 1 level, and 6 (1%) pairs by 2 levels.

CONCLUSION:

In this study, the modified ESI demonstrated excellent inter-rater reliability when used by residents and interns for our ED patients.