The use of "war games" to evaluate performance of students and residents in basic clinical scenarios: a disturbing analysis.

BACKGROUND: "Failure to Rescue" is a term applied to clinical issues that, if unrecognized or improperly treated, lead to adverse outcomes. We examined the cognitive components of rescue through the use of a "War Games" simulator format. Our hypothesis was that junior and senior medical students would be less able than interns and residents to detail the actions needed to assess, intervene, and stabilize patients. METHODS: Medical students and residents rotating on the trauma and surgical intensive care unit service participated. Twelve scenarios were created to focus on basic floor emergencies. Scores were assigned for clinical actions ordered. The scenarios were validated by two critical care attending physicians, and these scores were used as the expert group. Scores were assigned by two examiners, and the average of the grades in each area was used. The scores are a ratio of actual to possible correct responses in each section, and in the entire exercise. RESULTS: Subjects were divided into third-year medical students (MS3), fourth-year students (MS4), first-year residents (PGY1), residents beyond their first year (PGY2+), and experts. There were 20 subjects and 5 experts (n = 85) in each group for a total of 140 simulated cases examined. On initial evaluation, MS4 and PGY2+ performed significantly worse than expert, and MS3 and PGY1 performed similarly to experts. On secondary evaluation, all groups performed significantly worse than the expert group. In determining the diagnosis, only MS3 differed significantly from the experts. On follow-up, and in total score, all performed significantly worse than the experts. DISCUSSION: All groups had significant deficits in cognitive performance compared with experts in the areas of secondary evaluation, follow-up of the presenting problem, and total performance in simple clinical scenarios. We must design educational systems that rapidly enhance the cognitive performance of students and residents before they are left to independently diagnose and intervene in life-threatening clinical situations.

Proactive versus reactive: the effect of experience on performance in a critical care simulator.

BACKGROUND: We sought to study the cognitive performance of residents in a critical care patient simulator. METHODS: Residents in general surgery and emergency medicine were recruited to participate in the study. Subjects were read a morning report and presented with written data for 4 critical care patients. The subjects were evaluated on completing essential clinical tasks, cognitive errors, and directionality of reasoning. RESULTS: Nine residents completed the study. Months of clinical residency training did not significantly affect performance. Residents with more than 10 weeks of intensive care unit (ICU) experience (EXP) made significantly fewer cognitive errors than those with less than 10 weeks of ICU experience (N-I) (EXP: .75 +/- .96 vs N-I: 7 +/- 5.6 errors per subject, P < .05). An unexpected finding was that EXP performed far more proactive actions than N-I (EXP: 21.8 +/- 9.9/subject vs N-I: 5.7 +/- .6/subjects, P < .01). CONCLUSIONS: A unique finding was that residents with more than 10 weeks of ICU experience initiated a large number of proactive actions immediately following presentation of patient information, while N-I rarely performed these actions. In addition, residents with this degree of experience committed significantly fewer cognitive errors. These differences might play a role in efficiency, cost, and overall outcome in the care of ICU patients.