The relationship between faculty performance assessment and results on the in-training examination for residents in an emergency medicine training program.

BACKGROUND: Medical knowledge (MK) in residents is commonly assessed by the in-training examination (ITE) and faculty evaluations of resident performance. OBJECTIVE: We assessed the reliability of clinical evaluations of residents by faculty and the relationship between faculty assessments of resident performance and ITE scores. METHODS: We conducted a cross-sectional, observational study at an academic emergency department with a postgraduate year (PGY)-1 to PGY-3 emergency medicine residency program, comparing summative, quarterly, faculty evaluation data for MK and overall clinical competency (OC) with annual ITE scores, accounting for PGY level. We also assessed the reliability of faculty evaluations using a random effects, intraclass correlation analysis. RESULTS: We analyzed data for 59 emergency medicine residents during a 6-year period. Faculty evaluations of MK and OC were highly reliable (κ  =  0.99) and remained reliable after stratification by year of training (mean κ  =  0.68-0.84). Assessments of resident performance (MK and OC) and the ITE increased with PGY level. The MK and OC results had high correlations with PGY level, and ITE scores correlated moderately with PGY. The OC and MK results had a moderate correlation with ITE score. When residents were grouped by PGY level, there was no significant correlation between MK as assessed by the faculty and the ITE score. CONCLUSIONS: Resident clinical performance and ITE scores both increase with resident PGY level, but ITE scores do not predict resident clinical performance compared with peers at their PGY level.

Direct observation evaluations by emergency medicine faculty do not provide data that enhance resident assessment when compared to summative quarterly evaluations.

OBJECTIVES: The purpose of this study was to compare quarterly global evaluations with direct observation evaluations to determine if direct observation evaluations provide unique data compared to those obtained from quarterly global evaluations. METHODS: This observational, cohort study was performed at a 3-year emergency medicine (EM) residency program with 10 residents per year. Faculty used an online Web-based evaluation system to complete quarterly global evaluations and patient-specific direct observation evaluations. Two scores were collected for each resident within each quarterly evaluation period: 1) the quarterly evaluation score was the mean score across all faculty who performed a quarterly evaluation and, 2) the direct observation score was the mean score across all faculty who performed a direct observation evaluation. Pearson correlation coefficients were performed across these two groups of evaluations. RESULTS: Over the 4-year period of the study 296 complete data sets were available for the analysis. When the quarterly evaluation score was correlated with the direct observation score for each resident at the same evaluation period, we found a very high correlation for each of the eight evaluation questions (r = 0.95-0.96, p < 0.0001). When these evaluations were stratified based on the number of direct observation evaluations that were performed during the evaluation period of interest, the correlation between the quarterly evaluation and the direct observation scores increased as the number of direct observations in the evaluation period increased. The evaluation scores from the faculty who had performed both direct observation and quarterly evaluation methods during the same resident evaluation period were highly correlated even with small numbers of evaluators. CONCLUSIONS: Direct observations are highly correlated with quarterly evaluations when there are greater than three direct observation evaluations completed; however, this correlation drops significantly when the number of direct observations is lower. Direct observation evaluations provide similar data when compared with data obtained from quarterly global evaluations.

Opioids prolong nuclear hepatobiliary imaging when given prior to scanning.

Opioid-mediated contraction of the distal common bile duct (CBD) may delay tracer passage during nuclear hepatobiliary imaging (NHI), mimicking pathologic obstruction. We sought to determine if opioid administration before NHI delays CBD visualization and prolongs imaging. The records of 198 Emergency Department patients who underwent NHI were reviewed (after excluding those with evidence for pathologic CBD obstruction). Opioids were administered before NHI in 56 cases. Delayed CBD visualization occurred in 28.6% of subjects who had received opioids and in 12.0% of those who had not (p < 0.01). Delayed imaging was performed in 77.8% of those who had received opioids and in 53.5% of those who had not (p < 0.01). The relative risk of delayed CBD visualization was 1.46 [95%CI 0.65-3.28] for meperidine, 4.18 [95%CI 2.00-8.82] for morphine, and 2.38 [95%CI 1.29-4.39] for any opioid. We conclude that opioids given before NHI are associated with delayed CBD visualization and more imaging sessions.