Residents' reflections on quality improvement: temporal stability and associations with preventability of adverse patient events.

PURPOSE: Residents' reflections on quality improvement (QI) opportunities are poorly understood. The authors used the Mayo Evaluation of Reflection on Improvement Tool (MERIT) to measure residents' reflection scores across three years and to determine associations between reflection scores and resident and adverse patient event characteristics. METHOD: From 2006 to 2009, 48 Mayo Clinic internal medicine residents completed biannual reflections on adverse events and classified event severity and preventability. Faculty assessed residents' reflections using MERIT, which contains 18 Likert-scaled items and measures three factors-personal reflection, systems reflection, and event merit. ANOVA was used to identify changes in MERIT scores across three years of training and among factors, paired t tests were used to identify differences between MERIT factor scores, and generalized estimating equations were used to examine associations between MERIT scores and resident and adverse event characteristics. RESULTS: The residents completed 240 reflections. MERIT reflection scores were stable over time. Individual factor scores differed significantly (P < .0001), with event merit being the highest and systems reflection the lowest. Event preventability was significantly associated with MERIT factor scores and overall scores (beta = 0.415; CI = 0.186-0.643; P = .0004). No significant associations between MERIT scores and resident characteristics or event severity were identified. CONCLUSIONS: Residents' reflections on adverse events remained constant over time, were lowest for systems factors, and were associated with adverse event preventability. Future research should explore learners' emphasis on systems aspects of QI and the relationship between QI and event preventability.

Relationship between critical reflection and quality improvement proposal scores in resident doctors.

OBJECTIVES: transformative learning theory supports the idea that reflection on quality improvement (QI) opportunities and the ability to develop successful QI projects may be fundamentally linked. We used validated methods to explore associations between resident doctors' reflections on QI opportunities and the quality of their QI project proposals. METHODS: eighty-six residents completed written reflections on practice improvement opportunities and developed QI proposals. Two faculty members assessed residents' reflections using the 18-item Mayo Evaluation of Reflection on Improvement Tool (MERIT), and assessed residents' QI proposals using the seven-item Quality Improvement Project Assessment Tool (QIPAT-7). Both instruments have been validated in previous work. Associations between MERIT and QIPAT-7 scores were determined. Internal consistency reliabilities of QIPAT-7 and MERIT scores were calculated. RESULTS: there were no significant associations between MERIT overall and domain scores, and QIPAT-7 overall and item scores. The internal consistency of MERIT and QIPAT-7 item groups were acceptable (Cronbach's α 0.76-0.94). CONCLUSIONS: the lack of association between MERIT and QIPAT-7 scores indicates a distinction between resident doctors' skills at reflection on QI opportunities and their abilities to develop QI projects. These findings suggest that practice-based reflection and QI project development are separate constructs, and that skilful reflection may not predict the ability to design meaningful QI initiatives. Future QI curricula should consider teaching and assessing QI reflection and project development as distinct components.

Validation of a method to measure resident doctors' reflections on quality improvement.

OBJECTIVES: Resident reflection on the clinical learning environment is prerequisite to identifying quality improvement (QI) opportunities and demonstrating competence in practice-based learning. However, residents' abilities to reflect on QI opportunities are unknown. Therefore, we developed and determined the validity of the Mayo Evaluation of Reflection on Improvement Tool (MERIT) for assessing resident reflection on QI opportunities. METHODS: The content of MERIT, which consists of 18 items structured on 4-point scales, was based on existing literature and input from national experts. Using MERIT, six faculty members rated 50 resident reflections. Factor analysis was used to examine the dimensionality of MERIT instrument scores. Inter-rater and internal consistency reliabilities were calculated. RESULTS: Factor analysis revealed three factors (eigenvalue; number of items): Reflection on Personal Characteristics of QI (8.5; 7); Reflection on System Characteristics of QI (1.9; 6), and Problem of Merit (1.5; 5). Inter-rater reliability was very good (intraclass correlation coefficient range: 0.73-0.89). Internal consistency reliability was excellent (Cronbach's alpha 0.93 overall and 0.83-0.91 for factors). Item mean scores were highest for Problem of Merit (3.29) and lowest for Reflection on System Characteristics of QI (1.99). CONCLUSIONS: Validity evidence supports MERIT as a meaningful measure of resident reflection on QI opportunities. Our findings suggest that dimensions of resident reflection on QI opportunities may include personal, system and Problem of Merit factors. Additionally, residents may be more effective at reflecting on 'problems of merit' than personal and systems factors.

Internal medicine resident self-report of factors associated with career decisions.

BACKGROUND: Little is known about factors contributing to the career decisions of internal medicine residents. OBJECTIVE: To evaluate factors self-reported by internal medicine residents nationally as important to their career decisions. DESIGN: Cross-sectional survey conducted in October of 2005, 2006, and 2007 as part of the national Internal Medicine In-Training Examination (IM-ITE). PARTICIPANTS: Postgraduate year 3 internal medicine residents taking the IM-ITE. MEASUREMENTS: Residents rated the importance of nine factors in their career decisions on 5-point Likert scales. Univariate statistics characterized the distribution of responses. Associations between variables were evaluated using Cochran-Mantel-Haenszel statistics for ordinal data. Multivariate analyses were conducted using logistic regression. RESULTS: Of 17,044 eligible residents taking the IM-ITE, 14,890 (87.4%) completed the career decision survey questions. Overall, time with family was the factor most commonly reported as of high or very high importance to career decisions (69.6%). Women were more likely to assign greatest importance to family time (OR 1.22, 95% confidence interval 1.12-1.31, p < 0.001) and long-term patient relationships (OR 1.34, 95% confidence interval 1.23-1.46, p < 0.001). Across debt levels, financial considerations were of greatest importance more often for residents owing >$150,000 (OR 1.33, 95% confidence interval 1.09-1.62, p < 0.001). Across specialties, mentor specialty was rated lowest in importance by residents pursuing hospitalist and general internal medicine careers. CONCLUSIONS: Greater attention to factors reported by residents as important to their career decisions may assist efforts to optimize the distribution of physicians across disciplines. In addition to lifestyle and practice considerations, these factors may include mentor specialty. As this factor is less commonly reported as important by residents planning careers in generalist fields, attention to effective mentoring may be an important element of efforts to increase interest in these areas.