The importance of measuring competency-based outcomes: standard evaluation measures are not surrogates for clinical performance of internal medicine residents.

BACKGROUND: Despite recent emphasis on educational outcomes, program directors still rely on standard evaluation techniques such as tests of knowledge and subjective ratings. PURPOSES: To assess the correlation of standard internal medicine (IM) residency evaluation scores (attending global evaluations, In-Training examination, and Mini-Clinical Examination Exercise) with documented performance of preventive measures for continuity clinic patients. METHODS: Cross-sectional study of 132 IM residents attending an IM teaching clinic, July 2000 to June 2003, comparing standard evaluations with chart audit. RESULTS: Mean resident performance ranged from 53% (SD = 24) through 89% (SD = 20) across the 6 preventive measures abstracted from 1,102 patient charts. We found weak and mostly not significant correlations between standard measures and performance of preventive services. CONCLUSIONS: Standard measures are not adequate surrogates for measuring clinical outcomes. This supports the Accreditation Council for Graduate Medical Education's recommendations to incorporate novel Toolbox measures, like chart audit, into residency evaluations.

Measuring resident physicians' performance of preventive care. Comparing chart review with patient survey.

BACKGROUND: The Accreditation Council for Graduate Medical Education has suggested various methods for evaluation of practice-based learning and improvement competency, but data on implementation of these methods are limited. OBJECTIVE: To compare medical record review and patient surveys on evaluating physician performance in preventive services in an outpatient resident clinic. DESIGN: Within an ongoing quality improvement project, we collected baseline performance data on preventive services provided for patients at the University of Alabama at Birmingham (UAB) Internal Medicine Residents' ambulatory clinic. PARTICIPANTS: Seventy internal medicine and medicine-pediatrics residents from the UAB Internal Medicine Residency program. MEASUREMENTS: Resident- and clinic-level comparisons of aggregated patient survey and chart documentation rates of (1) screening for smoking status, (2) advising smokers to quit, (3) cholesterol screening, (4) mammography screening, and (5) pneumonia vaccination. RESULTS: Six hundred and fifty-nine patient surveys and 761 charts were abstracted. At the clinic level, rates for screening of smoking status, recommending mammogram, and for cholesterol screening were similar (difference <5%) between the 2 methods. Higher rates for pneumonia vaccination (76% vs 67%) and advice to quit smoking (66% vs 52%) were seen on medical record review versus patient surveys. However, within-resident (N=70) comparison of 2 methods of estimating screening rates contained significant variability. The cost of medical record review was substantially higher ($107 vs $17/physician). CONCLUSIONS: Medical record review and patient surveys provided similar rates for selected preventive health measures at the clinic level, with the exception of pneumonia vaccination and advising to quit smoking. A large variation among individual resident providers was noted.

Improving the quality of clinical care for patients with osteoporosis.

Osteoporosis, leading to hip fractures and other fragility fractures, is prevalent in countries with a high life expectancy. Fractures have a high associated societal cost and disease burden. As a result, many countries have established screening guidelines for osteoporosis, especially targeting those at a higher risk. However, primary- and secondary-fracture prevention is currently suboptimal despite existence of effective medications. In addition, adherence to therapy is low. This special report outlines some of these challenges and evaluates different techniques for improving the quality of clinical care for patients with osteoporosis.