Is there a relationship between attending physicians' and residents' teaching skills and students' examination scores?

PURPOSE: Faculty development programs and faculty incentive systems have heightened the need to validate a connection between the quality of teaching and students' learning. This study was designed to determine the association between attending physicians' and residents' teacher ratings and their students' examination scores. METHOD: From a database of 362 students, 138 faculty, and 107 residents in internal medicine, student-faculty (n = 476) and student-resident (n = 474) pairs were identified. All students were in their third year, rotating on inpatient general medicine and cardiology services, July 1994 through June 1996, at a single institution. The outcome measure for students' knowledge was the NBME Subject Examination in internal medicine. To control for students' baseline knowledge, the predictors were scores on the USMLE Step 1 and a sequential examination (a clinically-based pre- and post-clerkship examination). Teaching abilities of faculty and residents were rated by a global item on the post-clerkship evaluation. Faculty's ratings used only scores from prior to the study period; residents' ratings included those scores students gave during the study period. RESULTS: Multivariate analyses showed faculty's teaching ratings were a small but significant predictor of the increase in students' knowledge. Residents' teaching ratings did not predict an increase in students' knowledge. CONCLUSION: Attending faculty's clinical teaching ability has a positive and significant effect on medical students' learning.

The accountability of clinical education: its definition and assessment.

BACKGROUND: Medical education is not exempt from increasing societal expectations of accountability. Competition for financial resources requires medical educators to demonstrate cost-effective educational practice; health care practitioners, the products of medical education programmes, must meet increasing standards of professionalism; the culture of evidence-based medicine demands an evaluation of the effect educational programmes have on health care and service delivery. Educators cannot demonstrate that graduates possess the required attributes, or that their programmes have the desired impact on health care without appropriate assessment tools and measures of outcome. OBJECTIVE: To determine to what extent currently available assessment approaches can measure potentially relevant medical education outcomes addressing practitioner performance, health care delivery and population health, in order to highlight areas in need of research and development. METHODS: Illustrative publications about desirable professional behaviour were synthesized to obtain examples of required competencies and health outcomes. A MEDLINE search for available assessment tools and measures of health outcome was performed. RESULTS: There are extensive tools for assessing clinical skills and knowledge. Some work has been done on the use of professional judgement for assessing professional behaviours; scholarship; and multiprofessional team working; but much more is needed. Very little literature exists on assessing group attributes of professionals, such as clinical governance, evidence-based practice and workforce allocation, and even less on examining individual patient or population health indices. CONCLUSIONS: The challenge facing medical educators is to develop new tools, many of which will rely on professional judgement, for assessing these broader competencies and outcomes.

Medical students' self-assessments and their allocations of learning time.

PURPOSE: To investigate the impact of self-assessed diagnostic strengths and weaknesses on medical students' allocation of learning time (one indicator of self-directed learning) during a third-year internal medicine clerkship. METHOD: In 1997-98, 107 students at the University of Michigan Medical School self-assessed their diagnostic skills in 14 clinical areas before and after the clerkship and reported the relative amounts of time spent learning about these topics during the clerkship. RESULTS: Individual-level analyses indicated that, for the average student, self-assessed strengths and weaknesses did not correlate with allocation of educational time, but that time allocation was positively related to changes in self-assessed skill. Considerable variations in these relationships, however, suggest a need for closer study. CONCLUSION: Although individual students evidenced different levels of self-directed learning, this study suggests overall that students at this level of training are neophytes in applying both information generated through self-assessment and principles of self-directed learning in their clinical education. Attempts to advance students beyond this level depend on many factors, including the extent to which the learning environment encourages or even permits self-directed learning, the progression and time frame through which students become self-directed practitioners, and the impact of educational interventions to promote this development.

Oral staphylococcus in older subjects with rheumatoid arthritis.

OBJECTIVE: To determine if Staphylococcus aureus and Staphylococcus epidermidis, etiologic bacterial agents to late prosthetic joint infections (LPJI), are more prevalent in the oral flora of older individuals with rheumatoid arthritis (RA) than in an age and gender-matched nonarthritic control population (NA). DESIGN: Cultures were obtained from the nares, oropharynx, saliva, tongue, and gingival crevice, and the results were compared between older patients with RA and controls. SETTING: University of Michigan Medical Center, Ann Arbor, VA Medical Center, and University of Michigan School of Dentistry. PARTICIPANTS: A total of 111 community-dwelling subjects with a diagnosis of RA and 83 gender-matched control subjects. MEASUREMENTS: Colistin nalidixic acid agar plates with 5% sheep's blood were inoculated and incubated. Isolates were speciated using the API Staph Trac micro method and catalase and coagulase tests. MAIN RESULTS: Individuals with RA had a higher prevalence of S. aureus isolated from the oral cavity. However, only the oropharynx and tongue revealed higher rates; all other sites were insignificant. The presence of oral S. aureus was associated with xerostomia. Staphylococcus epidermidis was not detected from any of the oral sites sampled. Sixty-two percent (10/16) of the S. aureus isolates from the RA subjects were resistant to penicillin and ampicillin, whereas none were resistant to a cephalosporin. CONCLUSIONS: These findings suggest that rheumatoid arthritis may be a risk factor for LPJI in older prosthetic joint patients undergoing invasive dental procedure in the posterior oral cavity. This increased risk is caused, in part, by a higher prevalence of S. aureus in the posterior oral cavity. The prevalence and the antibiotic resistance of S. aureus must be considered when determining the need for chemoprophylaxis.

Assessment of a surgical pattern recognition examination.

BACKGROUND: Recent cognitive research in medicine has demonstrated that expert diagnosticians solve many clinical problems through "pattern recognition" rather than deductive reasoning strategies. Development of the ability to recognize and diagnose common surgical problems is a critical objective of undergraduate surgical education. A surgical pattern recognition examination (PAT) was developed to assess surgical diagnosis recognition in clerkship students. METHODS: Surgical faculty selected 18 diagnoses for 20 possible presenting complaints (eg, leg pain, soft tissue mass). A distinctive patient description was written for each diagnosis. An examination of 200 items was compiled covering 20 presenting complaints. RESULTS: The PAT was administered to clerkship students during the 1992-1993 academic year (n = 77). PAT scores ranged from 33% to 90% with a mean of 65% (SD = 12.7). PAT scores correlated highly with the National Board of Medical Examiners (NBME) Surgery Subject Examination (SSE) scores. The validity of the examination was assessed by a speeded administration of the exam to surgical house staff. Senior residents scored significantly higher than junior residents (mean 82% versus 63%, P = 0.004). Performance on the PAT was significantly affected by the order of the student rotation in the academic year (P = 0.02) while performance on the NBME SSE was not (P = 0.40), which is consistent with diagnostic ability improving with clinical experience. CONCLUSIONS: The surgical PAT examination is a new, reliable, and valid method for assessing diagnostic ability in third-year students.

Who will teach? A fundamental challenge to medical education.

The level of expertise possessed by medical school faculty members is unprecedented. Unfortunately, faculty members' broad understanding of their domains has atrophied as the specialization they need to compete successfully in the clinical and research arenas has increased. Medical students are novices, needing teachers who possess broad knowledge and experience, who can integrate the specific areas of a subject with overarching themes, and who can teach at the students' level. Clinical subspecialists and researchers on faculties often are ill-equipped to teach medical students. Likewise, busy clinical preceptors may no longer have the latest understanding of pathophysiology. The usual solution of bringing a series of basic science and clinical faculty members to classrooms and seminar rooms often results in disjointed coverage of material. Expanding the values of the university to once again include the scholarship of integration and teaching would provide the best type of faculty.

Effect of search experience on sustained MEDLINE usage by students.

PURPOSE: While educators agree that medical students should learn to use MEDLINE for clinical application, there is a lack of consensus on an optimal level of exposure to this resource during training that will result in sustained usage. This study sought to identify the level of search experience (1) to increase the odds that the student searcher will continue to search MEDLINE in the absence of search assignments, and (2) to make an appreciable difference in the odds of retrieving items of relevance from the MEDLINE database. METHOD: Search frequencies of MEDLINE via the PaperChase interface by 184 fourth-year students (class of 1992) at the University of Michigan Medical School were analyzed using the log cross-product technique. The students were required to take the Comprehensive Clinical Assessment, an examination that included a search assignment, as they entered their fourth year of medical school. Their levels of MEDLINE use and their retrieval performances before the examination were compared with those achieved during the subsequent five months as fourth-year medical students. RESULTS: For those who searched an average of at least once a month during their first three years of medical school, there was a 7.38:1 chance that they would conduct three searches per month in the fourth year, compared with those who searched less frequently. The odds of retrieving at least one item of definite relevance were 8.27:1 for those who had searched at least one and one-half times per month before the search assignment. CONCLUSION: Searching once a month through the first few years of medical school provided an experience level that improved the odds that a student would continue to search MEDLINE: Data indicated that a history of a minimum of 1.5 online sessions per month increased the odds of retrieving relevant items to 8.27:1. Implications for educational strategy are clear.

Assessment of a pattern-recognition examination in a clinical clerkship.

PURPOSE: To develop a pattern-recognition examination (called PAT) for use in assessing surgical clerkship students. METHOD: The PAT was developed in 1992-93 by the surgery faculty at the Wright State University School of Medicine. The top 30 diagnoses of patients seen by clerkship students in 1991-92 were identified, and possible presenting complaints were selected for each diagnosis. Then for each complaint faculty selected 18 common or catastrophic diagnoses, and a distinctive patient description was written for each diagnosis. An examination of 200 items was compiled covering 20 presenting complaints, each with a set of ten patient descriptions and 18 diagnoses from which to choose the most likely diagnosis. Scoring was by the percentage of correct responses. The PAT was given to 77 students during the 1992-93 clerkship along with other forms of evaluation. The PAT was also administered to 18 surgical residents. RESULTS: The students' mean score on the PAT was 65%, SD, 12.7. Split-half reliability was high (Spearman-Brown r = .91). The students' PAT performances correlated strongly with other cued-response testing formats, e.g., rs = .79, PAT versus the surgery subject examination of the National Board of Medical Examiners. Moderate correlations with essay scores (r = .49) and preceptors' ratings (r = .49) were found. The residents' PAT performances were significantly higher for senior than junior residents (82% versus 63%, p = .004). CONCLUSION: Utilizing this examination format, it is possible to locally develop a reliable examination that is grounded in the actual clinical problems seen by clerkship students and that assesses more than factual recall.