Medical students' clinical self-assessments: comparisons with external measures of performance and the students' self-assessments of overall performance and effort.

PURPOSE: To find predictors of students' initial self-assessments of their clinical performances, the predictive value of their self-assessments, and factors that relate to their final self-assessments. METHOD: Of the 142 third-year students at the University of Michigan Hospital in 1988-89, 137 (96%) completed a self-assessment questionnaire on the first day of their internal medicine clerkship and a slightly expanded questionnaire on the last day. The questionnaires asked the students to use Likert-type scales to rate their abilities regarding clinical skills, use of knowledge in the clinical setting, and discharge of patient care responsibilities. Also collected were data on the students' performances as measured externally: college grade-point averages; standardized examinations taken before, during, and after the clerkship; and ratings given the students by the faculty and residents with whom they worked in the clerkship. Pearson product-moment correlations were then calculated between the students' self-assessment ratings and their scores on the external measures of performance. RESULTS: Weak to absent correlations were found between prior-performance measures and initial self-assessments. The lower-performing students, as measured by college grade-point averages and Medical College Admission Test scores, tended to rate their performances higher than did their peers at initial self-assessment. IN contrast, the higher-performing students rated themselves lower than would be warranted given their prior performances. There were significant increases in the initial self-assessments as the year progressed and the students entered the clerkship after having had more experience. The correlations between the students' final self-assessments and the ratings by faculty and residents were generally weak. The strongest (.267, p < or = .001) concerned the students' medical knowledge. In addition, there was a moderate correlation (.413, p < or = .001) between the students' self-assessments of how hard they had worked and their self-assessments of overall performance. CONCLUSION: The findings in this study were similar to those of previous studies in that they showed relatively poor agreement between external measures of students' performances and students' self-assessments of their performances.

M NET: a statewide referring physician computer network.

M NET, the Referring Physician Computer Network, is a joint project between the University of Michigan Medical Center (UMMC) and IBM. A graphical user interface was developed to allow referring physicians easy access to a variety of patient, clinical, and institutional data. File transfer mechanisms were created and current clinical database files and programs were enhanced and restructured to facilitate remote data transmission. M NET is currently installed in referring physician office sites across the state, with additional physician sites identified and program enhancements under development.

The association between community physician's attendance at a medical center's CME courses and their patient referrals to the medical center.

This study attempts to quantify an overall association between CME course attendance and referrals. Attendance at formal CME courses given by the University of Michigan Medical School and referrals to the University Hospitals were examined over a two-year period. Attendance and referrals were linked to physicians in Michigan identified through the Michigan Department of Licensing and Regulation and through the American Medical Association. For physicians who are office-based and likely to be in active practice (age less than 70), those who attended at least one of the University's CME courses referred more patients than those who did not attend one (means of 1.9 referrals per physician and 1.3 referrals per physician, p less than .001). The causal direction of the relationship is not clear, but probably operates in both directions. It is reasonable for medical center marketers to consider CME as an indirect method for marketing clinical services. It is also reasonable for CME directors to identify referring physicians as high-priority groups for marketing CME. Both marketing efforts may be significantly enhanced by linking data bases for referrals and for CME attendance. CME directors must also ensure that marketing efforts do not compromise the objectivity and integrity of the content of the institution's CME program.

Problem-solving model in radiology for medical students.

Current undergraduate medical education is criticized for not preparing physicians to be independent thinkers. The rapid development of new imaging techniques and the problem of escalating medical costs call for efficient patient management. The development of algorithms in imaging work-up of patient problems is an excellent example of problem solving or medical decision making. The senior elective in radiology at our institution incorporates this type of problem-solving session. Small groups (15-25 students) with faculty guidance discuss 5-6 common patient problems to develop an investigative plan in imaging. Algorithms are thus developed by the group, but not presented for memorization. Small changes are then made in the case history so that the students are forced to make new hypotheses and generate a modified algorithm. Correlative costs are included. Flexibility and initiative in development of patient management algorithms are stressed.

House officer interviewing techniques: impact on data elicitation and patient perceptions.

The relationships among physicians' interviewing techniques, the amount and type of data gathered, and patients' perceptions of the interviewing process were studied. Thirty-one Internal Medicine house officers each interviewed one of three standardized patients. The house officers' thoroughness of data collection was assessed by the patients and by a trained evaluator. A videotape of each interview was analyzed at the National Board of Medical Examiners using the interaction analysis system for interview evaluation, ISIE-81, to define house officers' interviewing techniques. From the physicians' problem-solving perspective, data elicitation was positively related to the length of the interview, asking psychosocial questions, the use of narrow questions, and the amount of time the patient talked. The patients' assessments of house officers' data-gathering thoroughness were also positively influenced by interview length, the use of narrow questions, and inquiries about their psychosocial histories. The use of broad questions by the house officer was positively related to the patient's feelings about and reaction to the interaction. This study potentially explains some of the differences that appear to exist between patients' and physicians' judgments about and perceptions of the medical interviewing process.

The integration of computer conferencing into the medical school curriculum.

A computer conference was developed for use with students, faculty, and staff at a large state-funded medical school. The conference was intended to increase communication among these groups which study, work, and train at a variety of locations across the state. To date, 583 participants (91% are students, 6% are faculty, and 2% are staff) have discussed a variety of academic, ethical, social, and administrative issues. Conference use is required during the obstetrics-gynecology third year clerkship, and has also been used to evaluate courses and critique course examinations. The procedures for creating a computer conference, implementing its use in the curriculum, and evaluating its effectiveness are discussed.

Medical student attitudes toward radiology. A multi-institutional survey.

A questionnaire was administered to 238 students at four institutions before and after a radiology clerkship to evaluate student attitudes regarding radiology as a career, radiologist expertise, and clerkship expectations. There was no statistically significant difference among the clerkships, although they varied in types of experience, geographic location, and size. After the clerkship, the students agreed more strongly that the radiologist should be involved in patient workup, screening of requests, interpretation of emergency room films, and consultations. A significant change in attitude could be identified after the four-week experience, indicating a desire for the radiologist to have a more active role as imaging consultant.

The surgical clerkship experience: self, departmental, and institutional assessment.

The purpose of this study was to investigate to what extent students' assessments of their clerkship experiences varied in relation to perceptions of self-improvement, the time of year in which the clerkship experience took place, and the origin of the assessment device (departmental versus institutional). All students enrolled in each of six surgery clerkship rotations over the course of 1 academic year (224 students) were administered two questionnaires on completion of the clerkship. One questionnaire was institutionally developed and supported, whereas the other was departmentally sponsored. Results of multiple regression analyses indicated that (1) when students felt their medical skills had improved as a result of clerkship participation, the clerkship was considered of high quality; (2) students' perceived improvement in communication skills decreased as the academic year progressed; and (3) students did not alter their responses to the questionnaires based on whether the instrument was institutionally or departmentally sponsored.

Medical student attention to preventive medicine: change with time and reinforcement.

We followed medical students' attention to documentation in their write-ups of appropriate preventive medicine information, based upon a patient's age, sex, and existing medical conditions, as well as the translation of critical findings to the problem list, for three years. The proportion of relevant items documented was .50 in year one, .80 in year two, and .69 in year three. Significant differences (p less than .001) were found between all three years. Similarly, the proportion of important items translated to the problem list was .04 in year one, .22 in year two, and .18 in year three. There was significant improvement in years two and three as compared to year one (p less than .001). The impact of interventions designed to reinforce the medical student's attention to preventive medicine was also studied. Cued forms on which to record the patient's history were compared to written feedback regarding the student's write-ups. Both significantly improved student performance. The effect of the written feedback also persisted in the follow-up period.

Journal publications in radiology education. A review of the literature, 1966-1986.

Published scholarly activity pertaining to radiology education is relatively uncommon compared with an extensive clinical literature. To guide future radiology educational efforts in a time of changing priorities and practice patterns, the authors undertook an in-depth review and analysis of the existing radiology education literature covering the 20-year period from 1966 to 1986. A total of 194 articles was identified via a computerized literature search and manual cross-checking of cited references. Articles pertinent to medical student radiology education (36.6%) or radiology resident education (29.9%) predominated. The majority of published radiology education articles were editorials (35.5%) or expository in nature (49.5%). Surveys (5.1%) and experimental research (8.7%) comprised the balance. Although editorial opinions and personal reflections are important and valuable resources for radiology educators, a substantial lack of empirical research in radiology education exists. Additional knowledge in underdeveloped research areas is essential for future curriculum planning and development in radiology education.

Internal medicine house officers' performance as assessed by experts and standardized patients.

Three chronically ill patients were trained to evaluate the performance of 31 second-year internal medicine house officers based upon: a checklist for the medical data elicited during the medical interview; the process of the interview (that is, the house officer's interviewing style); and the physical examination technique. Four standardized rating scales were completed by the patients for the evaluations. Expert evaluations of the same house officers were conducted by a trained evaluator who reviewed videotapes of the interview and a faculty member who reviewed the adequacy of the medical history write-ups. Results of the study indicate a high correlation between the patients' and the experts' evaluations of the adequacy of the medical history data collected; less agreement was found in evaluations of the adequacy of the psychosocial data collected. These findings suggest that, if carefully selected and trained, patients with chronic medical conditions can provide reliable and standardized evaluations of house officers' performance at a level consistent with but less expensive than faculty members' evaluations.

A controlled experiment in teaching students to respond to patients' emotional concerns.

Medical education faces increasing criticism because of the perception that it is difficult to produce physicians who are both technically competent and compassionate. One approach to addressing this problem is to train physicians to address the emotional concerns that patients experience as a result of their medical problems. Results of the authors' randomized, controlled experiment in training medical students to respond to patients' emotional concerns provide evidence of the efficacy and feasibility of such training. Only students who received small-group instruction in addition to large-group lectures exhibited statistically significant (p less than .001) improvement in their ability to respond to the emotional concerns of patients in hypothetical cases. The students who received only the lectures exhibited significant improvement (p less than .001) in their preference for responses that addressed patients' emotional concerns but not in their responses to written, hypothetical patient cases. Even though these students' preferences for responses to emotional concerns on the average did increase significantly, the preferences were significantly (p less than .001) less favorable than the preferences of the students who received small-group instruction. The small group instruction consisted of both interviews with elderly nursing home residents, who served as simulated patients, and structured practice and review sessions.

Guidelines for writing multiple-choice questions in radiology courses.

Objective evaluation of students frequently includes the utilization of multiple-choice test questions. Due to constraints on faculty time, however, the writing of these questions is usually completed quickly and somewhat haphazardly. The guidelines contained in this article were written to help radiologists produce better test questions for the purpose of improving the evaluation of clerkship students. The guidelines include choosing an item that is worth testing and has one definite answer, asking a single, clearly formulated question, avoiding true-false stems, being concise when writing the question, avoiding the use of negatives in the stem, avoiding using "all of the above" or "none of the above" as a response, avoiding subtle clues toward the keyed answer or away from the distractors, avoiding one-word definitions as responses, and placing the responses in logical or numerical order. A definition of terms used in the educational literature and a discussion of a quantitative evaluation of test questions are included.

Constructing pre- and post-tests in a medical student elective.

The importance of evaluating medical student performance has been widely addressed in the medical education literature. Multiple-choice examinations are the preferred method of testing because they are easy to administer and score. Development of parallel or equivalent pre- and post-examinations provides objective information regarding actual knowledge gains by the students during the elective and points to strengths and weaknesses in the elective. This article provides information for developing the parallel multiple-choice questions for effective pre- and post-tests.

Oral examinations: actual and perceived contributions to surgery clerkship performance.

Controversy persists regarding the validity, utility, and practicality of oral examinations. This study of a recent 3-year experience at the University of Michigan Medical School was undertaken to determine the value of the oral examination in the required junior year surgery clerkship. Multiple regression analysis of the relationship between the oral examination and ward grades, written examination grades, and the final grades for the clerkship years 1980 to 1981 (n = 198), 1981 to 1982 (n = 234), and 1982 to 1983 (n = 215) (classes of 1982, 1983, and 1984) revealed significant correlation between the oral examination and other evaluation parameters. Stepwise regression analysis revealed that the oral examination contributed unique information to final grade determination. Two surveys identified faculty and student opinion regarding the oral examination. The results were consistent during the 3-year period of study. Faculty and student opinion regarding the examination as an evaluation process and motivational device were positive, as was opinion regarding the breadth and depth of knowledge tested and the examination as a learning experience and an accurate reflection of students' performance. Both faculty and students agreed that the examination was worth the time and effort invested by faculty, was an appropriate evaluation process, and was a source of worthwhile faculty and student interaction.