The impact of communication between physician and evaluator on assessments of clinical performance.

Scores, ratings of score validity, and the precision of scores were compared for four methods of assessing clinical performance by case review that differ primarily in the communication allowed between physician and assessor. Results indicate that, at least for board-certified physicians, scores and ratings of score validity are significantly improved by increased communication.

The effects of taking a practice examination on scores in the Qualifying Examination of the Medical Council of Canada.

A practice examination has been prepared annually at the University of Toronto for use by those who will take the Qualifying Examination of the Medical College of Canada. Passing the Qualifying Examination is an important step in gaining the Licentiate of the Medical College of Canada (the LMCC) and thus in becoming licensed. The effect of taking the practice examination on scores on the LMCC exam was investigated. The practice examination is a smaller replica of the LMCC exam, with questions spanning major medical specialties presented in both multiple-choice and patient management problem (PMP) format. It was conjectured that the effect of practice would be greater on the PMP score than on the multiple-choice score. Analysis of covariance was used to compare the LMCC examination scores of senior students, who had taken the practice examination, with the scores of those who had not. The comparison was performed for two graduating classes, separately, and involved the use of a score summarizing each student's performance in the third year of the curriculum as a covariate. In each graduating class, the practice effect on PMP scores is statistically significant. In neither class alone is the practice effect on multiple-choice scores statistically significant, although a combination of results in the two classes is statistically significant. The presence of practice effects can be taken to indicate a flaw in the testing process. The elimination of practice effects, through experiences like the practice examination, would make the entire testing process more valid.

Evaluation of the 1977 CREOG in-training examination.

Questionnaires dealing with the 1977 in-training examination for residents in obstetrics and gynecology were sent to residency directors and residents. More than 70% were returned. Responses indicated a generally positive view of the examination, especially the multiple-choice questions (MCQs). There was strong support for the retention of patient management problems (PMPs) and feedback, but there was significant criticism of the format for feedback and the appropriateness of PMP options. Correlational analysis indicated that in evaluating the examination, residents emphasized the quality of the MCQs and feedback, while residency directors based their evaluation on a broader range of examination qualities. The analysis suggested that the questionnaire results were generally representative of the views of residency directors and residents.

A monitor for the medical curriculum.

A set of standardized questionnaires that provides a monitor for the operation of the medical curriculum has been developed at Michigan State University's College of Human Medicine.