Comparison of performance of final-year students from three Australian medical schools.

Between 1984 and 1986, 888 final-year medical students from The Universities of Sydney, Western Australia and Newcastle took part in trial examinations that comprised multiple-choice questions, modified essay questions and a patient management problem. Differences in student performance among the schools were small. In 1984 and 1985, Newcastle students performed less well than did Sydney students on multiple-choice questions that were prepared by The University of Sydney, but there was no difference between the schools in 1986. Sydney students performed better than did Newcastle students on the multiple-choice question paper that was prepared by The University of New South Wales in 1984, but in the last two years no differences have been detected between the schools in performance on this paper. The performance on modified essay questions in 1984 suggested that Newcastle students were stronger in behavioural sciences and weaker in pathological sciences than were Sydney students. Sydney students performed less well than did Newcastle students in the patient management problem in 1985, particularly in the area of the use of clinical investigations. On the one occasion of testing that involved students from The University of Western Australia (in 1985), these students performed best of the three schools in the patient management problem, and roughly equally with students from Sydney and Newcastle in the multiple-choice question papers. Differences among the schools usually amounted to less than 10% and might have been accounted for by differences in familiarity with test instruments. We conclude that medical students are likely to graduate from The University of Sydney, Western Australia and Newcastle with similar levels of knowledge of internal medicine. Possible differences in problem-solving ability require further study, particularly in the clinical setting.

Modified essay questions: are they worth the effort?

The method chosen for important examinations strongly influences the nature of student learning. The Newcastle Medical School in Australia developed a 5-year problem-based curriculum and adopted the Modified Essay Question (MEQ) as the main written instrument for assessing students' problem-solving skills. Even with the best of intentions the MEQ has been abused by its over-use, as shown in this review of annual assessment for years 1, 3 and 5 of the course. Analysis of staff and students' dissatisfaction with this instrument showed some difficulties attributable to the nature of MEQ items. However, a number of problems also emerged as a result of subtle changes in the teaching staff's approach to learning and assessment. This paper concludes by acknowledging the strengths and limitations of the MEQ. It recommends ways in which the MEQ can be effective, as part of a balanced programme involving other suitable instruments.

Evaluating a new approach to selecting medical students.

Dissatisfaction with the traditional methods of selecting Australian medical students, which use only secondary school achievement, led to an innovative alternative method at the Newcastle Medical School. This multistage approach uses tests of problem-solving ability, empathy, creativity and moral dilemmas to screen applicants otherwise suitable on academic achievement. In the 5-year trial since its inception, this process appears reasonably reliable and valid. There is some merit in using a composite score for ranking applicants, based on weighted contributions from the psychological tests used in its multifactorial battery. However, the ultimate effectiveness of individual tests of a composite score will depend on their predictive validity, which is yet unmeasured.

Problem solving in undergraduate medical students.

In recent years the systematic development of the skill of clinical reasoning has come to assume a high priority as an explicit aim of medical education. Clinical reasoning, it is contended, is the application of general reasoning and problem-solving skills to the specific knowledge base of medicine. The results presented in this paper constitute a preliminary study designed to investigate the ability of first-year medical undergraduates to solve abstract problems using a simple nonmedical knowledge base. Further studies are being carried out and will continue to extend this into the specific medical knowledge base area. The potential implications for medical education and the development of clinical reasoning are discussed.

Student perceptions of the learning environment in a new medical school.

Students' perceptions of their learning environment in the Faculty of Medicine of the University of Newcastle were tested in 1979, in the second year of the school's existence, when the first two cohorts of students were surveyed. It was thought important to re-test those same cohorts in 1982, when they were in the later years of the curriculum, to see whether their perceptions had changed, and also to test the perceptions of subsequent cohorts of students (still in the earlier years of the course) to examine whether the favourable perceptions of the earlier cohorts were being maintained. Accordingly, this paper reports the results of a cross-sectional study undertaken on all five years of Newcastle medical students in 1982, and it compares their perceptions with those obtained 3 years earlier.

Evaluating patients' satisfaction with medical students' interviewing skills.

Two studies are reported which examine the use and credibility of patients' reports of medical students' clinical interviewing skills. In the first study patients' high satisfaction via a standard rating form tended to match clinicians' assessment of second-year students' history-taking and physical examination skills. A second study used pre-defined 'ideal' and 'not-ideal' bedside roles by a small group of fourth-year students to examine how perceptive or tolerant hospital inpatients are about clinical interviews, whether this effect is sex-linked, and consequently how useful direct and indirect information obtained from patients is in training medical students' interviewing skills.

Assessment of independent learning.

In this series, we print evaluation instruments, questionnaires, rating scales and similar resource materials useful to teachers, evaluators and planners. You may wish to adapt such instruments for your own purposes. The feature in this issue has been prepared by Grahame I. Feletti, Nicholas A. Saunders, Anthony J. Smith and Charles E. Engel. Further information is available from Charles E. Engel.

Medical students' evaluation of tutors in a group-learning curriculum.

This study focused on students' evaluation of tutors' effectiveness during an academic term, using a nineteen-item questionnaire. The study took place within an integrated problem-solving medical course. Four major factors in the tutor's behaviour were identified as having importance in the rating process: the ability to care for students, a knowledge of course structure and teaching staff philosophy, the ability to encourage independent thinking in students and a knowledge of the specific medical problems being studied. Further analysis suggested that students distinguish between good and bad tutors, those in different medical strands and with differing status. In general, the findings complement those dealing with student evaluation of lectures and to some extent, patient satisfaction with doctors.

Clinical Supervisor's Report.

In this series, we print evaluation instruments, questionnaires, rating scales and similar resource materials useful to teachers, evaluators and planners. You may wish to adapt such instruments for your own purposes. The feature in this issue has been prepared by Nicholas A. Saunders, Charles E. Engel, and Grahame I. Feletti from whom further information is available.

Review of psychometric features of the Medical School Learning Environment Survey.

Growing concern over the stresses imposed on medical students by their undergraduate educational programme has led to a proliferation of surveys designed to measure students' global perceptions of the medical school as a learning environment. The present study aimed at a comprehensive analysis of the psychometric features of a recently-introduced questionnaire. Students at four Australian medical schools completed the Medical School Learning Environment Survey (Marshall, 1978). Data on the retest reliability, internal consistency and factorial validity of the MSLES are presented. Its potential usefulness for future evaluations within and between medical schools is discussed.

Reliability and validity studies on modified essay questions.

Modified essay questions (MEQs) have been developed at the University of Newcastle to assess clinical problem-solving by first- and second-year medical students. Reliability estimates (given as coefficient alpha [60]) as high as 0.91 were reported for term assessment in 1979 based on MEQs. Lower estimates can be expected as statistical artifacts due to the faculty's criterion-referenced assessment of competence. More appropriate measures of reliability are currently being evaluated. The validity of assessment by MEQ was based on one model of medical problem-solving and another of cognitive skill taxonomies. Internal consistency estimates (coefficient alpha) based on each model yielded some interesting anomalies on the nature of problem-solving which may be reexamined in terms of cognitive preference indexes.