When enough is enough: a conceptual basis for fair and defensible practice performance assessment.

INTRODUCTION: An essential element of practice performance assessment involves combining the results of various procedures in order to see the whole picture. This must be derived from both objective and subjective assessment, as well as a combination of quantitative and qualitative assessment procedures. Because of the severe consequences an assessment of practice performance may have, it is essential that the procedure is both defensible to the stakeholders and fair in that it distinguishes well between good performers and underperformers. LESSONS FROM COMPETENCE ASSESSMENT: Large samples of behaviour are always necessary because of the domain specificity of competence and performance. The test content is considerably more important in determining which competency is being measured than the test format, and it is important to recognise that the process of problem-solving process is more idiosyncratic than its outcome. It is advisable to add some structure to the assessment but to refrain from over-structuring, as this tends to trivialise the measurement. IMPLICATIONS FOR PRACTICE PERFORMANCE ASSESSMENT: A practice performance assessment should use multiple instruments. The reproducibility of subjective parts should not be increased by over-structuring, but by sampling through sources of bias. As many sources of bias may exist, sampling through all of them may not prove feasible. Therefore, a more project-orientated approach is suggested using a range of instruments. At various timepoints during any assessment with a particular instrument, questions should be raised as to whether the sampling is sufficient with respect to the quantity and quality of the observations, and whether the totality of assessments across instruments is sufficient to see 'the whole picture'. This policy is embedded within a larger organisational and health care context.

Linking assessment to learning: a new route to quality assurance in medical practice.

BACKGROUND: If continuing professional development is to work and be sensible, an understanding of clinical practice is needed, based on the daily experiences of doctors within the multiple factors that determine the nature and quality of practice. Moreover, there must be a way to link performance and assessment to ensure that ongoing learning and continuing competence are, in reality, connected. Current understanding of learning no longer holds that a doctor enters practice thoroughly trained with a lifetime's storehouse of knowledge. Rather a doctor's ongoing learning is a 'journey' across a practice lifetime, which involves the doctor as a person, interacting with their patients, other health professionals and the larger societal and community issues. OBJECTIVES: In this paper, we describe a model of learning and practice that proposes how change occurs, and how assessment links practice performance and learning. We describe how doctors define desired performance, compare actual with desired performance, define educational need and initiate educational action. METHOD: To illustrate the model, we describe how doctor performance varies over time for any one condition, and across conditions. We discuss how doctors perceive and respond to these variations in their performance. The model is also used to illustrate different formative and summative approaches to assessment, and to highlight the aspects of performance these can assess. CONCLUSIONS: We conclude by exploring the implications of this model for integrated medical services, highlighting the actions and directions that would be required of doctors, medical and professional organisations, universities and other continuing education providers, credentialling bodies and governments.

Country report: Australia.

The last 10 years has been an interesting time for Australian medical education despite reduced funding. WORKFORCE: There are five main workforce trends: a rural/urban maldistribution, a need for more specialists, public hospital staffing difficulties, increasing female practitioners and under-representation of indigenous practitioners. ISSUES FACING THE DEANS: Lack of resources is a problem facing Deans, with pressure for clinical service in teaching hospitals. Entrepreneurial activities have been undertaken including the enrollment of overseas students. Medical schools have also responded to important government initiatives. DEVELOPMENTS IN MEDICAL EDUCATION: Australia's 11 medical schools have undergone significant reform in the last decade. There is a mix of four (graduate), five and six year courses. AUSTRALIA'S NEW MEDICAL SCHOOL: James Cook University opened the first medical school in northern Australia in 2000. The School admits students from rural, northern Australian and indigenous backgrounds. It has a strong regional mission. RURAL AND COMMUNITY-BASED EDUCATION: Government funding to address the maldistribution of the workforce has led to the establishment of rural clubs, Departments of Rural Health and community-based programs. THE FIRST TWO POSTGRADUATE YEARS: There have been recent moves to improve education in the two years following graduation. This includes the initiation of national projects in curriculum and assessment. POSTGRADUATE AND CONTINUING MEDICAL EDUCATION: Postgraduate programs in Australia are being reformed to build on the changes in undergraduate education. CME is also under review. CONCLUSION: Australian medical educators should build on the recent reforms and take on some of the new directions in medical education.

Continuing medical education in clinical nutrition.

To achieve better promotion and acceptance of clinical nutrition at all levels of public and medical education requires involvement in educational services based on sound educational theory and practice. Adult learning theory has established principles that support effective learning which, in turn, influence people's attitudes and behaviour. Application of these principles in Continuing Medical Education implies that as much emphasis must be placed on the educational skills necessary to produce self-directed lifelong learners as on clinical content. This article presents guidelines to assist in the planning and implementation of activities in clinical nutrition, and offers illustrative examples from other disciplines.

An interdisciplinary course unit in basic pharmacology and neuroscience.

A newly developed course unit for second-year medical students entitled 'Introduction to Pharmacology and Neuroscience' is described. Novel features of the course relate to its interdisciplinary approach to teaching of topics traditionally taught in separate departmental-based courses. An evaluation of this unit was conducted, in order to provide information to be used for decision-making and improvement and development of the ongoing programme. The evaluation involved a number of interviews with those involved in the course, including course convenors, teachers of the course, students, and members of the Medical Faculty Curriculum Review Committee. A questionnaire was also administered to students. As a result of this evaluation, a set of recommendations for future changes were made. A majority of the students interviewed were in favour of the integration of basic pharmacology and neuroscience, as in this course unit.

Using case methods effectively in clinical medicine.

This paper reviews the potential advantages of using case methods in clinical medicine, focusing on paediatric medicine as an example. Case methods describe possible approaches based on realistic case material which allow the teaching-learning process to proceed. Cases may be used to illustrate a disease or, alternatively, the care of the patient with the disease. It is possible to observe and demonstrate data collection, appropriate interviewing, examination and interpersonal skills. The clinician may take on a role model for the students to emulate, his pastoral role increasing as he gets to know his students better. It is essential that the clinician observe and verbalize the diagnostic and clinical decision-making processes of the students or himself. Case methods may not always provide complete data for each area under consideration, and on occasions the clinician may not be fully conversant with the content of the problem at hand. Students would need to learn how to function in such situations, being set specific learning tasks to acquire the necessary knowledge and skills. The case material thereby provides a focus to problem-based learning.

Role play for the clinical tutor: towards problem-based learning.

A method of helping students learn paediatrics was developed utilizing role play as a stimulus to problem-based learning to arrive at a diagnosis. The tutor briefly stated a common paediatric problem and then took on the role of the patient and/or child, allowing himself to be interviewed by students working co-operatively in a group. This process was interrupted by the students to discuss their progress and to determine what further information was required. A suitable patient was available to be examined if requested by the students. Although there was an initial tendency on the part of students to regard the exercise as 'unreal', they delighted in refining their communication skills and trying out their skills in problem solving. The tutor had to adopt a non-traditional role, as he was not directing the students or interrupting, so as to 'teach' the students more efficient information-gathering techniques or to correct any error made. The students were able to define and complete set tasks which they derived out of the session. The introduction of this technique encourages independent learning and the acquisition of problem-solving skills, and the students enthusiastically accepted this novel way of learning. The ready availability and versatility of the tutor allowed him to be used as the principal resource, and did away with the need for (and the additional expense of) specially constructed problem-based learning units, often unavailable in most departments. The method is described in detail so that other clinical tutors may feel encouraged to adopt a similar strategy.

A review of the final F.F.A.R.A.C.S. examination 1980-1987.

The Final Examination for Fellowship of the Faculty of Anaesthetists, Royal Australasian College of Surgeons has been functioning since 1956. A description of its development and an analysis carried out on the examinations in 1969 and 1970 are contained in the paper by Fisk et al. Since that time the examination process has been continually monitored and refined, although the basic structure has remained much the same for the last twenty years. This report presents an analysis and interpretation of data for 1025 candidates who presented themselves for the examinations from May 1980 until October 1987. Recent development within the Faculty is described to indicate the influence these events have had upon the training and examination of candidates for the Fellowship, and statistical data is provided to illustrate the effectiveness of the examination process. Recommendations include possible directions that could be taken to enhance both the training and examination of candidates for the Fellowship examination in the future.

Implications for vigilance of a method for evaluating patterns of change in heart rate during anaesthesia.

Instantaneous heart rate was recorded in five patients and analysed for change in rate using the criterion of percentage variation either side of the starting rate with resetting throughout the anaesthetic each time the permitted variation was exceeded. The number of occasions on which change occurred was counted, the patterns of change were analysed and the effect of two sampling methods (six-second sample and six-second average) on these patterns was compared. A large number of patients (131) was investigated using the six-second sampling method and the number of changes and patterns of change were evaluated. It is suggested that the six-second sampling technique, combined with the 10 or 15% variation, may be useful for the evaluation of vigilance, and has implications for the design of monitoring apparatus in the future.

Factors affecting an anaesthetist's work: some findings on vigilance and performance.

This paper reviews factors which may influence the vigilance tasks of an anaesthetist during an anaesthetic. Vigilance tasks are found to be unlike any other automatic, repetitive or monotonous tasks. While many factors affect vigilance and may result in performance decrements during psychological testing, little of this work has been done on anaesthetists. We suggest, however, that there are many parallels between findings related to vigilance in general and the type of work done by anaesthetists. Implications of major factors are discussed, and recommendations for further study are proposed.

Teaching anaesthetics in the operating theatre.

During an anaesthetic procedure the anaesthetist's main concern is for the patient and his vigilance ensures that the patient is given the best care possible. When a trainee anaesthetist is administering an anaesthetic a tutor is often present to further improve the trainee's practical knowledge or technique. This report presents the results of an investigation of the typical patterns of trainee anaesthetist's behaviour when a tutor is either present or absent in order to establish whether the teaching which occurs in the operating theatre affects the pattern of activity and vigilance. Results indicate that the patterns of behaviour are unaffected by a tutor's presence, and that teaching anaesthetics in the operating theatre may be a legitimate activity which does not interfere with the trainee's prime function of patient care.

Tutor-student interaction in the operating theatre.

This paper examines the view that productive teaching in the operating theatre requires a recognition of the unique set of factors which may influence the process of communication. For tutor-student interaction to occur in a productive manner, the tutor's behaviour will need to be modified to suit the demands of the other tasks in which both tutor and student are involved. The particular value of recognizing the role of nonverbal behaviours during interaction is discussed, with the use of specific examples related to the positioning of the tutor and student, and their use of facial expression, the eyes and voice.

Teaching and learning in the operating theatre.

The anaesthetist's role during an anaesthetic is one of vigilance. Any interaction between tutor and student in this situation must be structured around the vigilance task. Teaching that recognizes this role may be educationally fruitful and may even improve vigilance. Teaching which disregards this function is less likely to result in learning and may interfere with patient care. Some of the types of teaching that are likely to be appropriate or inappropriate to different phases of an anaesthetic are indicated.