Reliability and acceptance of the mini-clinical evaluation exercise as a performance assessment of practicing physicians.

BACKGROUND: The purpose of this study was to evaluate the reliability and acceptance of the mini-Clinical Evaluation Exercise (mini-CEX) as an assessment of practicing primary care physicians. METHOD: Six raters were recruited to conduct the assessments. After a training session, their ability to discriminate between levels of performance was evaluated using videotaped clinical scenarios. Fifteen physicians were assessed in an office setting by the raters who scored multiple clinical encounters using a validated mini-CEX form for each encounter. Participants were given a postassessment survey regarding the process. RESULTS: Raters distinguished between performance levels on the videotaped scenarios (P < .001). A total of 188 physician-patient interactions were assessed. The generalizability coefficient for 10 encounters was 0.92. In the postassessment survey, the raters (94%) and physicians assessed (75%) both felt that the mini-CEX is an acceptable assessment. CONCLUSIONS: The mini-CEX seems to be a reliable and acceptable instrument for the assessment of practicing physicians.

The impact of a personal digital assistant (PDA) case log in a medical student clerkship.

BACKGROUND: Medical education literature emphasizes that reflection and self-audit are pivotal steps in learning and that personal digital assistants (PDAs) have potential as decision support tools. DESCRIPTION: The purpose was to examine the efficacy of PDA-based resources and patient-encounter logging systems among 3rd-year medical clerks during pediatrics rotations. EVALUATION: Students in rotations were assigned to control (using paper-based logs and references) or intervention groups (using PDA-based logs and resources). Students completed pre- and postrotation Paediatrics Competency Surveys, participated in focus groups, and were compared on year-end examination grades. Use of PDA logs far outweighed that of paper logs (1,020 PDA logs and 87 paper logs). PDA logs were ranked significantly higher in enhancing learning and reflection than paper logs (t = 2.52, p < .01). PDA logs also facilitated specific learning experiences. CONCLUSION: PDA-based patient-encounter logs appear to be effective case documentation and reflection tools. The difference in number of logs between control and intervention groups demonstrates the utility of the PDA for "point-of-care" patient logging.

It's about TIME: a general-purpose taxonomy of subjects in medical education.

CONTEXT: Modern computer technology permits the creation of detailed, dynamic electronic curriculum maps to facilitate curriculum searching, organisation and quality assurance. However, when attempting to map curricular content, a common question to arise is: 'To what should we map our curriculum?' With respect to content (i.e. the subject being taught, learned or examined), mapping to terminal outcomes or competencies may be too broad, whereas mapping to learning objectives is too specific. METHODS: To address this problem, the authors created TIME-ITEM (topics for indexing medical education; en Français: index des thèmes pour l'éducation médicale), a hierarchical taxonomy of topics relevant to medical education. It is a general-purpose, intermediate-granularity, standardised index that covers the entire range of subject matter in medical education. The content and structure of topics within TIME was developed in consultation with medical educators and librarians at several Canadian medical schools. As far as possible, the language used is standardised to the Unified Medical Language System. RESULTS: TIME is available as a web application that allows users from various schools to enter their school-specific outcomes, competencies and learning objectives, and then link these to the standardised topics in a way that is meaningful to the school. The entire TIME content and structure can then be exported, via xml, to external applications and used as an index for curriculum mapping, meta-tagging learning objects, or categorising examination questions. TIME can be viewed at http://www.time-item.org (username: 'guest'; password: 'guest').

TIME as a generic index for outcome-based medical education.

TIME (Topics for Indexing Medical Education) is a general-purpose, intermediate-granularity taxonomy of topics that describe the content of undergraduate medical education. Within outcome-based education systems, curriculum planning focuses on the desired product rather than process, and the contributions of curricular elements to achievement of the outcomes must be made visible. In this paper, we discuss how TIME may be used as a content index in curriculum maps to link curricular elements to multiple outcome frameworks. This assists with curriculum development and evaluation, quality assurance, curriculum searching, detection of gaps and redundancies, and sharing of educational objects. TIME is available at http://www.time-item.org (username "guest"; password "guest").

Commitment to change instrument enhances program planning, implementation, and evaluation.

INTRODUCTION: This study investigates the use of a commitment to change (CTC) instrument as an integral approach to continuing medical education (CME) planning, implementation, and evaluation and as a means of facilitating physician behavior change. METHODS: Descriptive statistics and grounded theory methods were employed. Data were collected from 20 consecutive CME programs. Physicians were asked to list up to three things they intended to change in their clinical practice as a result of the program. A copy was sent 3 weeks later as a reminder. Six months later, a summary of peer-intended changes was sent to reinforce intended behavior change. RESULTS: Of 602 participants, 291 (48%) completed CTC forms, resulting in 803 citations. Responses were congruent with the educational objectives and intentions of the program planners. Using the constant comparative method of analysis, a framework was identified for interpreting physician learning strategies. It included change strategies and motivation, learning issues, better doctoring, changes to clinic practice, and diffusion. DISCUSSION: CTC was useful as a multipurpose tool providing planners with meaningful feedback to (1) assess congruence of intended changes in physician behavior with program objectives, (2) document unanticipated learning outcomes, and (3) enable and reinforce intended behavior change.

MEDICOL: online learning in medicine and dentistry.

OBJECTIVE: MEDICOL (Medicine and Dentistry Integrated Curriculum Online) provides a variety of Web-based resources that act as important adjuncts to all the teaching components of the medical and dental undergraduate curriculum. It uses WebCT, a course-management system, to provide the following educational functions: (1) track students' progress and present course information such as time-tables, learning objectives, handout materials, images, references, course assignments, and evaluations; (2) promote student-to-student and student-to-instructor interactions (through e-mail and bulletin boards); and (3) deliver self-directed learning components, including weekly self-assessment quizzes that provide immediate feedback and multimedia learning modules (clinical skills, radiology, evidence-based medicine, etc.). DESCRIPTION: The University of British Columbia Faculties of Medicine and Dentistry feature a problem-based learning (PBL) curriculum in which students access many of the same tools they will utilize in their professional practice. In the PBL curriculum, students must access the relevant clinical data and educational resources. A MEDICOL site has also been developed for medical students to use during their rural family practice, a four- to six-week experience in the summer after their second year. This site has been designed to be a supplemental learning environment for not only these students, but also for their physician preceptors. It is intended to foster communication among participants, bring new resources to the rural setting, and allow preceptors to develop their Internet skills with the help of students who are already familiar with the electronic environment. The MEDICOL sites enable the exchange of information about the learning issues between, as well as within, tutorial groups. MEDICOL also provides students with faculty-reviewed resources that are listed online; multimedia presentations; and access to histology, radiology, and pathology images through an online image database. Each week, students have access to a new interactive and automatically graded self-assessment quiz for individual study. These quizzes test learning objectives from tutorial, lecture, and lab material for each week of the curriculum and are modeled after summative examinations held twice each year. Question authors provide immediately accessible quality feedback to students. A comprehensive quiz databank of approximately 1,500 questions has been attained. WebCT enables MEDICOL to deliver anonymous, online program-evaluation questionnaires during clinical clerkships (resulting in a 99% response rate after a few e-mail reminders), with easy and timely data collection and reporting methods. Summative assessments have also been delivered through MEDICOL. DISCUSSION: Use statistics indicate that over 90% of students regularly use the MEDICOL sites and have found them helpful. University of British Columbia medical school enrollment will increase because of collaborations with campuses and medical centers across the province. MEDICOL will likely play an increased role in distance learning by continuing to deliver the resources already described, as well as facilitating synchronous communications (e.g., PBL chat rooms) and teaching (e.g., video-streamed lectures) to students located across the province.