Linking assessment to learning: a new route to quality assurance in medical practice.

BACKGROUND: If continuing professional development is to work and be sensible, an understanding of clinical practice is needed, based on the daily experiences of doctors within the multiple factors that determine the nature and quality of practice. Moreover, there must be a way to link performance and assessment to ensure that ongoing learning and continuing competence are, in reality, connected. Current understanding of learning no longer holds that a doctor enters practice thoroughly trained with a lifetime's storehouse of knowledge. Rather a doctor's ongoing learning is a 'journey' across a practice lifetime, which involves the doctor as a person, interacting with their patients, other health professionals and the larger societal and community issues. OBJECTIVES: In this paper, we describe a model of learning and practice that proposes how change occurs, and how assessment links practice performance and learning. We describe how doctors define desired performance, compare actual with desired performance, define educational need and initiate educational action. METHOD: To illustrate the model, we describe how doctor performance varies over time for any one condition, and across conditions. We discuss how doctors perceive and respond to these variations in their performance. The model is also used to illustrate different formative and summative approaches to assessment, and to highlight the aspects of performance these can assess. CONCLUSIONS: We conclude by exploring the implications of this model for integrated medical services, highlighting the actions and directions that would be required of doctors, medical and professional organisations, universities and other continuing education providers, credentialling bodies and governments.

Development of a computerized telecommunication system for in-training evaluation of residents in a laparoscopic educational program.

OBJECTIVE: To describe our initial experience with a computerized telecommunication system, termed the interactive voice-response system, to record resident performance of laparoscopic surgery. METHODS: After completing a laparoscopic procedure, the surgeon and resident telephone a toll-free number independently and respond to three prerecorded statements using a Likert scale of 1 to 5. The caller then is asked to describe the resident's response to critical incidents or elements of surprise that arose during the surgery. The ratings and verbal comments are compiled, transcribed, and forwarded to the respective resident. The resident (and program director) can hear the verbal comments by entering a four-digit code. RESULTS: Between May 1, 1995, and May 31, 1996, 430 cases were reported by 11 surgeons and 16 residents using the interactive voice-response system. One hundred ninety-five (45%) procedures were entered by both the resident and surgeon. A survey undertaken during the introductory phase of the project revealed that five of the seven residents exposed to the system found that it provided useful feedback and preferred the system to traditional in-service reporting methods. In addition, five residents thought that the system complemented the personal feedback they received in the operating room. CONCLUSION: The system has been accepted by both residents and surgeons and has addressed the important components of resident in-training evaluation, namely, evaluation on a case-by-case basis, timely feedback, and self-assessment of resident performance.

Physicians Innovation Awareness Program: helping radiologists learn about innovations.

The Physicians Innovation Awareness Program (PIAP) was designed to study the strategies of Canadian radiologists implementing innovations into their practices and to use the information to design an educational program to help radiologists learn about innovations. Canadian radiologists were asked, by means of a mail survey, for information about innovations in practice adopted in the previous 2 years and planned for the next 2 years. Surveys were sent to the 1077 practising radiologists registered in the Canadian Association of Radiologists (CAR) as of Sept. 30, 1991; 325 (30.2%) responded. Most (281) of the responding radiologists were practising in communities with a population of 50,000 or more; 103 of the 188 respondents for whom faculty status was known were affiliated with a university. Most of the respondents reported that they spent up to 25% of their working time in at least one subspecialty pursuit. From the information in the surveys, an educational strategy was designed to help radiologists in learning about innovations. This program involved special sessions on six selected topics presented at two annual meetings of the CAR, as well as publications, mailings and other lectures. The authors conclude that a specialty society can obtain adequate information from its members to design an educational program to help practitioners in adopting innovations.

Adoption of innovations by Canadian radiologists.

The purpose of this study was to determine the factors that help or hinder the adoption of new procedures and practices by Canadian radiologists. Canadian radiologists were asked, by means of a mail survey, for information about innovations in practice adopted during the previous 2 years or planned for the next 2 years. Surveys were sent to the 1077 practising radiologists registered in the Canadian Association of Radiologists as of Sept. 30, 1991; 325 responded (30.2%). The responses were correlated with demographic information obtained in the same survey and through the Maintenance of Competence Program of the Royal College of Physicians and Surgeons of Canada. Magnetic resonance imaging (MRI) was more likely to be adopted in larger communities (those with a population of more than 500,000), computed tomography in medium-sized and larger communities (more than 100,000), mammography in medium-sized and smaller communities (500,000 or less) and ultrasonography (US) in smaller communities (100,000 or less). Radiologists with a faculty appointment at a university were more likely to adopt MRI and digital imaging, whereas those with no faculty status were more likely to adopt US and structural changes to a practice. The authors conclude that the size of the community in which a practice is located and the practitioner's faculty status both play a role in the adoption of innovations.

The Maintenance of Competence (MOCOMP) Program: motivating specialists to appraise the quality of their continuing medical education activities.

The goal of the Maintenance of Competence (MOCOMP) Pilot Project is to develop a comprehensive CME strategy that will motivate specialists to continuously update their clinical practice. In its 1st year the pilot program has taken several significant steps. A credit system has been implemented to facilitate recognition of CME of the highest educational quality and to encourage specialists to compare their CME efforts with those of their colleagues. The self-directed CME curriculum currently followed by the typical MOCOMP Program member has been described. A diary has been implemented for specialists to record CME activities and their potential impact on practice. The MOCOMP Program is the first attempt to motivate self-directed continuing medical education (CME) through the use of a diary and the first attempt to use this instrument to encourage critical appraisal of personal CME habits.

A randomized controlled trial of early amniotomy.

OBJECTIVE-To determine if a policy of early amniotomy resulted in a reduction in mean labour duration when compared to a policy of conservation of the membranes. DESIGN-A single-centre randomized controlled trial. SETTING-A tertiary care teaching hospital in Alberta, Canada. SUBJECTS-Ninety-seven term nulliparae in spontaneous labour, baby in cephalic presentation. INTERVENTION-Early amniotomy versus intent to keep membranes intact. MAIN OUTCOME MEASURES-Interval from randomization to delivery, rate of abnormalities of fetal heart rate tracings, cord artery blood pH, Apgar scores. RESULTS-The mean interval from randomization to delivery was 390.9 (SE 29.1) min in the amniotomy group and 442.9 (SE 34.1) min in the control group (P = 0.251). There were no differences between groups in the occurrence of fetal heart rate tracing abnormalities, nor was there a difference in the proportion of babies with abnormal Apgar scores, or abnormal cord pH (less than 7.20). CONCLUSION-The results of the study fail to support the long held belief that early amniotomy is an effective method for reducing labour duration.

A medical information networking system between practitioners and academia.

This paper reports on a one-year experience with an information networking system (MIS) between 47 rural practitioners and an academic center. Physicians were invited to phone in non-emergency clinical questions specific to daily practice needs to a telephone answering service located in the medical school library. Two-hundred-forty questions triggered by patient visits, colleagues, local rounds, allied health or local professionals, and on-site administrative meetings were forwarded to the MIS. All inquiries were classified according to the International Classification of Disease-9th Revision-Clinical Modification, and categorized into three areas of practice: diagnostic/investigative, general treatment, and pharmacology (therapeutics). The paper outlines how specific practice questions are being screened and adopted for decisions relating to four current activities which assist the ongoing maintenance of competence: 1) CME program planning, 2) residency/undergraduate curriculum development, 3) individualized CME for specific practitioners and sites, and 4) future CME research. The physician inquiries represent true needs in rural medical practice and as such should be given high priority in programs and assessments addressing the maintenance of competence.

Practice-generated questions: a method of formulating true learning needs of family physicians.

The importance of identifying true learning needs prior to planning and conducting educational programs is well documented. In this study a collection of 579 clinical questions forwarded by 87 family physicians in southern Alberta was reviewed and analysed to determine if the questions would cluster and, therefore, be useful as sources for planning continuing medical education (CME) conferences and newsletters. The nature of submitted questions was also examined to determine if particular groups of physicians, as identified by socio-demographic characteristics, significanly differed by type of questions submitted. The study findings indicate that the questions generated by the clinical practices of family practitioners did cluster into meaningful categories, and did vary significantly by identified physician groups. Questions relating to the genito-urinary system; adverse drug effects, interactions and contraindications; and nervous system and sense organs were most frequently submitted. The nature of questions generated differed significantly by gender, size of practice setting, and year and place of medical graduation. When highly relevant and specific clinical questions, as contrasted with broadly expressed needs, serve as sources of identified learning needs for CME planners, the daily clinical problems faced by practitioners are better addressed. The learning needs identified by this approach provide an optimum framework for planned CME to influence physician behaviour or practice patterns.

Nurses' perceptions of inter- and intra-professional relationships: an analysis of support from nursing and physician colleagues in rural hospitals.

Nurses employed in ten rural hospitals in Canada completed questionnaires designed to examine their perceptions of the strength of the professional relationship which existed among nurses and between nurses and physicians. Four questions addressed the availability of nursing education programs and the attendance of nurses and physicians at such programs. Seven questions examined the nurses' perception of physician interest in nursing education and the encouragement received from their nursing colleagues to participate in learning activities and medical management decision making. Chi square analysis was undertaken to determine whether the colleagial relationships differed by hospital or medical staff characteristics. Forty percent of the nurses felt that the majority of of physicians were supportive of continuing nursing education. Sixty-three percent indicated their nursing colleagues encouraged them to attend educational programs. The results indicate that physician participation in nursing continuing education was greatest in communities with the fewest health resources (an acute hospital with fewer than 50 beds). In contrast, encouragement to participate in continuing nursing education was highest in settings where the hospital provided both acute and extended care services, had more patient admissions and a greater availability of educational programming.

Teleconferencing cme programs to rural physicians: the university of calgary teleconference program.

Continuing medical education (CME) provides practising family physicians with the cornerstone of maintenance and improvement of skills. In rural areas the problems of isolation and distance are a barrier to continuing medical education. Provision of CME programs by audio-teleconferencing is an attempt to overcome these problems. This article describes the teleconference program of The University of Calgary, how it has developed over the five years of its existence, and its impact on rural physicians.

The role of standardized risk assessment in the provision of prenatal care.

Errors in the provision of prenatal care are more likely to be the result of mental saturation than of inadequacies in the physician's fund of knowledge. To eliminate medical errors the physician should commit more time to the processing of clinical data. There are, however, limits to the physician's ability to handle large amounts of information in a busy clinic. This article outlines the benefits and limitations of risk scoring, and recommends the use of standardized risk assessment as a screening device to detect those pregnancies in need of further investigation or consultation with a specialist.

Case studies in electronic fetal heart rate monitoring.

Subtle changes in the characteristics of the fetal heart rate are currently used to assess the condition of the fetus in late pregnancy and during labour. The authors present three case studies of fetal heart rate monitoring.

A conjoint medical/nursing diabetes education program: impact on care in a rural community.

This paper describes the planning, teaching format, and evaluation of an interdisciplinary education program in diabetes mellitus for physicians and nurses in a rural Canadian community. Most participants preferred the conjoint teaching format to the unidisciplinary didactic programs traditionally offered. An evaluation, by chart review, of the management of diabetic patients cared for before and at 3, 6, and 12 months after the teaching program revealed significant changes in the referral practices of both the home care nurses and the physicians. Our findings suggest the occurrence of a program-related change in collaborative care practiced by the physicians and nurses who participated. A more extensive three year project has been initiated to further document the effectiveness of interdisciplinary continuing education in this community.