Shortcomings in the evaluation of students' clinical skills and behaviors in medical school.

The authors review the methods by which U.S. medical schools have evaluated student achievement during the twentieth century, especially for the assessment of noncognitive abilities, including clinical skills and behaviors. With particular reference to the current decade, information collected by the Liaison Committee on Medical Education (LCME) is used to examine the congruence of assessment methods with the rising tide of understanding--and accreditation requirements--that knowledge, competence, and behavioral objectives require different methods of assessment to measure the extent of students' learning in each domain. Amongst 97 medical schools having accreditation surveys between July 1993 and June 1998, only 186 of 751 basic science courses tested students' noncognitive achievements in things such as the preparation for and participation in small-group conferences, the quality of case-based discussion, library research and literature reviews, and research projects, despite staking out scholarship, habits of life-long learning, and reasoned thinking as educational objectives. In the clerkships of these schools, structured and observed assessments of clinical skills--with standardized patients and/or OSCEs--contributed 7.4-23.1% to a student's grade (depending on the clerkship discipline), while the predominant contribution (50-70% across the clerkships) was made by resident and faculty ratings that were based largely on recollections of case presentations and discussions having little relationship to interpersonal skills, rapport with patients, and logical and sequenced history taking and physical examination. On a more optimistic note, the results show that the number of schools using standardized patients in one or more clerkships increased between 1993 and 1998 from 34.1% to 50.4% of the 125 schools in the United States, and the number of schools using standardized patients in comprehensive fourth-year examinations increased from 19.1% to 48% of the total. Despite such progress, this study shows that too many medical schools still fail to employ evaluation methods that specifically assess students' achievement of the skills and behaviors they need to learn to practice medicine. The findings of this article explain why accreditors are paying closer attention to how well schools provide measured assurances that students learn what the faculties set out to teach.

On the importance and validity of medical accreditation standards.

In late 1997, the authors conducted a national survey of communities of interest about the importance and clarity of 44 accreditation standards applied to teaching, learning, and evaluation in medical schools by the Liaison Committee on Medical Education (LCME). Questionnaires were mailed to deans and educational administrators at U.S. medical schools; current LCME members and surveyors and those who had served during the preceding five years; a random selection of residency program directors drawn from both general practice and speciality disciplines; sample groups of medical students and residents; and a cohort of practicing physicians not affiliated with academic medical institutions. Altogether 1,659 questionnaires were mailed, and 701 responses were received (42%). The recipients were asked to use a five-point Likert scale to rate each of the 44 standards both for its perceived importance as an indicator of the quality of undergraduate medical education and for the clarity with which the standard's intent was conveyed. Although the mean ratings of importance all fell in the "moderately important" and "highly important" areas across the respondent groups, the ratings divided into three groups, semantically and statistically. At the high end for importance are standards dealing with fundamental qualities of students, instruction, and the structuring of resources. At the low end of the importance scale are standards dealing largely with matters of process. The ratings for clarity were systematically lower than the ratings for importance, and in some cases the rating for clarity were even more widely discrepant with the ratings for importance. Individual comments by respondents about certain standards were critical of their complicated construction and of confusion about their meaning and measures of compliance. One or more of these hallmarks--being rated of lower importance or clarity, and being the target of criticism by survey respondents--distinguished most of the standards that earlier study had shown are often neglected by surveyors. The predictive validity of each of a number of standards was examined by testing the association between the standard (or its derivative) and outcomes expressed in annual student and school questionnaires and compiled in databases of the Association of American Medical Colleges and the American Medical Association. The result was a mixed bag, confounded by the absence of specific dimensions of many accreditation standards (independent variables) and the lack of discriminating measures of outcome (dependent variables). Nevertheless, the LCME's accreditation standards are believed to be important by those most affected by them. And beyond validating that medical accreditation is guided by relevant standards for teaching, learning, and evaluation, the results of this study point to ways by which the process can be made more precise and useful.

The meaning and application of medical accreditation standards.

The authors examined the operational meaning of the 48 items that state the accreditation standards for teaching, learning, and evaluation in medical school, and determined the extent to which these standards were applied by schools and by on-site evaluators for 59 programs surveyed by the Liaison Committee on Medical Education (LCME) in 1994-1996. In this study, "application" meant that evidence was offered, not necessarily that it proved compliance with the standard. The data sources employed were the medical education databases and self-studies prepared by schools undergoing accreditation surveys, and the reports prepared by ad hoc teams of surveyors. The frequency with which evidence of compliance was offered by the schools and cited by evaluators was determined for each of the 48 accreditation requirements. In addition, the authors compared the patterns of surveyors' concerns about noncompliance at schools surveyed during 1984-1986 and at those visited during 1994-1996. In 1994-1996, schools addressed 42 of the 48 accreditation requirements in 90% of instances of more. The areas of particularly low attention dealt with the definition and communication of educational objectives (47% of schools provided evidence); faculty authority and control of academic programs in clinical affiliates (12%); and the faculty's commitment to being effective teachers and their understanding of pedagogy, curricular design, and methods of evaluation (8%). Survey teams, in contrast, accounted in their reports for only 26 (55%) of the standards during the same time period. Among those least frequently addressed were the definition and communication of educational objectives by schools (accounted for in 59% of the reports); assessment of students' problem-solving ability (51%); comparability of educational experiences and student evaluation across dispersed teaching sites (49%); faculty understanding of pedagogy, curriculum construction, and the evaluation of students (8%); faculty authority and control of academic programs in clinical affiliates (7%); and knowledge of the administration and faculty about methods for measuring student performance (2%). Over the past decade, surveyors' most frequently cited concerns about schools' noncompliance with accreditation standards dealt with student counseling and health services, institutional financial and space/facilities resources, faculty issues, and vacant decanal and department chair positions. Next in order were concerns about various aspects of the educational program leading to the MD degree. Among the high-profile concerns about the educational program that increased significantly over the decade were those about curriculum design, management, and evaluation; primary/ambulatory care experiences; and student advancement policies and due-process issues. Schools paid high attention to most of the 48 standards, in large part because they were prompted by the formatting of the medical education database and self-study guidelines. In those instances of lesser attention, the fault lies as much or more with ambiguities in the construction and meaning of the standards as with institutional laxity. The surveyors' inattention to accreditation standards is more troubling. In some cases it can be attributed to uncertainties about the meaning of the requirements and the quantities that need to be audited; or surveyors may be comfortable reaching a "substantial compliance" threshold without adducing all the evidence. The authors argue that many of the standards given scant attention on surveys are important to educational program development and quality control. The LCME will need to consider whether more prominent definition and highlighting should be given to neglected standards, or whether some of the requirements are at the margin as quality indicators. A planned survey of communities of interest-educators, practitioners, students, graduates, and residency program directors, among others-may help confirm

The objectives of medical education: reflections in the accreditation looking glass.

The authors explored the extent to which medical schools have established institutional and departmentalized educational objectives, by examining the accreditation databases, institutional self-studies, and site visit reports of 59 schools surveyed by the Liaison Committee on medical Education (LCME) in 1994-1996. In this study, the individual school was the unit of analysis, and the dependent variables were statements--in outcomes' terms--of institutional and departmental learning objectives. Objectives were classified as "robust" when they were expressed as measurable learning outcomes in the domains of knowledge and skills and behaviors. Departmental objectives were assessed separately for core basic science courses and clinical clerkships. A number of independent variables were studied for their association with the elaboration of outcomes-based educational objectives: centralized management of the curriculum, curricular reform, an office of education, an institutionalized faculty development program, problem-based learning, interdisciplinary teaching, systematized assessment of educational outcomes, and structured clinical skills assessment. Of the 59 schools, 15 (25%) had robust institutional objectives that identified items of measurable knowledge, skills, and behavior, while 44 (75%) had modest objectives, usually descriptions of the purpose of the curriculum, global statements about the knowledge and behavior qualities that students should acquire, and the subjects that instructors intended to tech. Schools with well-articulated learning outcomes were more likely to have centralized management of the curriculum, a record of curricular reform, a program of problem-based learning, rich basic science course and clerkship objectives, and structured assessment of students' clinical skills. Well-stated and diverse educational objectives were found for basic science courses in only 13 of the 59 schools (22%). Thirty of the schools (51%) had clerkships with robust learning objectives, more commonly in the generalist specialties of pediatrics, family medicine, and internal medicine. Clerkships with stout objectives more often employed structured assessments of students' clinical skills. Schools that paid attention to the explication of learning objectives were more likely to link specific institutional and departmentalized outcomes with methods of evaluating students, in turn guiding the content and methods of instruction. Schools with vapid objectives attracted 40% more accreditation citations for shortcomings in curricular management, course and clerkship quality, and the evaluation of student achievement, especially in the clinical skills domain. In the future, accreditors should focus more closely on this association.

The influence of accreditation on educational change in U.S. medical schools.

The authors examined the influence of accreditation on educational change and reform in U.S. medical schools in the past decade, by reviewing the survey databases and site visit reports of 90 schools that had comprehensive accreditation surveys by the Liaison Committee on Medical Education (LCME) between July 1992 and June 1997. In this study, substantive change was defined as centralizing the design and management of the curriculum, as well as one or more of the following reforms: integrating basic and clinical science instruction and/or conversion to interdisciplinary courses; implementing methods of active, small-group, independent, and hypothesis-based learning; and substantially increasing students' exposure to ambulatory and primary care. Accreditation reports were reviewed to determine the extent to which the LCME previously had admonished schools for shortcomings in their educational programs and advised curricular changes. Notice was taken of grant support by national foundations promoting educational reform, in relation both to the correction of accreditation deficiencies and to curricular reform undertaken by schools on their own initiative. The study also scrutinized the evolution of accreditation standards promoting educational reform, and the LCME's support of initiatives for shortening the period of medical education and promoting performance-based teaching and the assessment of clinical skills. On entering the 1990s, the LCME toughened the standards for design and management of the medical curriculum and for the evaluation of educational program effectiveness that schools must conduct. The greater assessment rigor identified educational shortcomings in 61 of 90 medical schools coming up for accreditation surveys during 1992-1997. On those occasions, 34 of the 61 schools had instituted reforms or were on the verge of doing so. Twenty-five of the schools carrying out reforms (73%) had received major foundation grants, compared with ten of the 27 schools (37%) that had accomplished little. Fifteen schools that had not been reproached earlier were found on the 1992-1997 surveys to have undertaken substantial innovation on their own initiative, five with the help of major foundation awards. The study also shows that a number of schools implemented parts of more sweeping reforms with the help of smaller foundation grants for more discrete purposes. In some instances, it has not been possible to differentiate the influence of the LCME as a force for educational reform from the incentives for change created by national foundations. Overall, the LCME, through its standards and assessment practices, and in synergy with schools and kindred agencies promoting change, is now on the leading edge of improved education and evaluation in the nation's medical schools.

Morphometrics of the anterior dentition in strepsirhine primates.

Size variations in the anterior dentition were analyzed for 26 species of strepsirhine primates. The upper and lower incisor rows of strepsirhines, like those of anthropoid primates, scale isometrically with body size. Within the order Primates, strepsirhines exhibit the smallest incisors relative to body size, followed in increasing size by tarsiers, platyrrhines, and catarrhines. If the lateral teeth of the indriid toothcomb are interpreted as incisors and not canines, correlations between mandibular tooth size variables and body weight are maximized. The upper incisors of strepsirhines are extremely small and frequently widely separated, most likely to minimize occlusion with the toothcomb. Species deviations for assorted size variables of the anterior dentition generally fail to reflect functional variations in the use of the anterior teeth; some of the variables, however, do reflect taxonomic differences within the Strepsirhini. Although toothcomb size variations among extant strepsirhines are more readily interpreted in terms of gum feeding and bark scraping than they are in terms of grooming, anterior dental morphology as a whole is more easily explained by a grooming hypothesis when existing models of toothcomb origins are considered.

Incisor size and diet revisited: the view from a platyrrhine perspective.

Allometric relationships between incisor size and body size were determined for 26 species of New World primates. While previous studies have suggested that the incisors of Old World primates, and anthropoids in general, scale isometrically with body size, the data presented here indicate a negative allometric relationship between incisor size and body size among New World species. This negative allometry was exhibited by platyrrhines when either upper or lower incisor row length was regressed against body weight, and when either least-squares or bivariate principal axis equations were used. When upper incisor length was plotted against skull length, negative allometry could be sustained using both statistical techniques only when the full sample of 26 species was plotted. The choice of variables to represent incisor size and body size, and the choice of a statistical technique to effect the allometric equation, had a more pronounced impact on the location of individual species with regard to lines of best fit. Platyrrhines as a group have smaller incisors relative to body size than do catarrhines, regardless of diet. Among New World primates, small incisors represent a plausible primitive condition; species with relatively large incisors manifest a phyletic change associated with a dietary shift to foods that require increased incisal preparation. The opposite trend characterizes Old World primates. In spite of the taxonomic differences in relative incisor size between platyrrhine and catarrhine primates, inferences about diet derived from an allometric equation for all anthropoids should prove reliable as long as the species with unknown diet does not lie at the upper end of the body size range for platyrrhines or catarrhines.

The birth and early development of a captive sifaka, Propithecus verreauxi coquereli.

We report here our observations on parturition and infant development up to five months in a captive sifaka, Propithecus verreauxi coquereli. Comparisons are made with other reports of Propithecus. In describing the acquisition of developmental landmarks, a distinction is made between the first sighting of a behavior pattern and its regular occurrence in the infant's repertoire. We describe the development of locomotor abilities, grooming, feeding, play, and social interactions. In general, maturation of Propithecus is slow relative to species of the genus Lemur.