Factors affecting physician performance: implications for performance improvement and governance.

BACKGROUND: A physician's personal and professional characteristics constitute only one, and not necessarily the most important, determining factor of clinical performance. Our study assessed how physician, organizational and systemic factors affect family physicians' performance. METHOD: Our study examined 532 family practitioners who were randomly selected for peer assessment by the College of Physicians and Surgeons of Ontario. A series of multivariate regression analyses examined the impact of physician factors (e.g., demographics, certification) on performance scores in five clinical areas: acute care, chronic conditions, continuity of care and referrals, well care and records. A second series of regressions examined the simultaneous effects of physician, organizational (e.g., practice volume, hours worked, solo practice) and systemic factors (e.g., northern practice location, community size, physician-to-population ratio). RESULTS: OUR STUDY HAD THREE KEY FINDINGS: (a) physician factors significantly influence performance but do not appear to be nearly as important as previously thought; (b) organizational and systemic factors have significant effects on performance after the effects of physician factors are controlled; and (c) physician, organizational and systemic factors have varying effects across different dimensions of clinical performance. CONCLUSIONS: We discuss the implications of our results for performance improvement and physician governance insofar as both need to consider the broader environmental context of medical practice.

Physician-patient encounters: the structure of performance in family and general office practice.

INTRODUCTION: The College of Physicians and Surgeons of Ontario, the regulatory authority for physicians in Ontario, Canada, conducts peer assessments of physicians' practices as part of a broad quality assurance program. Outcomes are summarized as a single score and there is no differentiation between performance in various aspects of care. In this study we test the hypothesis that physician performance is multidimensional and that dimensions can be defined in terms of physician-patient encounters. METHODS: Peer assessment data from 532 randomly selected family practitioners were analyzed using factor analysis to assess the dimensional structure of performance. Content validity was confirmed through consultation sessions with 130 physicians. Multiple-item measures were constructed for each dimension and reliability calculated. Analysis of variance determined the extent to which multiple-item measure scores would vary across peer assessment outcomes. RESULTS: Six performance dimensions were confirmed: acute care, chronic conditions, continuity of care and referrals, well care and health maintenance, psychosocial care, and patient records. DISCUSSION: Physician performance is multidimensional, including types of physician-patient encounters and variation across dimensions, as demonstrated by individual practice. A conceptual framework for multidimensional performance may inform the design of meaningful evaluation and educational recommendations to meet the individual performance of practicing physicians.

Effectiveness of an enhanced peer assessment program: introducing education into regulatory assessment.

INTRODUCTION: The College of Physicians and Surgeons of Ontario developed an enhanced peer assessment (EPA), the goal of which was to provide participating physicians educational value by helping them identify specific learning needs and aligning the assessment process with the principles of continuing education and professional development. In this article, we examine the educational value of the EPA and whether physicians will change their practice as a result of the recommendations received during the assessment. METHODS: A group of 41 randomly selected physicians (23 general or family practitioners, 7 obstetrician-gynecologists, and 11 general surgeons) agreed to participate in the EPA pilot. Nine experienced peer assessors were trained in the principles of knowledge translation and the use of practice resources (tool kits) and clinical practice guidelines. The EPA was evaluated through the use of a postassessment questionnaire and focus groups. RESULTS: The physicians felt that the EPA was fair and educationally valuable. Most focus group participants indicated that they implemented recommendations made by the assessor and made changes to some aspect of their practice. The physicians' suggestions for improvement included expanding the assessment beyond the current medical record review and interview format (eg, to include multisource feedback), having assessments occur at regular intervals (eg, every 5 to 10 years), and improving the administrative process by which physicians apply for educational credit for EPA activities. CONCLUSIONS: The EPA pilot study has demonstrated that providing detailed individualized feedback and optimizing the one-to-one interaction between assessors and physicians is a promising method for changing physician behavior. The college has started the process of aligning all its peer assessments with the principles of continuing professional development outlined in the EPA model.

Use of "standardized examinees" to screen for standardized-patient scoring bias in a clinical skills examination.

BACKGROUND: Clinical skills examinations using standardized patients (SPs) are important in documenting the proficiency of trainees. "Standardized examinees" (SEs) are individuals trained to a specific level of performance; they can be used as internal controls in a high-stakes, clinical skills examination. PURPOSE: The purpose of this study was to determine whether SEs can be trained to portray a specified level of confidence and whether SPs' checklist scoring is affected by the personal manner of the examinee. METHODS: Eight SEs were trained as "students" and trained to achieve a failing score on six cases in an National Board of Medical Examiners (NBME) Prototype Clinical Skills Examination. Four SEs were coached to be confident in manner, and 4 were coached to be insecure. Checklist scores were compared. Seven lay reviewers scored the SEs as confident or insecure on a behavioral assessment form. RESULTS: SEs were not detected as simulations. There was no difference between the checklist scores of confident versus insecure SEs, but their manner was rated as significantly different on all scales in the behavioral assessment. CONCLUSIONS: SEs can be trained to a specified performance level and a desired level of confidence. In this small study, personal manner did not affect SPs' checklist scoring. The use of the SEs provides a mechanism to screen for bias in high-stakes SP examinations.