Borderline competence--from a complexity perspective: conceptualization and implementation for certifying examinations.

BACKGROUND: Passing or failing a candidate is a complex decision with one essential aspect being the definition of the required minimal level of competence. Commonly used methods, such as the Angoff method, are based on the 'borderline competence' concept; however, the method describes a 'borderline candidate'--a fictitious person, rather than describing a set of clearly defined criteria that describe a real person whose competencies fall between the clearly incompetent and clearly competent--the borderline zone. APPROACH AND FINDINGS: We argue that competence arises from the combination of discrete abilities across the entire subject domain including the interaction of some of those abilities, where strengths in some interacting abilities could partly compensate for weaknesses in others. We further argue that scores in the borderline zone may include combinations of abilities in which deficiencies are sufficiently compensated for by areas of strength, thus ensuring competency, while other, similarly scoring combinations do not include required compensations to ensure competency. We offer evidence that standard setting judges systematically and consistently assess the level of required ability on each individual task, taking into account the characteristics of those tasks based on the significance--being a function of the severity and incidence of that condition, and the ability to help--and the difficulty of executing that task. IMPLICATIONS: Consequently, a borderline zone of competence implies an overlap of competence and incompetence with respect to the level of performance on tasks of varying clinical significance. We further suggest that this overlap can be conceptualized as a folding of the 'competence plane' (consistent with a cusp catastrophe model) that cuts the space defined by performance levels on clinical conditions of varying significance, with their associated skills requirements. That folding defines the borderline zone in which competent candidates will perform most of the tasks in the borderline zone correctly whereas incompetent candidates will fail most tasks.

Assessing general practice clinical decision making skills: the key features approach.

The key feature problem (KFP) is an innovative assessment format that tests clinical decision making skills by focussing on only the critical steps -- or 'key features' -- of each decision, thereby increasing the number of problems that can be answered in a 3 hour examination. It has been used successfully in The Royal Australian College of General Practitioners Examination for Fellowship and is gaining in popularity in other high stakes testing settings. This article describes how the KFP paper is designed and constructed, how the pass mark is set, and how the approach performs.

The RACGP Examination--changes from 1999-2004.

Promoting and maintaining the quality and standards of Australian general practice on behalf of the profession and the Australian community is The Royal Australian College of General Practitioners' central activity. An important part of this process is assessment of doctors wishing to attain Fellowship of the RACGP. For general practice registrars, the summative assessment process is the college examination. This article presents details of the assessment process, outlines changes in the RACGP Examination, and publishes its outcomes. This is the first in a series of three articles focussing on changes in the RACGP Examination, its candidature, analysis of results, and plans for future development.