Defining competency-based evaluation objectives in family medicine: dimensions of competence and priority topics for assessment.

OBJECTIVE: To develop a definition of competence in family medicine sufficient to guide a review of Certification examinations by the Board of Examiners of the College of Family Physicians of Canada. DESIGN: Delphi analysis of responses to a 4-question postal survey. SETTING: Canadian family practice. PARTICIPANTS: A total of 302 family physicians who have served as examiners for the College of Family Physicians of Canada's Certification examination. METHODS: A survey comprising 4 short-answer questions was mailed to the 302 participating family physicians asking them to list elements that define competence in family medicine among newly certified family physicians beginning independent practice. Two expert groups used a modified Delphi consensus process to analyze responses and generate 2 basic components of this definition of competence: first, the problems that a newly practising family physician should be competent to handle; second, the qualities, behaviour, and skills that characterize competence at the start of independent practice. MAIN FINDINGS: Response rate was 54%; total number of elements among all responses was 5077, for an average 31 per respondent. Of the elements, 2676 were topics or clinical situations to be dealt with; the other 2401 were skills, behaviour patterns, or qualities, without reference to a specific clinical problem. The expert groups identified 6 essential skills, the phases of the clinical encounter, and 99 priority topics as the descriptors used by the respondents. More than 20% of respondents cited 30 of the topics. CONCLUSION: Family physicians define the domain of competence in family medicine in terms of 6 essential skills, the phases of the clinical encounter, and priority topics. This survey represents the first level of definition of evaluation objectives in family medicine. Definition of the interactions among these elements will permit these objectives to become detailed enough to effectively guide assessment.

Examination outcomes for international medical graduates pursuing or completing family medicine residency training in Quebec.

OBJECTIVE: To review the success of international medical graduates (IMGs) who are pursuing or have completed a Quebec residency training program and examinations. DESIGN: We retrospectively reviewed IMGs' success rates on the pre-residency Collège des médecins du Québec medical clinical sciences written examination and objective structured clinical examination, as well as on the post-residency Certification Examination in Family Medicine. SETTING: Quebec. PARTICIPANTS: All IMGs taking their examinations between 2001 and 2008, inclusive, and Canadian and American graduates taking their examinations during this same period. MAIN OUTCOME MEASURES: Success rates for IMGs on the pre-residency and post-residency examinations, compared with success rates for Canadian and American graduates. RESULTS: Success rates on the pre-residency clinical examinations remained below 50% from 2001 to 2008 for IMGs. Similarly, during the same period, the average success rate on the Certification examination was 56.0% for IMGs, compared with 93.5% for Canadian and American medical graduates. CONCLUSION: Despite pre-residency competency screening and in-program orientation and supports, a substantial number of IMGs in Quebec are not passing their Certification examinations. Another study is under way to analyze reasons for some IMGs' lack of success and to find ways to help IMGs complete residency training successfully and pass the Certification examination.

Defining core procedure skills for Canadian family medicine training.

OBJECTIVE: To create a list of core and enhanced procedures suitable for family medicine training. DESIGN: Mailed or e-mailed survey using a Delphi technique. SETTING: Randomly selected family physician practices across Canada. PARTICIPANTS: Family physicians from urban, small-town, and rural practice locations and academic family physicians. All were experienced family physicians with from 3 to 36 years in practice. INTERVENTIONS: Participant physicians were asked to rate each of 158 procedures as to whether they would expect a graduate from a Canadian family practice training program to have learned and be capable of performing that procedure in their own community. In a second survey, participants were asked to verify the core and enhanced procedures lists produced from the first survey. MAIN OUTCOME MEASURES: Physicians' opinions about a comprehensive list of skills. RESULTS: Twenty-two physicians responded to the first survey (92% response rate) and 14 to the second (58% response rate). Sixty-five core procedures and 15 enhanced procedures were identified in the surveys. More procedures were ranked on the core list and were performed by rural and small-town physicians than by urban physicians. Physicians' agreement with placement of procedures on the core list ranged from 55% to 100% and of procedures on the enhanced list from 50% to 64%. Fifty-five of the procedures on the core list had agreement from more than 70% of participants. CONCLUSION: Procedure lists represent the opinions of Canadian family physicians about the importance of specific procedure skills for new family physicians in their communities. Procedure lists will be helpful for family medicine training programs to evaluate and refine their teaching of procedure skills.