Applying Kane's validity framework to a simulation based assessment of clinical competence.

Assessment of clinical competence is complex and inference based. Trustworthy and defensible assessment processes must have favourable evidence of validity, particularly where decisions are considered high stakes. We aimed to organize, collect and interpret validity evidence for a high stakes simulation based assessment strategy for certifying paramedics, using Kane's validity framework, which some report as challenging to implement. We describe our experience using the framework, identifying challenges, decisions points, interpretations and lessons learned. We considered data related to four inferences (scoring, generalization, extrapolation, implications) occurring during assessment and treated validity as a series of assumptions we must evaluate, resulting in several hypotheses and proposed analyses. We then interpreted our findings across the four inferences, judging if the evidence supported or refuted our proposed uses of the assessment data. Data evaluating "Scoring" included: (a) desirable tool characteristics, with acceptable inter-item correlations (b) strong item-total correlations (c) low error variance for items and raters, and (d) strong inter-rater reliability. Data evaluating "Generalizability" included: (a) a robust sampling strategy capturing the majority of relevant medical directives, skills and national competencies, and good overall and inter-station reliability. Data evaluating "Extrapolation" included: low correlations between assessment scores by dimension and clinical errors in practice. Data evaluating "Implications" included low error rates in practice. Interpreting our findings according to Kane's framework, we suggest the evidence for scoring, generalization and implications supports use of our simulation-based paramedic assessment strategy as a certifying exam; however, the extrapolation evidence was weak, suggesting exam scores did not predict clinical error rates. Our analysis represents a worked example others can follow when using Kane's validity framework to evaluate, and iteratively develop and refine assessment strategies.

Barriers to offering French language physician services in rural and northern Ontario.

INTRODUCTION: Rural and Northern Ontario francophones face many health-related challenges including poor health status, a poor supply of French-speaking physicians, and the potential for an inability or reduced ability to effectively communicate with anglophone healthcare providers. As such, it can reasonably be expected that rural and Northern Ontario francophones experience barriers when receiving care. However, the experience of physicians working in areas densely populated by francophones is largely unexplored. This paper identifies barriers experienced by French-speaking and Non-French-speaking rural and Northern Ontario physicians when serving francophone patients. METHODS: A series of key informant interviews were conducted with 18 family physicians practicing in rural and urban francophone communities of Northeastern Ontario. Interviews were analyzed using a thematic analysis process. RESULTS: Five categories of barrier were identified: (1) language discordance, (2) characteristics of francophone patients, (3) dominance of English in the medical profession, (4) lack of French-speaking medical personnel, and (5) physicians' linguistic (in)sensitivity. Some barriers identified were unique to Non-French-speaking physicians (eg language discordance, use of interpreters, feelings of inadequacy), some were unique to French-speaking physicians (eg limited French education and resources), and some were common to both groups (eg lack of French-speaking colleagues/staff, added time commitments, and the particularities of Franco-Ontarian preferences and culture). CONCLUSIONS: Healthcare providers and decision makers may take interest in these results. Although physicians were the focus of the present article, the barriers expressed are likely experienced by other healthcare providers, and thus the lessons learned from this article extend beyond the physician workforce. Efforts must be made to offer educational opportunities for physicians and other healthcare providers working in areas densely populated by francophones; these include linguistic and cultural sensitivity training, in addition to teaching strategies for the practice of 'active offer' of French-language services. In sum, the present study outlines the importance of linguistic concordant communication in healthcare delivery, and describes some of the challenges faced when providing French-language services in rural and Northern Ontario.

Strategies for improved French-language health services: Perspectives of family physicians in northeastern Ontario.

OBJECTIVE: To identify strategies to improve the quality of health services for Francophone patients. DESIGN: A series of semistructured key informant interviews. SETTING: Northeastern Ontario. Participants A total of 18 physicians were interviewed. Ten physicians were interviewed in French, 7 physicians were women, and 10 physicians were located in urban communities. METHODS: Purposive and snowball sampling strategies were used to conduct a series of semistructured key informant interviews with family physicians practising in communities with a large Francophone population. Principles of grounded theory were applied, guided by a framework for patient-professional communication. Results were inductively derived following an iterative data collection­data analysis process and were analyzed using a detailed thematic approach. MAIN FINDINGS: Respondents identified several strategies for providing high-quality French-language health services. Some were unique to non­French-speaking physicians (eg, using appropriate interpreter services), some were unique to French-speaking physicians (eg, using a flexible dialect), and some strategies were common to all physicians serving French populations (eg, hiring bilingual staff or having pamphlets and posters in both French and English). CONCLUSION: Physicians interviewed for this study provided high-quality health care by attributing substantial importance to effective communication. While linguistic patient-to-physician concordance is ideal, it might not always be possible. Thus, conscious efforts to attenuate communication barriers are necessary, and several effective strategies exist.