Correction to: CASPer, an online pre-interview screen for personal/professional characteristics: prediction of national licensure scores.

In re-examining the paper "CASPer, an online pre-interview screen for personal/professional characteristics: prediction of national licensure scores" published in AHSE (22(2), 327-336), we recognized two errors of interpretation.

CASPer, an online pre-interview screen for personal/professional characteristics: prediction of national licensure scores.

Typically, only a minority of applicants to health professional training are invited to interview. However, pre-interview measures of cognitive skills predict for national licensure scores (Gauer et al. in Med Educ Online 21 2016) and subsequently licensure scores predict for performance in practice (Tamblyn et al. in JAMA 288(23): 3019-3026, 2002; Tamblyn et al. in JAMA 298(9):993-1001, 2007). Assessment of personal and professional characteristics, with the same psychometric rigour of measures of cognitive abilities, are needed upstream in the selection to health profession training programs. To fill that need, Computer-based Assessment for Sampling Personal characteristics (CASPer)-an on-line, video-based screening test-was created. In this paper, we examine the correlation between CASPer and Canadian national licensure examination outcomes in 109 doctors who took CASPer at the time of selection to medical school. Specifically, CASPer scores were correlated against performance on cognitive and 'non-cognitive' subsections of both the Medical Council of Canada Qualifying Examination (MCCQE) Parts I (end of medical school) and Part II (18 months into specialty training). Unlike most national licensure exams, MCCQE has specific subcomponents examining personal/professional qualities, providing a unique opportunity for comparison. The results demonstrated moderate predictive validity of CASPer to national licensure outcomes of personal/professional characteristics three to six years after admission to medical school. These types of disattenuated correlations (r = 0.3-0.5) are not otherwise predicted by traditional screening measures. These data support the ability of a computer-based strategy to screen applicants in a feasible, reliable test, which has now demonstrated predictive validity, lending evidence of its validation for medical school applicant selection.

The etiology of diagnostic errors: a controlled trial of system 1 versus system 2 reasoning.

PURPOSE: Diagnostic errors are thought to arise from cognitive biases associated with System 1 reasoning, which is rapid and unconscious. The primary hypothesis of this study was that the instruction to be slow and thorough will have no advantage in diagnostic accuracy over the instruction to proceed rapidly. METHOD: Participants were second-year residents who volunteered after they had taken the Medical Council of Canada (MCC) Qualifying Examination Part II. Participants were tested at three Canadian medical schools (McMaster, Ottawa, and McGill) in 2010 (n = 96) and 2011 (n = 108). The intervention consisted of 20 computer-based internal medicine cases, with instructions either (1) to be as quick as possible but not make mistakes (the Speed cohort, 2010), or (2) to be careful, thorough, and reflective (the Reflect cohort, 2011). The authors examined accuracy scores on the 20 cases, time taken to diagnose cases, and MCC examination performance. RESULTS: Overall accuracy in the Speed condition was 44.5%, and in the Reflect condition was 45.0%; this was not significant. The Speed cohort took an average of 69 seconds per case versus 89 seconds for the Reflect cohort (P < .001). In both cohorts, cases diagnosed incorrectly took an average of 17 seconds longer than cases diagnosed correctly. Diagnostic accuracy was moderately correlated with performance on both written and problem-solving components of the MCC licensure examination and inversely correlated with time. CONCLUSIONS: The study demonstrates that simply encouraging slowing down and increasing attention to analytical thinking is insufficient to increase diagnostic accuracy.

The orthopaedic in-training examination: perspectives of program directors and residents from the United States and Canada.

OBJECTIVE: Objective assessment of resident performance continues to task program directors (PDs) with a formidable challenge. This study evaluated attitudes toward the Orthopaedic In-Training Examination (OITE), compared its value between countries, assessed its value against other metrics of resident performance, and examined program and resident factors predictive of high achievement. DESIGN: Survey. SETTING: Orthopedic surgery residency programs across the United States and Canada. PARTICIPANTS: One hundred sixty-six PDs and 945 residents. RESULTS: Eighty-eight PDs and 331 residents completed the surveys (response rates, 54% and 35%, respectively). PDs and residents in the United States assigned greater importance to the OITE than did those in Canada and reported OITE scores from the United States were significantly higher. PDs in the United States reported greater consequences for residents with poor scores than did PDs from Canada, including remediation and reprimand. Observed structured clinical examinations, internal examinations, and in-training evaluation reports were assigned greater importance by PDs and residents in Canada, but low or no importance by those in the United States. In preparation for the OITE, residents strongly favored prior OITE and American Academy of Orthopaedic Surgeons self-assessment questions, the 'AAOS Comprehensive Orthopaedic Review' textbook, the Journal of the American Academy of Orthopaedic Surgeons, and an OITE-based multiple-choice question website. Regression analysis identified resident and program emphasis on OITE studying and higher level of training as positive predictors for higher OITE scores. CONCLUSIONS: The OITE is more important to PDs and residents in the United States than it is in Canada, and the reported OITE scores reflect these attitudes. PDs in Canada also employ a greater diversity of evaluative tools, a practice in keeping with recent advances toward competency-based medical education. The findings of this report may help PDs be aware of alternative methods of formative resident evaluation and ultimately improve the training of future independent surgeons.

The relationship between response time and diagnostic accuracy.

PURPOSE: Psychologists theorize that cognitive reasoning involves two distinct processes: System 1, which is rapid, unconscious, and contextual, and System 2, which is slow, logical, and rational. According to the literature, diagnostic errors arise primarily from System 1 reasoning, and therefore they are associated with rapid diagnosis. This study tested whether accuracy is associated with shorter or longer times to diagnosis. METHOD: Immediately after the 2010 administration of the Medical Council of Canada Qualifying Examination (MCCQE) Part II at three test centers, the authors recruited participants, who read and diagnosed a series of 25 written cases of varying difficulty. The authors computed accuracy and response time (RT) for each case. RESULTS: Seventy-five Canadian medical graduates (of 95 potential participants) participated. The overall correlation between RT and accuracy was -0.54; accuracy, then, was strongly associated with more rapid RT. This negative relationship with RT held for 23 of 25 cases individually and overall when the authors controlled for participants' knowledge, as judged by their MCCQE Part I and II scores. For 19 of 25 cases, accuracy on each case was positively related to experience with that specific diagnosis. A participant's performance on the test overall was significantly correlated with his or her performance on both the MCCQE Part I and II. CONCLUSIONS: These results are inconsistent with clinical reasoning models that presume that System 1 reasoning is necessarily more error prone than System 2. These results suggest instead that rapid diagnosis is accurate and relates to other measures of competence.

The reliability and acceptability of the Multiple Mini-Interview as a selection instrument for postgraduate admissions.

BACKGROUND: The Multiple Mini-Interview (MMI) is useful in selecting undergraduate medical trainees. Postgraduate applicant pools have smaller numbers of more homogeneous candidates that must be actively recruited while being assessed. This paper reports on the MMI's use in assessing residency candidates. METHOD: Canadian and international medical graduates to three residency programs--obstetrics-gynecology and pediatrics (McMaster University) and internal medicine (University of Alberta)--underwent the MMI for residency selection (n = 484) in 2008 and 2009. Reliability was determined and candidates and interviewers completed an exit survey assessing acceptability. RESULTS: Overall reliability of the MMI was acceptable, ranging from 0.55 to 0.72. Using 10 stations would increase reliability to 0.64-0.79. Eighty-eight percent of candidates believed they could accurately portray themselves, while 90% of interviewers believed they could reasonably judge candidates' abilities. CONCLUSIONS: The MMI provides a reliable way to assess residency candidates that is acceptable to both candidates and assessors across a variety of programs.

Effects of bilingualism, aging, and semantic relatedness on memory under divided attention.

We examined how encoding and retrieval processes were affected by manipulations of attention, and whether the degree of semantic relatedness between words in the memory and distracting task modulated these effects. We also considered age and bilingual status as mediating factors. Monolingual and bilingual younger and older adults studied a list of words from a single semantic category presented auditorily, and later free recalled them aloud. During either study or retrieval, participants concurrently performed a distracting task requiring size decisions to words from either the same or a different semantic category as the words in the memory task. The greatest disruptions of memory from divided attention (DA) were for encoding rather than retrieval. The effect of semantic relatedness was significant only for DA at encoding. Older age and bilingualism were associated with lower recall scores in all conditions, but these factors did not influence the magnitude of memory interference. The results suggest that encoding is more sensitive to semantic similarity in a distracting task than is retrieval. The role of attention at encoding and retrieval is discussed.

Divided attention in younger and older adults: effects of strategy and relatedness on memory performance and secondary task costs.

Divided attention at encoding leads to a significant decline in memory performance, whereas divided attention during retrieval has relatively little effect; nevertheless, retrieval carries significant secondary task costs, especially for older adults. The authors further investigated the effects of divided attention in younger and older adults by using a cued-recall task and by measuring retrieval accuracy, retrieval latency, and the temporal distribution of attentional costs at encoding and retrieval. An age-related memory deficit was reduced by pair relatedness, whereas strategy instructions benefited both age groups equally. Attentional costs were greater for retrieval than for encoding, especially for older adults. These findings are interpreted in light of notions of an age-related associative deficit (M. Naveh-Benjamin, 2000) and age-related differences in the use of self-initiated activities and environmental support (F. I. M. Craik, 1983, 1986).