Effect of Access to an Electronic Medical Resource on Performance Characteristics of a Certification Examination: A Randomized Controlled Trial.

BACKGROUND: Electronic resources are increasingly used in medical practice. Their use during high-stakes certification examinations has been advocated by many experts, but whether doing so would affect the capacity to differentiate between high and low abilities is unknown. OBJECTIVE: To determine the effect of electronic resources on examination performance characteristics. DESIGN: Randomized controlled trial. SETTING: Medical certification program. PARTICIPANTS: 825 physicians initially certified by the American Board of Internal Medicine (ABIM) who passed the Internal Medicine Certification examination or sat for the Internal Medicine Maintenance of Certification (IM-MOC) examination in 2012 to 2015. INTERVENTION: Participants were randomly assigned to 1 of 4 conditions: closed book using typical or additional time, or open book (that is, UpToDate [Wolters Kluwer]) using typical or additional time. All participants took the same modified version of the IM-MOC examination. MEASUREMENTS: Primary outcomes included item difficulty (how easy or difficult the question was), item discrimination (how well the question differentiated between high and low abilities), and average question response time. Secondary outcomes included examination dimensionality (that is, the number of factors measured) and test-taking strategy. Item response theory was used to calculate question characteristics. Analysis of variance compared differences among conditions. RESULTS: Closed-book conditions took significantly less time than open-book conditions (mean, 79.2 seconds [95% CI, 78.5 to 79.9 seconds] vs. 110.3 seconds [CI, 109.2 to 111.4 seconds] per question). Mean discrimination was statistically significantly higher for open-book conditions (0.34 [CI, 0.32 to 0.35] vs. 0.39 [CI, 0.37 to 0.41] per question). A strong single dimension showed that the examination measured the same factor with or without the resource. LIMITATION: Only 1 electronic resource was evaluated. CONCLUSION: Inclusion of an electronic resource with time constraints did not adversely affect test performance and did not change the specific skill or factor targeted by the examination. Further study on the effect of resource inclusion on other examinations is warranted. PRIMARY FUNDING SOURCE: ABIM Foundation.

Factors influencing the decision of surgery residency graduates to pursue general surgery practice versus fellowship.

OBJECTIVES: Surgery residency serves 2 purposes-prepare graduates for general surgery (GS) practice or postresidency surgical fellowship, leading to specialty surgical practice (SS). This study was undertaken to elucidate factors influencing career choice for these 2 groups. METHODS: All US allopathic surgery residency graduates from 2009 to 2013 (n = 5512) were surveyed by the American Board of Surgery regarding confidence, autonomy, and reasons for career selection between GS and SS. Surveys were distributed by mail in November 2013, with follow-up mailings to initial nonrespondents. RESULTS: Sixty-one percent (3354) of graduates completed the survey; 26% pursued GS, and 74% SS. GS expressed greater levels of confidence than SS across the common surgical procedures queried. Confidence increased with each year after completion of residency for GS but not SS. The decision to pursue GS or SS was made during residency by 77% and 74%, respectively. Fifty-seven percent of those who chose GS indicated that a GS mentor significantly influenced their decision. GS rated procedural variety, opportunity for practice autonomy, choice of practice location, and influence of a mentor as reasons to pursue GS practice. SS listed control over scope of practice, prestige, salary, and specialty interest as reasons to pursue SF. Both groups expressed a high degree of satisfaction with their career choice (GS, 94%; SS, 90%). CONCLUSIONS: Most graduates who pursue GS practice are confident and content. The decision to pursue GS is strongly influenced by a GS mentor. Lack of confidence may be a more significant factor for choosing SS. These findings suggest opportunities for improvements in confidence and mentorship during residency.

Practice administration training needs of recent general surgery graduates.

INTRODUCTION: Practice administration education and experience during surgery residency are variable among residency programs. To better understand these issues, a survey of recent General Surgery residency (GS) graduates was compared with the results from a survey of GS program directors (PD). METHODS: All GS graduates completing residency from 2009 to 2013 (n = 5,512) were surveyed to assess opinions regarding the desire for more instruction during residency in practice administration. General surgeons were defined as those not pursuing fellowship training; specialist surgeons (SS) completed additional training after their GS residency. Separately, all GS residency PDs were surveyed regarding practice administration education in their programs. RESULTS: A total of 3,354 responded to the GS graduate survey (response rate = 61%). GS comprised 26% of the respondents. The vast majority of all respondents desired more training in practice administration. There were no significant differences in the degree to which instruction was desired among GS, SS, residency program type, or current practice setting. The GS PD response rate was 68% (171/252 programs). Only 28% of programs included practice administration in the residency curriculum. CONCLUSION: Practice administration education is highly desired by GS and SS graduates. Our findings indicate a clear need for a curriculum in practice administration during residency.

Using SAS PROC MCMC for Item Response Theory Models.

Interest in using Bayesian methods for estimating item response theory models has grown at a remarkable rate in recent years. This attentiveness to Bayesian estimation has also inspired a growth in available software such as WinBUGS, R packages, BMIRT, MPLUS, and SAS PROC MCMC. This article intends to provide an accessible overview of Bayesian methods in the context of item response theory to serve as a useful guide for practitioners in estimating and interpreting item response theory (IRT) models. Included is a description of the estimation procedure used by SAS PROC MCMC. Syntax is provided for estimation of both dichotomous and polytomous IRT models, as well as a discussion on how to extend the syntax to accommodate more complex IRT models.