Should Objective Structured Clinical Examinations Assist the Clinical Competency Committee in Assigning Anesthesiology Milestones Competency?

BACKGROUND: With the integration of Objective Structured Clinical Examinations into the Anesthesiology primary board certification process, residency programs may choose to implement Objective Structured Clinical Examinations for resident skill assessment. The aim of this study was to evaluate Objective Structured Clinical Examination-based milestone assessment and compare with Clinical Competency Committee milestone assessment that is based purely on clinical evaluations. METHODS: An annual Objective Structured Clinical Examination event was used to obtain milestone assessment of clinical anesthesia year 0-clinical anesthesia year 3 residents for selected milestones in patient care, professionalism, and interpersonal/communication skills. The Objective Structured Clinical Examination scenarios were different for each training level. The Clinical Competency Committee evaluated each resident semiannually based on clinical evaluations of resident performance. The Clinical Competency Committee milestone assessments from 2014 to 2016 that were recorded closest to the Objective Structured Clinical Examination event (±3 months) were compared to the Objective Structured Clinical Examination milestone assessments. A total of 35 residents were included in this analysis in 3 different training cohorts: A (graduates 2016, n = 12); B (graduates 2017, n = 10); and C (graduates 2018, n = 13). All residents participated in Objective Structured Clinical Examinations because their clinical anesthesia year 0 year and Clinical Competency Committee milestone data had been reported since December 2014. RESULTS: Both assessment techniques indicated a competency growth proportional to the length in training. Despite limited cumulative statistics in this study, average trends in the Objective Structured Clinical Examination-Clinical Competency Committee relationship indicated: (1) a good proportionality in reflecting competency growth; (2) a grade enhancement associated with Clinical Competency Committee assessment, dominated by evaluations of junior residents (clinical anesthesia year 0-clinical anesthesia year 1); and (3) an expectation bias in Clinical Competency Committee assessment, dominated by evaluation of senior residents (clinical anesthesia year 2-clinical anesthesia year 3). CONCLUSIONS: Our analysis confirms the compatibility of the 2 evaluation methods in reflecting longitudinal growth. The deviation of Objective Structured Clinical Examination assessments versus Clinical Competency Committee assessments suggests that Objective Structured Clinical Examinations may be providing additional or different information on resident performance. Educators might consider using both assessment methods to provide the most reliable and valid competency assessments during residency.

A Competitive Objective Structured Clinical Examination Event to Generate an Objective Assessment of Anesthesiology Resident Skills Development.

Residency programs are charged with teaching, assessing, and documenting resident competency for a multitude of skills throughout the course of residency training. An innovative, competition-based objective structured clinical examination event was designed in our department to objectively assess the skill level of anesthesiology residents. After conducting the identical event for 2 years in postgraduate year 1 (PGY1) and postgraduate year 2 (PGY2) residents, we tested the hypothesis that the event can provide adequate standardization to appropriately document progression in technical and nontechnical skills. Twenty-one residents participated in both events during their PGY1 and PGY2 years: n = 10, 2012/2013, n = 11, 2013/2014. The PGY1 participants in 2012 were retested in 2013 (as PGY2 residents) during an identical event, and their performance was compared as a group and on an individual level. The PGY1 residents in 2013 did the same in 2014. Four workstations were analyzed to determine whether improvement in performance occurred between the PGY1 and the PGY2 years: (1) preoperative assessment, (2) operating room anesthesia station checkout, (3) peripheral IV and endotracheal tube placement, and (4) transfer of care in the postanesthesia care unit. The performances of PGY1 and PGY2 residents were compared. The assessments were performed by anesthesiology faculty using checklists, time to complete task, and Likert scale ratings. Data analysis showed improved technical anesthesia skills (operating room setup, peripheral IV, and endotracheal tube placement) and more complete anesthesia-related information management in the preoperative assessment and postoperative transition of care in the postanesthesia care unit in PGY2 residents compared with the PGY1 performance of the same residents. The described event is a valuable tool for objective assessment of multiple anesthesia skills and possible milestones during residency.

Objective Assessment of Anesthesiology Resident Skills Using an Innovative Competition-Based Simulation Approach.

Residency programs are charged with teaching, assessing, and documenting resident competency for a multitude of skills. Documentation of competency requires demonstrating specific milestones mandated by the Accreditation Council for Graduate Medical Education. Our department designed an innovative, competition-based approach to objectively assess the skill level of postgraduate year 1 residents in performing basic anesthesia-related tasks after 1 month of anesthesiology training. We launched an "Olympic" event to assess requisite skills in an environment of friendly competition. A simulation format was chosen to allow standardized objective assessment of the resident's skill level at an early stage of training, with possible identification of and intervention for skills needing improvement. Our experience may serve as a template for other programs and specialties developing processes for assessing and documenting improvement in skill and competency over the course of residency training.

Anesthesiology resident personality type correlates with faculty assessment of resident performance.

STUDY OBJECTIVE: To study the association between anesthesiology residents' personality preference types, faculty evaluations of residents' performance, and knowledge. DESIGN: Convenience sample and prospective study. SETTING: Academic department of anesthesiology. SUBJECTS: Consenting anesthesiology residents (n = 36). INTERVENTIONS: All participants completed the Myers Briggs Type Indicator® (MBTI®). MEASUREMENTS: All residents' 6-month summation of daily focal evaluations completed by faculty [daily performance score (DPS); 1 = unsatisfactory, 2 = needs improvement, 3 = meets expectations, 4 = exceeds expectations], as well as a global assessment of performance (GAP) score based on placement of each resident into perceived quartile compared with their peers (ie,1 = first, or top, quartile) by senior faculty (n = 7) who also completed the MBTI, were obtained. The resident MBTI personality preferences were compared with the DPS and GAP scores, the United States Medical Licensing Examination (USMLE) I and II scores, and faculty MBTI personality type. MAIN RESULTS: There was no association between personality preference type and performance on standardized examinations (USMLE I, II). The mean GAP score was better (higher quartile score) for Extraverts than Introverts (median 2.0 vs 2.6, P = 0.0047) and for Sensing versus Intuition (median 2.0 vs 2.6, P = 0.0206) preference. Faculty evaluator MBTI preference type did not influence the GAP scores they assigned residents. Like GAP, the DPS was better for residents with Sensing versus Intuition preference (median 3.5 vs 3.3, P = 0.0111). No difference in DPS was noted between Extraverts and Introverts. CONCLUSIONS: Personality preference type was not associated with resident performance on standardized examinations, but it was associated with faculty evaluations of resident performance. Residents with Sensing personality preference were evaluated more favorably on global and focal faculty evaluations than those residents who chose the Intuition preference. Extraverted residents were evaluated more favorably on global but not focal assessment of performance.

Evaluating practice-based learning and improvement: efforts to improve acceptance of portfolios.

INTRODUCTION: The Accreditation Council for Graduate Medical Education (ACGME) recommends resident portfolios as 1 method for assessing competence in practice-based learning and improvement. In July 2005, when anesthesiology residents in our department were required to start a portfolio, the residents and their faculty advisors did not readily accept this new requirement. Intensive education efforts addressing the goals and importance of portfolios were undertaken. We hypothesized that these educational efforts improved acceptance of the portfolio and retrospectively audited the portfolio evaluation forms completed by faculty advisors. METHODS: Intensive education about the goals and importance of portfolios began in January 2006, including presentations at departmental conferences and one-on-one education sessions. Faculty advisors were instructed to evaluate each resident's portfolio and complete a review form. We retrospectively collected data to determine the percentage of review forms completed by faculty. The portfolio reviews also assessed the percentage of 10 required portfolio components residents had completed. RESULTS: Portfolio review forms were completed by faculty advisors for 13% (5/38) of residents during the first advisor-advisee meeting in December 2005. Initiation of intensive education efforts significantly improved compliance, with review forms completed for 68% (26/38) of residents in May 2006 (P < .0001) and 95% (36/38) in December 2006 (P < .0001). Residents also significantly improved the completeness of portfolios between May and December of 2006. DISCUSSION: Portfolios are considered a best methods technique by the ACGME for evaluation of practice-based learning and improvment. We have found that intensive education about the goals and importance of portfolios can enhance acceptance of this evaluation tool, resulting in improved compliance in completion and evaluation of portfolios.

Anesthesia for gastrointestinal endoscopic procedures.

Traditionally, sedation for gastrointestinal endoscopic procedures was provided by the gastroenterologist. Increasingly, however, complex procedures are being performed on seriously ill patients. As a result, anesthesiologists now are providing anesthesia and sedation in the gastrointestinal endoscopy suite for many of these patients. This article reviews the challenges encountered in this environment and anesthetic techniques that can be used successfully for these procedures.

Reduced duration of intrathecal sufentanil analgesia in laboring cocaine users.

UNLABELLED: On the basis of our previous clinical experience, we hypothesized in this study that the duration and/or quality of labor analgesia produced by intrathecal sufentanil was less in cocaine-abusing parturients compared with nonabusing parturients. Ten micro g of sufentanil was given intrathecally as part of a combined spinal-epidural (CSE) technique to two groups of laboring parturients: 1). those whose urine tested positive for cocaine (cocaine group), and 2). those whose urine tested negative for cocaine (control group). The epidural catheter was not injected with local anesthetic until the patient requested additional pain relief. The time from injection of intrathecal sufentanil until patient request for additional pain relief was defined as duration of analgesia. Baseline visual analog pain score (VAPS) and cervical dilation were measured before the CSE was performed. After injection of intrathecal sufentanil, VAPS was recorded at specific intervals. Cervical dilation was again documented when the patient requested additional analgesia. We found that both groups reported high baseline VAPS and a marked decrease in VAPS after injection of sufentanil that did not differ between groups. Geometric mean duration of pain relief with adjustment for cervical dilation was 87 min in the cocaine group compared with 139 min in the control group (P = 0.019). All patients experienced itching. We conclude that intrathecal sufentanil produces a similar quality but shorter duration of analgesia in cocaine-abusing parturients compared with nonabusing parturients. IMPLICATIONS: Intrathecal sufentanil administered as part of a combined spinal-epidural technique produces similar quality but reduced duration of labor analgesia in cocaine-abusing parturients compared with nonabusing parturients.