Identifying and Addressing Item Bias in the General Pediatrics Certifying Examination.

This article describes a 2-phase process implemented by the American Board of Pediatrics in 2021 to investigate and remove potential bias on its General Pediatrics Certifying Examination at the item (question) level based on gender or race and ethnicity. Phase 1 used a statistical technique known as differential item functioning (DIF) analysis to identify items in which 1 subgroup of the population outperformed another subgroup after controlling for overall knowledge level. Phase 2 involved a review of items flagged for statistical DIF by the American Board of Pediatrics' Bias and Sensitivity Review (BSR) panel, a diverse group of 12 voluntary subject matter experts tasked with identifying language or other characteristics of those items that may have contributed to the observed performance differences. Results indicated that no items on the 2021 examination were flagged for DIF by gender and 2.8% of the items were flagged for DIF by race and ethnicity. Of those items flagged for race and ethnicity, 14.3% (0.4% of total items administered) were judged by the BSR panel to contain biased language that may have undermined what the item was intending to measure and were therefore recommended to be removed from operational scoring. In addition to removing potentially biased items from the current pool of items, we hope that repeating the DIF/BSR process after each examination cycle will increase our understanding of how language nuances and other characteristics impact item performance so that we can improve our guidelines for developing future items.

Maintenance of Certification Pediatrics: Pediatric Emergency Medicine (PEM): The New Part 3 Maintenance of Certification Assessment Option.

ABSTRACT: Starting in 2022, the American Board of Pediatrics will launch the Maintenance of Certification Assessment for Pediatrics: Pediatric Emergency Medicine (MOCA-Peds: PEM) longitudinal assessment, which will provide an at-home alternative to the point-in-time examination. This longitudinal assessment will help engage PEM physicians participating in continuing certification in a more flexible and continuous lifelong, self-directed learning process while still providing a summative assessment of their knowledge. This commentary provides background information on MOCA-Peds and an introduction to MOCA-Peds: PEM and how it gives the PEM physician another option to participate in continuing certification.

Pediatrician Perspectives on Feasibility and Acceptability of the MOCA-Peds 2017 Pilot.

BACKGROUND AND OBJECTIVES: The American Board of Pediatrics (ABP) certifies that general and subspecialty pediatricians meet standards of excellence established by their peers, immediately after training and over the course of their careers (ie, Maintenance of Certification [MOC]). In 2015-2016, the ABP developed the Maintenance of Certification Assessment for Pediatrics (MOCA-Peds) as an alternative assessment to the current proctored, closed-book general pediatrics (GP) MOC examination. This article is 1 of a 2-part series examining results from the MOCA-Peds pilot in 2017. METHODS: We conducted quantitative and qualitative analyses with 5081 eligible pediatricians who registered to participate in the 2017 pilot; 81.4% (n = 4016) completed a quarter 4 survey and/or end-of-year survey (January 2018) and comprise the analytic sample. RESULTS: The majority of pediatricians considered the MOCA-Peds to be feasible and acceptable as an alternative to the proctored MOC GP examination. More than 90% of respondents indicated they would participate in the proposed MOCA-Peds model instead of the examination. Participants also offered recommendations to improve the MOCA-Peds (eg, enhanced focus of questions on outpatient GP, references provided before taking questions); the ABP is carefully considering these as the MOCA-Peds is further refined. CONCLUSIONS: Pilot participant feedback in 2017 suggested that the MOCA-Peds could be implemented for GP starting in January 2019, with all 15 subspecialties launched by 2022. Current and future evaluations will continue to explore feasibility, acceptability, and learning and practice change as well as sustainability of participation.

Pediatrician Perspectives on Learning and Practice Change in the MOCA-Peds 2017 Pilot.

BACKGROUND AND OBJECTIVES: This article is the second of a 2-part series examining results regarding self-reported learning and practice change from the American Board of Pediatrics 2017 pilot of an alternative to the proctored, continuing certification examination, termed the Maintenance of Certification Assessment for Pediatrics (MOCA-Peds). Because of its design, MOCA-Peds has several learning advantages compared with the proctored examination. METHODS: Quantitative and qualitative analyses with 5081 eligible pediatricians who registered to participate in the 2017 pilot; 81.4% (n = 4016) completed a quarter 4 survey and/or the end-of-year survey (January 2018) and compose the analytic sample. RESULTS: Nearly all (97.6%) participating pediatricians said they had learned, refreshed, or enhanced their medical knowledge, and of those, 62.0% had made a practice change related to pilot participation. Differences were noted on the basis of subspecialty status, with 68.9% of general pediatricians having made a practice change compared with 41.4% of subspecialists. Within the 1456 open-ended responses about participants' most significant practice change, responses ranged widely, including both medical care content (eg, "care for corneal abrasions altered," "better inform patients about. . .flu vaccine") and nonspecific content (eg, providing better patient education, using evidence-based medicine, increased use of resources in regular practice). CONCLUSIONS: As a proctored examination alternative, MOCA-Peds positively influenced self-reported learning and practice change. In future evaluation of MOCA-Peds and other medical longitudinal assessments, researchers should study ways to further encourage learning and practice change and sustainability.

MOCA-Peds: Development of a New Assessment of Medical Knowledge for Continuing Certification.

The American Board of Pediatrics (ABP) certifies that general and subspecialty pediatricians meet standards of excellence established by their peers. Certification helps demonstrate that a general pediatrician or pediatric subspecialist has successfully completed accredited training and fulfills continuous certification requirements (Maintenance of Certification [MOC]). One current component of the MOC program is a closed-book examination administered at a secure testing center (ie, the MOC Part 3 examination). In this article, we describe the development of an alternative to this examination termed the "Maintenance of Certification Assessment for Pediatrics" (MOCA-Peds) during 2015-2016. MOCA-Peds was conceptualized as an online, summative (ie, pass/fail), continuous assessment of a pediatrician's knowledge that would also promote learning. The system would consist of a set number of multiple-choice questions delivered each quarter, with immediate feedback on questions, rationales clarifying correct and incorrect answers, references for further learning, and peer benchmarking. Questions would be delivered quarterly and taken at any time within the quarter in a setting with Internet connectivity and on any device. As part of the development process in 2015-2016, the ABP actively recruited pediatricians to serve as members of a yearlong user panel or single-session focus groups. Refinements to MOCA-Peds were made on the basis of their feedback. MOCA-Peds is being actively piloted with pediatricians in 2017-2018. The ABP anticipates an expected launch in January 2019 of MOCA-Peds for General Pediatrics, Pediatric Gastroenterology, Child Abuse, and Pediatric Infectious Diseases with launch dates for the remaining pediatric subspecialties between 2020 and 2022.

Voluntarily Postponing Testing Is Associated with Lower Performance on the Pediatric Board Certifying Examinations.

OBJECTIVE: To investigate whether postponing certification testing, either voluntarily or involuntarily, affects a candidate's performance on pediatric certifying examinations. STUDY DESIGN: Both general pediatrics (GP) and pediatric subspecialty (PS) examination candidates were included in the study. Candidates were classified into 3 groups based on time since the completion of training: no delay (<12 months), short delay (12-24 months), and long delay (≥24 months). Examination scores and pass rates in the first GP and PS certifying examinations were compared to assess between-group differences. RESULTS: Significant differences in scores and pass rates were found for GP candidates who voluntarily waited 1 year or longer to take the certifying examination. Similarly, PS candidates who opted not to take the first examination available had significantly lower scores and pass rates. However, no significant difference was found for PS candidates who had to wait to take their examination owing to the Board's offered examination schedule. CONCLUSION: Candidates who postpone taking the certifying examination are less likely to pass the examination. The longer a candidate elects to wait to take the examination, the less likely he or she is to pass. The availability of the PS examinations once every 2 years does not affect pass rates and scores, as long as PS candidates take the first available examinations after completing fellowship.

Jobs and Career Plans of New Pediatric Subspecialists.

OBJECTIVES: The issues of importance in selecting a first job for new pediatric subspecialists, and their ability to find positions that match their professional and clinical goals, are unknown. The objectives were to (1) describe current employment patterns, practice characteristics, factors influencing choice of first position, and future work goals of new pediatric subspecialists; and (2) examine the relationship of these variables with the actual professional time allocation and clinical responsibilities compared with the desired first job. METHODS: The authors surveyed 3010 individuals sitting for ≥1 of the 14 subspecialty certification exams. The main outcomes were (1) most important factors in choosing employment; (2) ability to gain employment in positions that matched their goals; and (3) variation in employment characteristics among men versus women, time since completion of training, and part-time versus full-time status. RESULTS: Response rate was 97%. Lifestyle/spousal or family considerations was the factor identified as most important in the choice of first position after fellowship training for half of respondents (50%; n = 1277). There was a median of 75% of actual time spent in direct and/or consultative inpatient or outpatient care, with 5% in medical education, 5% in administration, and 5% in research. A majority (74%; n = 1825) reported this proportion to be approximately what they wanted. Most respondents (89%; n = 2194) reported that their allocation of patient care responsibilities (ie, inpatient versus outpatient) was approximately what they wanted. CONCLUSION: A large majority of pediatric subspecialists found initial positions matching their goals for professional responsibilities and clinical care.

Long-term Plans for Those Selecting Hospital Medicine as an Initial Career Choice.

BACKGROUND: The proportion of the newly graduated pediatric workforce that becomes hospitalists has been increasing slightly over the past decade. However, it is unknown what proportion of those who accept hospitalist positions as their first job intend to remain in the field longer term. This is important to workforce projections regarding the magnitude of those who will function in this role. METHODS: The American Board of Pediatrics incorporated a structured questionnaire within the online application process to the General Pediatrics certification application. Respondents identified as residents or chief residents who selected "hospitalist position" as their immediate postresidency plan were the focus this study. We compared survey responses by gender and location of the medical school attended. RESULTS: Since the initiation of the general pediatrics certification examination application survey, 6335 completed the questionnaire. 79% (n=5001) were either in residency training or were a chief resident. Of those, 8% (n=376) reported they planned to work as a pediatric hospitalist immediately after completing residency. Fewer than half (43%; n=161) reported this to be their long-term career plan. This finding varied by both medical school type and by gender. CONCLUSIONS: The majority of pediatric residents and chief residents who take hospitalist positions immediately after training do not intend for hospital practice to be the long-term focus of their careers. As the field of hospital medicine continues to develop, understanding career trajectories can help inform current and future efforts regarding the potential for different mechanisms for training and certification.

New pediatricians: first jobs and future workplace goals.

BACKGROUND AND OBJECTIVES: Concern is often expressed about the satisfaction of new physicians and the potential match of their workplace goals with available positions. We studied the interface of desired professional activities with actual initial positions. METHODS: Survey study of all general pediatricians taking the 2012 General Pediatrics Certifying Examination. RESULTS: Of the 5210 who sat for the General Pediatrics Certifying Examination, 5163 (>99%) completed the survey. Of the total respondents, 45% self- identified as general pediatricians (N = 2327). Of those who completed training <2 years ago (N = 1365), most were currently engaged in clinical care (87%; N = 1190). The most important factor, cited most frequently by both men and women, in the choice of their first job was lifestyle and spousal or family considerations. Most (83%; N = 977) reported that the allocation of time for specific duties in their current position was consistent with their goals, with no differences between men and women. Most had no desire for inpatient activity. CONCLUSIONS: Despite concerns about young general pediatricians being able to find positions that meet their career goals, most were in jobs that approximated their desired allocation of professional time and focus of clinical work.

Noncompletion in pediatric rheumatology fellowships.

BACKGROUND: One challenge facing the health care workforce is a paucity of pediatrics subspecialists. No prior studies have investigated fellowship noncompletion as an influence of the subspecialty workforce. OBJECTIVE: We sought to determine the noncompletion rate for pediatric rheumatology fellowships and to identify demographic characteristics associated with noncompletion. METHODS: A retrospective cohort study of all trainees entering US pediatric rheumatology fellowship programs between 1997 and 2007 was performed. American Board of Pediatrics tracking data were used to determine completion status (completer or noncompleter) for each trainee. Completers were compared with noncompleters, using the independent variables sex, medical school location, and age. The noncompletion rate was calculated overall and individually. Program size was examined as a predictor of nonompletion rate. Data analysis used χ(2) tests, Kruskal-Wallis tests, and Spearman correlation. RESULTS: The cohort included 182 trainees from 28 pediatric rheumatology fellowship programs. Program size ranged from 1 to 18 trainees. The overall noncompletion rate was 16%. Male fellows, especially male international medical graduates, were more likely to be noncompleters. Noncompletion rates varied among programs: 15 programs had noncompletion rates of 0% and 4 programs had noncompletion rates of 50% or higher. Program size was not associated with noncompletion rate. CONCLUSIONS: During the study period, 1 of 6 pediatric rheumatology fellows did not complete training. Noncompletion was concentrated in a small number of programs. Further research should investigate noncompletion across specialties, identifying the causes of noncompletion at the individual, program, and specialty levels to inform future interventions to improve fellowship completion.

Longitudinal assessment of the timing of career choice among pediatric residents.

OBJECTIVE: To determine the timing of and key factors in resident decision making to pursue either a career in general pediatrics or subspecialty training. DESIGN: We used a 10-item fixed-choice questionnaire that focused on exploring how and when pediatric residents make career choices. SETTING: The survey was administered to all categorical pediatric residents in the United States and Canada as part of the General Pediatrics In-Training Examination in 2007 and 2009. The 2007 level 1 residents and 2009 level 3 residents were matched by a unique person identifier to create a longitudinal data set. PARTICIPANTS: A total of 2305 individuals completed the survey as level 1 residents in 2007 and level 3 residents in 2009, representing a retention rate of 83.5%. MAIN OUTCOME MEASURES: Change in individual and aggregate pediatric resident response over time. RESULTS: A similar number of individuals planned to pursue fellowship training in 2007 and 2009 (1026 vs 1062). Among this group, 745 (72.6%) of the 2009 residents were the same individuals who had indicated that they planned to pursue fellowship training in 2007. A total of 258 (71.9%) of all residents who reported in 2007 that they intended to pursue careers in general pediatrics with little or no inpatient care were still planning to do so in 2009. CONCLUSIONS: Most pediatricians make their decisions regarding pursuit of a career in primary care or to complete a fellowship before they ever enter residency training. It is unknown whether a similar timeline of decision making is consistent across specialties.

General pediatrics resident perspectives on training decisions and career choice.

OBJECTIVE: Little is known regarding at what point during the training period residents in pediatrics make decisions on their future career choices. As part of a dedicated process of reexamining the structure of residency training in pediatrics, the American Board of Pediatrics sought information to better understand the influences, process, and sequencing of both residency program selection and career decision-making among residents. METHODS: All pediatrics resident physicians in all training programs in the United States and Canada (N = 8290) received the survey as part of the general pediatrics in-training examination. The survey focused on exploring how and when pediatrics residents make career choices and assessed perceived flexibility of their individual pediatrics residency program. RESULTS: The response rate was 95%. Location was the most important factor in selecting a residency program for 42% of all residents. Almost half of the pediatrics residents planned to pursue fellowship training after residency, a proportion that changed little across the 3 training years (level 1: 47%; level 2: 49%; level 3: 47%). Those who planned to pursue a general pediatrics career (either with or without inpatient care) were more likely than those who intended to pursue fellowship training to report that lifestyle was the most important factor in their career choice (63% vs 21%). CONCLUSIONS: Not surprisingly, different priorities motivate pediatricians to pursue specific programs for training and specific career options. The finding that those with the highest priority regarding lifestyle are more likely to pursue generalist training has implications for the generalist workforce, because those persons may also be more likely to seek part-time employment. Lifestyle concerns may need to be addressed in subspecialty training and subsequent subspecialty careers to ensure a continued flow of residents into fellowship training.

The in-training examination: an analysis of its predictive value on performance on the general pediatrics certification examination.

OBJECTIVE: This study investigates the predictive validity of the In-Training Examination (ITE). Although studies have confirmed the predictive validity of ITEs in other medical specialties, no study has been done for general pediatrics. STUDY DESIGN: Each year, residents in accredited pediatric training programs take the ITE as a self-assessment instrument. The ITE is similar to the American Board of Pediatrics General Pediatrics Certifying Examination. First-time takers of the certifying examination over a 5-year period who took at least 1 ITE examination were included in the sample. Regression models analyzed the predictive value of the ITE. RESULTS: The predictive power of the ITE in the first training year is minimal. However, the predictive power of the ITE increases each year, providing the greatest power in the third year of training. CONCLUSIONS: Even though ITE scores provide information regarding the likelihood of passing the certification examination, the data should be used with caution, particularly in the first training year. Other factors also must be considered when predicting performance on the certification examination. This study continues to support the ITE as an assessment tool for program directors, as well as a means of providing residents with feedback regarding their acquisition of pediatric knowledge.

Characteristics of general and subspecialty pediatricians who choose not to recertify.

OBJECTIVE: The goal was to determine the reasons given for nonparticipation in maintenance of certification by general pediatricians and pediatric subspecialists whose board certification had expired. METHODS: All United States-based pediatricians whose board certification had expired and who had not recertified as a generalist or subspecialist in 2004 or 2005 were surveyed. The total sample included 1001 pediatric generalists and 1237 subspecialists. We conducted a mail survey study, with a total of 3 mailings to nonrespondents. Analyses initially were conducted separately for generalists with expired certificates, subspecialists with active subspecialty but expired general pediatrics certificates, and subspecialists with expired subspecialty certificates (regardless of their general pediatrics certification status). RESULTS: The overall response rates were 68% for generalists and 76% for subspecialists. Of pediatricians who had allowed their certificates to expire, the majority still intended to recertify (65% of the generalists and 86% of the subspecialists, in their primary subspecialty). The most common reasons cited by the 35% of generalists with expired certificates who were not planning to participate in maintenance of certification were the expense, the time required to complete maintenance of certification, and the perceived lack of relevance to their current practice. The 14% of subspecialists who were not planning to recertify in their primary subspecialty most commonly cited the expense, a change in career path making recertification unnecessary, and the time required. CONCLUSIONS: Although the physicians in this study had allowed their certificates to expire, they still overwhelmingly believed that physicians who provide direct patient care should maintain their certification. There was general consensus among our respondents regarding the professional value to patients and peers of participation in the maintenance of certification program.