ABR examinations: the why, what, and how.

The American Board of Radiology (ABR) has provided certification for diagnostic radiologists and other specialists and subspecialists for more than 75 years. The Board certification process is a tangible expression of the social contract between the profession and the public by which the profession enjoys the privilege of self-regulation and the public is assured that it can expect medical professionals to put patients' interests first, guarantees the competence of practitioners, and guards the public health. A primary tool used by the ABR in fulfilling this responsibility is the secure proctored examination. This article sets forth seven standards based on authoritative sources in the field of psychometrics (the science of mental measurements), and explains in each case how the ABR implements that standard. Readers are encouraged to understand that, despite the multiple opinions that may be held, these standards developed over decades by experts using the scientific method should be the central feature in any discussion or critique of examinations given for the privilege of professional practice and for safeguarding the public well-being.

ABR examinations: the why, what, and how.

The American Board of Radiology (ABR) has provided certification for diagnostic radiologists and other specialists and subspecialists for more than 75 years. The Board certification process is a tangible expression of the social contract between the profession and the public by which the profession enjoys the privilege of self-regulation and the public is assured that it can expect medical professionals to put patients' interests first, guarantees the competence of practitioners, and guards the public health. A primary tool used by the ABR in fulfilling this responsibility is the secure proctored examination. This article sets forth seven standards based on authoritative sources in the field of psychometrics (the science of mental measurements), and explains in each case how the ABR implements that standard. Readers are encouraged to understand that, despite the multiple opinions that may be held, these standards developed over decades by experts using the scientific method should be the central feature in any discussion or critique of examinations given for the privilege of professional practice and for safeguarding the public well-being.

Reliability of oral examinations: Radiation oncology certifying examination.

PURPOSE: Oral examinations are used in certifying examinations by many medical specialty boards. They represent daily clinical practice situations more realistically than do written tests or computer-based tests. However, there are repeated concerns in the literature regarding objectivity, fairness, and extraneous factors from interpersonal interactions, item bias, reliability, and validity. In this study, the reliability of oral examination on the radiation oncology certifying examination, which was administered in May of 2010, was analyzed. METHODS AND MATERIALS: One hundred fifty-two candidates rotated though 8 examination stations. Stations consisted of a hotel room equipped with a computer and software that exhibited images appropriate to the content areas. Each candidate had a 25-30 minute face-to-face encounter with an oral examiner who was a content expert in one of the following areas: gastrointestinal, gynecology, genitourinary, lymphoma/leukemia/transplant/myeloma, head/neck/skin, breast, central nervous system/pediatrics, or lung/sarcoma. This type of design is typically referred to as a repeated measures design or a subject by treatment design, although the oral examination was a routine event without any experimental manipulation. RESULTS: The reliability coefficient was obtained by applying Feldt and Charter's simple computational alternative to analysis of variance formulas that yielded KR-20, or Cronbach's coefficient alpha of 0.81. CONCLUSIONS: An experimental design to develop a blueprint in order to improve the consistency of evaluation is suggested.

Introduction to value-based payment modifiers.

Value-based payment modifiers were legislated by Congress in the 2010 Patient Protection and Affordable Care Act. It is clear in the legislation, and the corresponding proposals published by the secretary of the US Department of Health and Human Services in late 2011, that the intent is to move from paying physicians for reporting to paying physicians for performance. The proposals, developed jointly with CMS, specify that the calculation of payments for performance will be a composite of quality and cost measures. The base year for determining performance benchmarks for the performance measures will be 2013, and the measures will be applied to physician payments on a limited basis beginning in 2015 and to all physician payments by 2017. The role of medical specialty boards, such as the ABR, in the development and deployment of measures is highlighted in this context. CMS's recent conversations with board representatives have indicated their view that the boards' measure development activities are key to increasing physician (especially specialist) participation in the Physician Quality Reporting System to 50% by 2015, from 20% to 30% today. The ABR will continue its past activities in this arena, working with the American Board of Medical Specialties, CMS, and specialty societies, so that ABR diplomates will be able to simultaneously complete their Maintenance of Certification requirements, satisfy the requirements for CMS incentives, and avoid penalties.

Web modules on professionalism and ethics.

Health care disciplines have always held resolutely to a commitment to professionalism and high ethical standards. With the present emphasis on public accountability, professionalism and ethics are receiving enhanced attention in health care education and practice. A challenge for radiologists, radiation oncologists, and medical physicists is to define the scope and depth of knowledge about professionalism and ethics that are necessary for the practice of the disciplines. A further challenge is to develop accessible educational materials that encompass this required knowledge. About 2 years ago, the ABR Foundation decided to address these challenges through the development of an ethics and professionalism curriculum and production of a series of Web-based educational modules that follow the curriculum. Six organizations agreed initially to contribute financially to construction of the curriculum and modules and were later joined by a seventh. The curriculum was developed by the ABR Foundation and included in a request for proposals that was widely distributed. Teams of authors for each of 10 modules were selected from respondents to the request for proposals. As the modules were developed, they were reviewed in 3 successive stages, including peer review by members of the ACR Committee on Professionalism and the RSNA-ACR Task Force on an Ethics Curriculum. After revisions were prepared in response to the reviews, the modules were translated into a format compatible with the e-learning platform on which they are mounted. The modules are now available to all who wish to study them.

Practice analysis: a basis for content validity for american board of radiology examinations in diagnostic radiology.

The ABR performs practice analysis every 3 years, according to its strategic plan, in an effort to strengthen the content validity of its qualifying and certifying examinations as well as its maintenance of certification examinations. A nationwide survey of diagnostic radiologists was conducted in July 2010 for the purpose of determining the critically important and frequently performed activities in 12 clinical categories. The survey instrument was distributed electronically to 17,721 members of the ACR, with a unique identification code for each individual. A 5-point scale was established for both frequency and importance variables. The frequency scale ranged from 1 to 5 as follows: 1 = not applicable, 2 = occasionally, 3 = monthly, 4 = weekly, and 5 = daily. The scale for importance also ranged from 1 to 5: 1 = not applicable, 2 = not important, 3 = somewhat important, 4 = important, and 5 = essential. A total of 2,909 diagnostic radiologists (19.32%) participated. Of these, 2,233 (76.76%) indicated that they spent ≥50% of their time in clinical practice. Because of its brevity of the list of the activities, results for the gastrointestinal category are presented in this article. The list of activities weighted according to importance and frequency is presented in this article and, as illustrated, could become the foundation for developing a more detailed blueprint for the gastrointestinal category certifying examinations in diagnostic radiology. Findings on demographic information are also presented.

American Board of Radiology radiation oncology initial qualifying ("written") examinations.

This report was prepared by those who are closely involved in the radiation oncology initial qualification examinations. The primary purpose of this article is to disseminate information concerning test preparation, test administration, scoring, and reporting processes of the ABR. The authors hope that the information contained in the article will be helpful to radiology residents, program directors, and other interested parties.

American Board of Radiology certifying examination: oral versus computer-based format.

OBJECTIVE: This pilot study of a computer-based examination for primary certification by the American Board of Radiology was designed to acquire comparative data on candidates that were measures of individual performance on the oral examination compared with the computer-based examination. MATERIALS AND METHODS: The pilot computer-based pediatric radiology examination was designed by experienced oral board examiners and the pediatric subspecialty trustees. Images were chosen from the examination repository of the American Board of Radiology. The 20-minute examination was designed to include 8-10 cases with 26-31 scorable units covering all aspects of pediatric radiology. RESULTS: Among the 1,317 candidates taking the oral board examination, 1,048 candidates (79.6%) participated in the voluntary pilot examination. The scores of the two examinations were subjected to statistical analysis. The sensitivity and specificity of the pilot examination were 94.5% and 45.7%. The overall accuracy was 92.8%. Seventy-five candidates (7.2%) who participated in this study received different verdicts on the pilot examination and the pediatric radiology category of the oral examination. Fifty-six of these candidates (5.3%) failed the pilot examination but passed in the oral pediatric radiology category; 19 of the candidates (1.8%) passed the pilot examination but failed the oral pediatric radiology test. Pilot examination scores were higher for candidates who passed the oral pediatric radiology category (median score, 80; interquartile range, 74.1-85.2) than for candidates who failed (median score, 65.4; interquartile range, 58.6-71.0) (p < 0.0001). CONCLUSION: The pediatric pilot examination was useful for differentiating passing candidates from failing candidates when the score in the pediatric radiology category of the oral examination was used as the reference standard. The overall accuracy was 92.8%.

American Board of Radiology diagnostic radiology initial qualifying (written) examinations.

The purpose of this article is to inform radiology residents, program directors, and other interested parties of the processes involved in developing, administering, and scoring the American Board of Radiology (ABR) diagnostic radiology qualifying (written) examinations. The residents, once certified, will have a lifelong professional relationship with the ABR. It is important for the ABR to be transparent about the processes it uses to ensure that its examinations are fair, valid, and reliable so that residents and their program directors have accurate information about these high-stakes examinations.