Detecting bad actors in value-based payment models.

The U.S. federal government is spending billions of dollars to test a multitude of new approaches to pay for healthcare. Unintended consequences are a major consideration in the testing of these value-based payment (VBP) models. Since participation is generally voluntary, any unintended consequences may be magnified as VBP models move beyond the early testing phase. In this paper, we propose a straightforward unsupervised outlier detection approach based on ranked percentage changes to identify participants (e.g., healthcare providers) whose behavior may represent an unintended consequence of a VBP model. The only data requirements are repeated measurements of at least one relevant variable over time. The approach is generalizable to all types of VBP models and participants and can be used to address undesired behavior early in the model and ultimately help avoid undesired behavior in scaled-up programs. We describe our approach, demonstrate how it can be applied with hypothetical data, and simulate how efficiently it detects participants who are truly bad actors. In our hypothetical case study, the approach correctly identifies a bad actor in the first period in 86% of simulations and by the second period in 96% of simulations. The trade-off is that 9% of honest participants are mistakenly identified as bad actors by the second period. We suggest several ways for researchers to mitigate the rate or consequences of these false positives. Researchers and policymakers can customize and use our approach to appropriately guard VBP models against undesired behavior, even if only by one participant. Supplementary Information: The online version contains supplementary material available at 10.1007/s10742-021-00253-9.

Nipple-sparing mastectomy in women at high risk of developing breast cancer.

Nipple-sparing mastectomy is a valuable addition to the options available for women at high risk of developing breast cancer. In this review, we summarize current knowledge about the high-risk genes, BRCA1, BRCA2 and TP53 and the associated guidelines with regard to risk-reducing surgery. We consider other genetic risks and high-risk lesions. We discuss the literature on bilateral mastectomy for breast cancer risk-reduction, and the results of nipple-sparing mastectomy in particular. Finally, we report on patient satisfaction with these procedures and the impact that nipple-sparing mastectomy may have on women at high-risk of breast cancer.

The performance of outside readings by radiology practices.

OBJECTIVE: The purpose of our study was to determine how many radiology practices perform outside readings, what characteristics affect the prevalence and volume of outside readings, and how practices are paid for outside readings. MATERIALS AND METHODS: We analyzed data from the American College of Radiology's 2007 Survey of Radiologists, a stratified random sample e-mail and telephone survey. A total of 480 survey responses were evaluated; responses were weighted to make them representative of all U.S. radiology practices. We provide descriptive statistics and multivariable regression analysis results. RESULTS: Overall, 40% of radiology practices in the United States performed outside readings in 2007. Outside readings constituted an average of 11% of the workload of these practices and 4% of the total workload of radiologists in the United States. Other practice characteristics being equal, academic practices, government practices, radiology units of multispecialty groups, and small practices had particularly low odds of performing outside readings. If they did perform outside readings, then, other practice characteristics being equal, small practices, solo practices, radiology units of multispecialty groups, practices in the main cities of large metropolitan areas, and those in nonmetropolitan areas had, on average, a relatively large portion of their workload consisting of outside readings. By far, the most common methods of payment were directly billing for the professional component or receiving a flat fee per study. CONCLUSION: Outside readings were a common activity among radiology practices in 2007. There was substantial variability among practice types, sizes, and locations in whether practices performed outside readings and, if so, how much outside reading they did.

Frequency of use of imaging tests in the diagnosis of pulmonary embolism: effects of physician specialty, patient characteristics, and region.

OBJECTIVE: The purpose of this study was to ascertain whether clinical practice in diagnosing pulmonary embolism is consistent with recommendations in the literature and to explore variations in practice across site of care (e.g., emergency department), physician and patient characteristics, and geographic location. MATERIALS AND METHODS: Medicare 5% research identifiable files were analyzed. The cases of patients with emergency department visits or inpatient stays for a diagnosis of pulmonary embolism or for symptoms related to pulmonary embolism (shortness of breath, chest pain, and syncope) were identified. We determined the number of patients who underwent each type of relevant imaging test and evaluated variations in the first non-chest-radiographic test by site of care and treating physician specialty. Using logistic regression, we studied variations in the use of common imaging tests, exploring variations associated with patient characteristics, physician specialty, site of care, and geographic location. RESULTS: For patients in whom pulmonary embolism might have been suspected, the most common tests were echocardiography (26% of the patients), CT or CT angiography of the chest (11%), cardiac perfusion study (6.9%), and duplex ultrasound (7.3%). For patients with an inpatient diagnosis of pulmonary embolism, the most common tests were chest CT or CT angiography (49%), duplex ultrasound (18%), echocardiography (10.9%), and ventilation-perfusion scintigraphy (10.9%). For patients for whom pulmonary embolism might have been suspected, many large variations were found in practice patterns among physician specialties and geographic locations. There were fewer variations among patients with the inpatient diagnosis of pulmonary embolism. CONCLUSION: Physician practice in the diagnosis of pulmonary embolism is broadly consistent with recommendations. However, variations by physician specialty and geographic location may be evidence of inappropriate imaging.

Radiology practices' use of external off-hours teleradiology services in 2007 and changes since 2003.

OBJECTIVE: Our objective is to report patterns of utilization of external off-hours teleradiology services (EOTSs) in 2007 and changes since 2003. MATERIALS AND METHODS: We analyzed non-individually identified data from the American College of Radiology's 2007 Survey of Member Radiologists and its 2003 Survey of Radiologists. Responses were weighted to be nationally representative of individual radiologists and radiology practices. We present descriptive statistics and multivariable regression analysis results on the use of EOTSs in 2007 and comparisons with 2003. RESULTS: Overall, 44% of all radiology practices in the United States reported using EOTSs in 2007. These practices included 45% of all U.S. radiologists. Out-of-practice teleradiology had been used by 15% of practices in 2003. Regression analysis indicates that, other practice characteristics being equal, in 2007, primarily academic practices had lower odds of using EOTSs than private radiology practices. Also, large practices (>or= 30 radiologists) had lower odds of using EOTSs than practices with 15-29 radiologists. Small practices (1-10 radiologists) had high odds, but nonmetropolitan practices did not. There were no significant differences by geographic region of the United States. CONCLUSION: Use of EOTSs was widespread by 2007, and it had been increasing rapidly in the preceding few years. Patterns of use were generally as might be expected except that nonmetropolitan practices did not have high odds of using EOTSs.

Trends in work hours and vacation time among radiologists in the United States.

OBJECTIVE: The purpose of this study was to describe recent trends in weekly work hours and annual vacation days among full-time radiologists in the United States with separate data for academic radiologists, radiologists in multispecialty groups, and radiologists in private practice. MATERIALS AND METHODS: We tabulated non-individually identified responses from the American College of Radiology 1995, 2003, and 2007 surveys of radiologists. These stratified random sample surveys had, respectively, 2,025, 1,924, and 487 responses and response rates of 75%, 63%, and 20%. Responses were weighted to make them representative of all U.S. radiologists. Respondents were assured of confidentiality. RESULTS: Mean weekly hours worked increased from 1995 to 2003 and from 2003 to 2007. The total increase was approximately 5 hours, or 10%. Mean vacation days also increased in both subperiods, from 27 in 1995 to 39 in 2007, yielding an approximately 5% decrease in days worked per year. The overall result was a mean increase of approximately 5% in annual work hours. In 2007, the 25th percentile of weekly hours was 45, and the 75th percentile was 55. The 25th percentile of annual vacation days was 25, and the 75th percentile was 50. Full-time radiologists responding about their own weekly hours reported, at the mean, working 10% more hours than they believed was the average for other full-time radiologists in the practice. CONCLUSION: Weekly hours and annual vacation days both have increased. The percentiles give radiologists a basis for comparison with other radiologists. Radiologists apparently often overestimate their work hours relative to the hours of others in their practices. Misperceptions of this kind might give rise to friction in radiology practices.

Diagnostic radiologists' subspecialization and the new final board examination.

OBJECTIVE: Recognizing that subspecialization can consist of concentration in multiple fields as well as in a single main field, we conducted this study to profile in detail the subspecialization of diagnostic radiologists in the United States in ways that illuminate issues related to the American Board of Radiology plan for a new final examination. MATERIALS AND METHODS: We tabulated nonindividually identified data from the American College of Radiology 2003 Survey of Radiologists, a stratified random-sample mail survey with 1,924 responses and a 63% response rate. Respondents were guaranteed confidentiality. Responses were weighted to make them representative of all radiologists in the United States. RESULTS: Sixty-nine percent of respondents reported specializing at least to a small extent. If concentration in a field is defined as spending 10% or more of clinical work time in the field, 51% of radiologists concentrated in one or two fields, 24% in three or four fields, and 21% in more than four fields. An examination covering a radiologist's four most time-intensive fields would cover 80% of the clinical work of the median radiologist. However, the one fourth of radiologists whose work is most varied would have 40% or more not covered by the examination, but the one fourth with the most concentrated work would have 100% covered. CONCLUSION: Most radiologists concentrate in a few fields, making the American Board of Radiology plan for an examination that covers four fields--or fewer, at an examinee's discretion--a major step forward in recognizing the nature of current practice. Four fields, however, are too many for the practice patterns of many radiologists but too few for the practice patterns of a substantial minority. We offer for consideration more far-reaching reforms.

Radiologists' clinical practice of neuroimaging.

PURPOSE: Because of the importance of neuroimaging as a radiology subspecialty, the aim of this study was to provide a detailed portrait of the demographics, clinical activities, and practices of radiologists heavily involved in neuroimaging. METHODS: The authors analyzed data from the ACR's 2003 Survey of Radiologists, a large, stratified random-sample survey in which respondents were guaranteed confidentiality. The survey achieved a 63% response rate, and responses were weighted to make them representative of all radiologists in the United States. RESULTS: Three-fourths of US radiologists reported doing neuroradiology; 9% reported that neuroradiology was their main subspecialty, and 9% reported spending more than 50% of their clinical work time doing neuroradiology. Of these latter two categories, more than about 75% had certificates of added qualification (CAQs) in neuroradiology, and more than 80% had done neuroradiology fellowships. However, of those spending more than 50% of their clinical work time doing neuroradiology, 7% neither had CAQs nor had done fellowships in the field. One-fourth of radiologists with CAQs or who had done neuroradiology fellowships spent less than 30% of their clinical work time doing neuroradiology. One-third to one-half of neuroimaging was performed by radiologists not heavily involved in the field. Only 6% to 8% of radiologists heavily involved in the field were women, compared with 22% in other subspecialties. CONCLUSIONS: Neuroimaging has the great strength of being a relatively well-integrated subspecialty in that a very large majority of those heavily involved in its practice have CAQs and did fellowships in the field. Among possible concerns are the relatively few women in the field and the apparent waste of expertise resulting from one-fourth of those with neuroradiology subspecialty training or certification being relatively little immersed in its practice.

Balloon endometrial ablation under hysteroscopic control of septate uterus complicated by postcesarean scar defect.

Use of second-generation endometrial ablation devices has largely supplanted hysteroscopic resection and rollerball ablation for treatment of menorrhagia. Pathology of the uterus may be diagnosed immediately before endometrial ablation procedure using these newer instruments, making the chosen device inappropriate and requiring a change in planned procedure. Preprocedural assessment with 1 or a combination of transvaginal ultrasound (TVS), saline-infusion sonography, and hysteroscopy is necessary to identify suitability for preferred technique. TVS is frequently used as the only investigation to assess uterine cavity, and undiagnosed or missed pathology on TVS may render chosen endometrial ablation procedure inappropriate. Previously undiagnosed septate uterus and postcesarean scar defect were diagnosed at hysteroscopy immediately before endometrial ablation, but missed by routine TVS. Balloon endometrial ablation in the septate uterus was used.

Radiation oncologists in the United States.

PURPOSE: To provide an extensive and detailed portrait of radiation oncologists, their professional activities, and the practices in which they work. METHODS AND MATERIALS: We analyzed non-individually identified data from the American College of Radiology's 2003 Survey of Radiation Oncologists, a stratified random sample survey that guaranteed respondents' confidentiality and achieved a 68% response rate, with a total of 472 responses. Responses were weighted to make them representative of all radiation oncologists in the United States. We use two-tailed z tests of percentages and means to compare information from the current survey with those from a similar 1995 survey. RESULTS: The number of posttraining, professionally active radiation oncologists grew from nearly 2900 in 1995 to nearly 3500 in 2003, an increase of approximately 21%. Twenty-three percent of posttraining, professionally active radiation oncologists were women. Among posttraining, professionally active radiation oncologists, 95% were board-certified. Forty-eight percent of radiation oncologists were in nonacademic, radiation-oncology-only private practices; 20% in academic practice; 14% in nonacademic, multispecialty practices; and 11% in solo practice. The largest percentage of radiation oncologists worked in the South (34%). The average annual number of patients treated (curative and palliative) per radiation oncologist was 264. On average, radiation oncologists preferred a 4% increase in their workload. The proportion of radiation oncologists planning a career change decreased from 8% in 1995 to 4% in 2003, and in 2003 34% said they were enjoying radiation oncology more than 5 years earlier, compared with 21% in 1995. CONCLUSION: Despite concerns in 2003 about lower-than-optimal workload, professional satisfaction, if anything, increased since 1995.

Satisfaction of radiologists in the United States: a comparison between 2003 and 1995.

PURPOSE: To prospectively ascertain what characteristics of radiologists, their practices, and their work environment affect professional satisfaction and to describe recent changes in satisfaction. MATERIALS AND METHODS: Survey respondents were guaranteed confidentiality. Those who consented to participate were informed of the nature of the study. The authors analyzed nonindividually identified data from the American College of Radiology 2003 Survey of Radiologists, a nationally representative, confidential, stratified random-sample mail survey of radiologists in the United States, which had a 63% response rate, with a total of 1924 responses. Data were weighted to be representative of all U.S. radiologists and were analyzed by using univariate and multivariate analyses. The five answer options to questions regarding level of satisfaction corresponded to scores of +2, +1, 0, -1, and -2. Results were compared with those of a similar 1995 survey. RESULTS: Although 93% of radiologists enjoyed radiology very much or somewhat, the mean satisfaction score for posttraining professionally active radiologists decreased from 1.62 in 1995 to 1.47 in 2003. Thirty-two percent of radiologists reported enjoying radiology more than 5 years ago; 41% said they enjoyed it less. Excessive workload reduced current satisfaction and satisfaction relative to 5 years ago. Working in the Midwest enhanced current satisfaction and satisfaction relative to 5 years ago. Practice type and practice ownership had more varied effects on professional satisfaction; subspecialty type had relatively little effect. In 2003, medicolegal climate, workload, and reimbursement and/or financial pressures were the three most common reasons for decreased satisfaction. In 1995, interference from managed care; government regulations, control, and red tape; and increased administrative burden were the three most common reasons. Lifestyle and/or work hours, and income were the most common causes of increased satisfaction in 2003, but these were also often mentioned as causes of decreased satisfaction. CONCLUSION: Radiologists have higher levels of professional satisfaction than do other physicians; however, as with physicians overall, their satisfaction has decreased over time.

Women Radiologists in the United States: results from the American College of Radiology's 2003 Survey.

PURPOSE: To retrospectively evaluate data from the 2003 American College of Radiology (ACR) survey of diagnostic radiologists with regard to characteristics of women radiologists, their professional activities, and the practices in which they work. MATERIALS AND METHODS: The authors analyzed nonindividually identified data from the ACR's 2003 Survey of Radiologists, a stratified random sample survey that guaranteed respondents confidentiality. A cover letter assured respondents that no individually identifiable information would be disseminated; to further enhance confidentiality, survey operations were conducted by a contractor rather than by the ACR itself. There was a 63% response rate, with a total of 1924 responses. Responses were weighted to make them representative of all radiologists in the United States. Two-tailed z tests of percentages and means and multiple regression analysis were used to compare information for women radiologists with that for men radiologists. RESULTS: Twenty-four percent of radiologists in training (residents and fellows) and 18% of posttraining, professionally active radiologists were women. Forty-one percent of posttraining, professionally active women were younger than 45 years in comparison with 29% of men (P = .004). Women radiologists were more likely to have fellowship training than men (69% vs 60%, P = .007), although they were less likely than men to have a subspecialty certificate (16% vs 27%, P < .001). Thirty-nine percent of women and 16% of men worked part-time (P < .001). Women were more concentrated in academia (22% vs 14%, P = .009) and breast imaging (27% vs 6%, P < .001) than their male peers but were underrepresented in interventional radiology (2% vs 13%, P < .001) and neuroradiology (3% vs 10%, P < .001). In situations where radiologists are likely to be practice owners, fewer women than men were owners (75% vs 91%) (P = .011). Women reported the same level of enjoyment of radiology as did men. CONCLUSION: Women radiologists differ from men in regard to age, fellowship training, full- versus part-time employment, academic versus nonacademic practice, subspecialty practice, and practice ownership.

A portrait of breast imaging specialists and of the interpretation of mammography in the United States.

OBJECTIVE: Because of the importance of breast imaging as a radiology subspecialty and concerns about malpractice, the purpose of our study is to provide a detailed portrait of breast imaging specialists, their professional activities and practices, and information on all radiologists who interpret mammograms. MATERIALS AND METHODS: We analyzed data from the American College of Radiology's 2003 Survey of Radiologists, a large, stratified random sample survey that achieved a 63% response. Responses were weighted to make them representative of all radiologists in the United States. RESULTS: Approximately 10% of all radiologists, or 2,700-2,800 radiologists, are breast imaging specialists, but 61% of radiologists interpret mammograms, and only approximately 30% of mammograms are interpreted by breast imaging specialists. Of radiologists who reported that breast imaging was their primary specialty, only 21% took a fellowship in the field (much lower than for other subspecialties), 59% spent > or = 50% of their clinical work time in the specialty, 82% interpret > or = 2,000 mammograms annually, and only 11% (also well below other subspecialties) report that the main subspecialty society (the Society of Breast Imaging) is one of the two most important professional organizations for them. On average, breast imaging specialists, like other radiologists, report that their workload is about as heavy as desired. Their level of enjoyment of radiology does not differ significantly from average. CONCLUSION: Breast imaging appears not to be as strongly organized to raise awareness of and support for its problems as are other subspecialties. Although others find evidence of likely future problems, breast imaging specialists are not currently overworked or less satisfied in their profession than other radiologists, despite relatively low revenue generation and a particularly high risk of a malpractice lawsuit.

A portrait of interventional radiologists in the United States.

OBJECTIVE: In recognition of the emergence of interventional radiology as an important "new component of...radiology," the objective of our study was to provide an extensive and detailed portrait of interventional radiologists, their professional activities, and the practices in which they work. MATERIALS AND METHODS: We tabulated data from the American College of Radiology's 2003 Survey of Radiologists, a stratified random-sample survey that oversampled interventionalists and achieved a 63% response rate with a total of 1,924 responses. Responses were weighted to make them representative of all radiologists in the United States. We compared information about interventionalists with that for other radiologists. RESULTS: Depending on the definition of who is an interventionalist, 8.5-11.5% of radiologists are interventionalists. By most definitions, only slightly under half of interventionalists spend 70% or more of their clinical work time performing interventional procedures. Interventionalists work, on average, 56-58 hr weekly, a few hours longer than other radiologists. The average interventionalist performs procedures in five of the seven categories of procedures into which we divided interventional radiology, compared with one or two categories for other radiologists. The average interventionalist performs procedures in five of the seven broad categories (such as MRI, CT, and nuclear medicine) into which we divided all of radiology, much the same breadth of practice as other subspecialists and also as nonsubspecialists. CONCLUSION: Interventionalists have become a sizable group within radiology. They are in some ways like other radiologists and in other ways different, but they do not spend as much of their time in their subspecialty as some assume and, overall, are not as different.

Relative procedure intensity with self-referral and radiologist referral: extremity radiography.

PURPOSE: To compare the relative use of bilateral versus unilateral extremity radiographic examinations when patients are referred to radiologists for imaging (radiologist referred) versus when studies are performed in the referring physician's office (self-referred). MATERIALS AND METHODS: We reviewed 1 year of claims data for extremity radiographic examinations performed by a referring physician or referred to a radiology facility and claims data for related patient office visits. Data were analyzed for orthopedics, podiatry, and rheumatology, and data were divided by the practice pattern of the referring physician into pure self-referring, pure radiologist-referring, and mixed-referring categories. We compared the percentage of unilateral and bilateral studies and the number of unilateral and bilateral studies per 100 office visits in each setting. Statistical analysis of each comparison was performed with a one-tailed Z test. RESULTS: A total of 13 094 (14%) self-referred studies were bilateral, while 778 (10%) radiologist-referred studies were bilateral (P < .001). The rate of self-referred bilateral examinations was 2.21 times higher per 100 office visits than the rate of radiologist-referred bilateral examinations. Combined bilateral and unilateral use by self-referrers was only 1.86 times higher than use by radiologist-referrers. Orthopedists had no clinically meaningful difference in the percentage of self-referred and radiologist-referred bilateral studies, but they ordered 1.98 times as many studies per 100 visits when they self-referred studies. Self-referring podiatrists and rheumatologists ordered bilateral studies up to 3.25 times more frequently than did their radiologist-referring colleagues. Mixed-referring podiatrists had 2.70-times increased use of bilateral examinations when performing imaging in their offices, whereas mixed-referring rheumatologists had 6.40-times increase in that setting. CONCLUSION: Orthopedists, podiatrists, and rheumatologists use extremity radiography at a higher rate when they self-refer. Moreover, self-referring podiatrists and rheumatologists order radiographic examinations of increased intensity compared with radiologist-referring physicians.

Diagnostic accuracy of mammography, clinical examination, US, and MR imaging in preoperative assessment of breast cancer.

PURPOSE: To prospectively assess accuracy of mammography, clinical examination, ultrasonography (US), and magnetic resonance (MR) imaging in preoperative assessment of local extent of breast cancer. MATERIALS AND METHODS: Institutional review board approval and informed patient consent were obtained. Results of bilateral mammography, US, and contrast-enhanced MR imaging were analyzed from 111 consecutive women with known or suspected invasive breast cancer. Results were correlated with histopathologic findings. RESULTS: Analysis included 177 malignant foci in 121 cancerous breasts, of which 89 (50%) foci were palpable. Median size of 139 invasive foci was 18 mm (range, 2-107 mm). Mammographic sensitivity decreased from 100% in fatty breasts to 45% in extremely dense breasts. Mammographic sensitivity was highest for invasive ductal carcinoma (IDC) in 89 of 110 (81%) cases versus 10 of 29 (34%) cases of invasive lobular carcinoma (ILC) (P < .001) and 21 of 38 (55%) cases of ductal carcinoma in situ (DCIS) (P < .01). US showed higher sensitivity than did mammography for IDC, depicting 104 of 110 (94%) cases, and for ILC, depicting 25 of 29 (86%) cases (P < .01 for each). US showed higher sensitivity for invasive cancer than DCIS (18 of 38 [47%], P < .001). MR showed higher sensitivity than did mammography for all tumor types (P < .01) and higher sensitivity than did US for DCIS (P < .001), depicting 105 of 110 (95%) cases of IDC, 28 of 29 (96%) cases of ILC, and 34 of 38 (89%) cases of DCIS. In anticipation of conservation or no surgery after mammography and clinical examination in 96 breasts, additional tumor (which altered surgical approach) was present in 30. Additional tumor was depicted in 17 of 96 (18%) breasts at US and in 29 of 96 (30%) at MR, though extent was now overestimated in 12 of 96 (12%) at US and 20 of 96 (21%) at MR imaging. After combined mammography, clinical examination, and US, MR depicted additional tumor in another 12 of 96 (12%) breasts and led to overestimation of extent in another six (6%); US showed no detection benefit after MR imaging. Bilateral cancer was present in 10 of 111 (9%) patients; contralateral tumor was depicted mammographically in six and with both US and MR in an additional three. One contralateral cancer was demonstrated only clinically. CONCLUSION: In nonfatty breasts, US and MR imaging were more sensitive than mammography for invasive cancer, but both MR imaging and US involved risk of overestimation of tumor extent. Combined mammography, clinical examination, and MR imaging were more sensitive than any other individual test or combination of tests.

RADPEER quality assurance program: a multifacility study of interpretive disagreement rates.

PURPOSE: To develop and test a radiology peer review system that adds minimally to workload, is confidential, uniform across practices, and provides useful information to meet the mandate for "evaluation of performance in practice" that is forthcoming from the American Board of Medical Specialties as one of the four elements of maintenance of certification. METHOD: RADPEER has radiologists who review previous images as part of a new interpretation record their ratings of the previous interpretations on a 4-point scale. Reviewing radiologists' ratings of 3 and 4 (disagreements in nondifficult cases) are reviewed by a peer review committee in each practice to judge whether they are misinterpretations by the original radiologists. Final ratings are sent for central data entry and analysis. A pilot test of RADPEER was conducted in 2002. RESULTS: Fourteen facilities participated in the pilot test, submitting a total of 20,286 cases. Disagreements in difficult cases (ratings of 2) averaged 2.9% of all cases. Committee-validated misinterpretations in nondifficult cases averaged 0.8% of all cases. There were considerable differences by modality. There were substantial differences across facilities; few of these differences were explicable by mix of modalities, facility size or type, or being early or late in the pilot test. Of 31 radiologists who interpreted over 200 cases, 2 had misinterpretation rates significantly (P < .05) above what would be expected given their individual mix of modalities and the average misinterpretation rate for each modality in their practice. CONCLUSIONS: A substantial number of facilities participated in the pilot test, and all maintained their participation throughout the year. Data generated are useful for the peer review of individual radiologists and for showing differences by modality. RADPEER is now operational and is a good solution to the need for a peer review system with the desirable characteristics listed above.

Disagreement in interpretation: a method for the development of benchmarks for quality assurance in imaging.

PURPOSE: To calculate disagreement rates by radiologist and modality to develop a benchmark for use in the quality assessment of imaging interpretation. METHODS: Data were obtained from double readings of 2% of daily cases performed for quality assurance (QA) between 1997 and 2001 by radiologists at a group practice in Dallas, Texas. Differences across radiologists in disagreement rates, with adjustments for case mix, were examined for statistical significance using simple comparisons of means and multivariate logistic regression. RESULTS: In 6703 cases read by 26 radiologists, the authors found an overall disagreement rate of 3.48%, with a disagreement rate of 3.03% for general radiology, 3.61% for diagnostic mammography, 5.79% for screening mammography, and 4.07% for ultrasound. Disagreement rates by radiologist for the 10 radiologists with at least 20 cases ranged from 2.04% to 6.90%. Multivariate analysis found that controlling for other factors, both differences among radiologists and across modalities, statistically significantly contributed to differences in disagreement rates. CONCLUSION: Disagreement rates varied by modality and by radiologist. Double reading studies such as these are a useful tool to rate quality of imaging interpretation and to establish benchmarks for QA.