Patterns of Screening Recall Behavior Among Subspecialty Breast Radiologists.

RATIONALE AND OBJECTIVES: Determine whether there are patterns of lesion recall among breast imaging subspecialists interpreting screening mammography, and if so, whether recall patterns correlate to morphologies of screen-detected cancers. MATERIALS AND METHODS: This Institutional Review Board-approved, retrospective review included all screening examinations January 3, 2012-October 1, 2018 interpreted by fifteen breast imaging subspecialists at a large academic medical center and two outpatient imaging centers. Natural language processing identified radiologist recalls by lesion type (mass, calcifications, asymmetry, architectural distortion); proportions of callbacks by lesion types were calculated per radiologist. Hierarchical cluster analysis grouped radiologists based on recall patterns. Groups were compared to overall practice and each other by proportions of lesion types recalled, and overall and lesion-specific positive predictive value-1 (PPV1). RESULTS: Among 161,859 screening mammograms with 13,086 (8.1%) recalls, Hierarchical cluster analysis grouped 15 radiologists into five groups. There was substantial variation in proportions of lesions recalled: calcifications 13%-18% (Chi-square 45.69, p < 0.00001); mass 16%-44% (Chi-square 498.42, p < 0.00001); asymmetry 13%-47% (Chi-square 660.93, p < 0.00001) architectural distortion 6%-20% (Chi-square 283.81, p < 0.00001). Radiologist groups differed significantly in overall PPV1 (range 5.6%-8.8%; Chi-square 17.065, p = 0.0019). PPV1 by lesion type varied among groups: calcifications 9.2%-15.4% (Chi-square 2.56, p = 0.6339); mass 5.6%-8.5% (Chi-square 1.31, p = 0.8597); asymmetry 3.4%-5.9% (Chi-square 2.225, p = 0.6945); architectural distortion 5.6%-10.8% (Chi-square 5.810, p = 0.2138). Proportions of recalled lesions did not consistently correlate to proportions of screen-detected cancer. CONCLUSION: Breast imaging subspecialists have patterns for screening mammography recalls, suggesting differential weighting of imaging findings for perceived malignant potential. Radiologist recall patterns are not always predictive of screen-detected cancers nor lesion-specific PPV1s.

Electronic Worklist Improves Timeliness of Screening Mammogram Interpretation in an Urban Underserved Population.

OBJECTIVES: To evaluate the impact of an electronic workflow update on screening mammography turnaround time and time to diagnostic imaging for mammography performed on our urban mobile mammography van and at an urban community health center. METHOD: Prior to 10/15/2019, screening exams for the mammography van and urban community health center were made available for interpretation to a single designated radiologist via a manually generated paper list. On 10/15/2019, screening exams were routed electronically onto PACS for any breast radiologist across our Network to interpret. Screening mammogram turnaround time (defined as time form image acquisition to report finalization), time to diagnostic imaging, and time to tissue sampling were collected for pre- and post-implementation periods (6/1-9/30/2019 and 11/1/2019-2/29/2020, respectively) and compared via student t-test and statistical process control analyses. RESULTS: The number of screening exams in the pre- and post-implementation periods were 851 and 728 exams, respectively. Patients were predominately Black and/or African American (400/1579, 25%), non-English speaking (858/1579, 54%) and insured by Medicaid (751/1579, 48%). After implementation of the electronic workflow, turnaround time decreased from 101.0 to 36.4 hours (63.9%, P <0.001) and statistical process control analyses showed sustained decrease in mean turnaround time. However, mean time to diagnostic imaging and tissue sampling were unchanged after implementation (39 vs 45, days; P = 0.330 and 43 vs 59; P = 0.187, respectively). CONCLUSION: Electronic workflow management can reduce screening mammography turnaround time for underserved populations, but additional efforts are warranted to improve time to imaging follow-up for abnormal screening mammograms.

Screening Mammography Performance Metrics of 2D Digital Mammography Versus Digital Breast Tomosynthesis in Women With a Personal History of Breast Cancer.

BACKGROUND. Patients with a history of breast cancer are at higher risk of subsequent breast cancers and need close clinical and imaging follow-up. Limited data are available on screening of these patients with digital breast tomosynthesis (DBT) versus full-field digital mammography (FFDM). OBJECTIVE. The purpose of this study was to evaluate the screening mammography performance of DBT compared with FFDM among patients with a history of breast cancer undergoing imaging at a large academic oncology center. METHODS. This retrospective study included consecutively registered patients with a personal history of breast cancer treated with mastectomy or lumpectomy who underwent screening FFDM from October 2014 through September 2016 (5706 examinations of 4091 patients) or screening DBT from February 2017 through December 2018 (4440 examinations of 3647 patients). An institutional mammographic database was queried to obtain imaging type, breast density, history of mastectomy or lumpectomy, and BI-RADS category. An institutional breast cancer registry identified cancer diagnoses. Screening performance metrics were compared between FFDM and DBT groups. RESULTS. Recall rate was significantly lower with DBT than with FFDM (7.9% vs 10.1%; p < .001). DBT and FFDM did not differ in PPV1 (7.7% vs 6.1%; p = .36) or cancer detection rate (CDR) (6.1/1000 vs 6.0/1000; p = .97). Sensitivity was 96.4% for DBT and 71.4% for FFDM (p = .008). Specificity was 92.3% for DBT and 90.0% for FFDM (p < .001). With stratification by breast density, patients with nondense breast tissue had a lower recall rate with DBT than with FFDM (5.9% vs 8.8%; p < .001) and a nonsignificant increase in PPV1 (12.0% vs 6.4%; p = .05). The metrics were not otherwise different between DBT and FFDM among patients with nondense and those with dense breast tissue. Recall rates were lower with DBT than with FFDM among both patients who underwent mastectomy (7.8% vs 9.1%; p = .09) and those who underwent lumpectomy (7.9% vs 11.0%; p = .002). PPV1 and CDR were not different between DBT and FFDM among patients who underwent mastectomy and those who underwent lumpectomy. CONCLUSION. For patients with a personal history of breast cancer who have nondense breasts, the use of DBT as opposed to FFDM reduces recall rate and improves sensitivity and specificity. CDR and PPV1 remain unchanged. CLINICAL IMPACT. For women with a personal history of breast cancer and nondense breasts, DBT offers the potential to maintain the benefits of early cancer detection while reducing the potential harms of false-positive findings.

Predictors of Self-Reported Burnout Among Radiology Faculty at a Large Academic Medical Center.

OBJECTIVE: Determine predictors of self-reported burnout among academic radiologists. METHODS: In 2017, radiologists at an urban medical center completed the Stanford Wellness Survey, rating burnout via Likert scale (0: no burnout; 1: occasional stress, no burnout; 2: one or more burnout symptoms; 3: persistent burnout symptoms; 4: completely burned out). Univariate analyses assessed age, gender, family situation, clinical versus research focus, and academic rank for association with burnout (Likert scale ≥ 2). Responses in 11 domains querying definitions of burnout (professional fulfillment, emotional exhaustion, interpersonal disengagement), individual factors (sleep-related impairment, self-compassion, negative work impact on personal relationships), institutional factors (perceived appreciation, control over schedule, organizational or personal values alignment, electronic health record experience, supervisor's leadership quality) were evaluated for association with burnout, using χ2 and logistic regression to calculate odds ratios (ORs). RESULTS: In 159 of 204 (77.9%) completed radiologist surveys, 35.2% (56 of 159) reported burnout. Age < 40 years (P = .0068) and clinical focus (P = .0111) were significantly associated with burnout. In univariate analysis, all domains except electronic health record were statistically significant: emotional exhaustion (OR = 1.93, P < .0001); professional fulfillment (OR = 0.78, P < .0001); self-compassion (OR = 1.36, P < .0001); perceived appreciation (OR = 0.78, P < .0001); sleep-related impairment (OR = 1.20, P < .0001); supervisor's leadership quality (OR = 0.91, P < .0001); interpersonal disengagement (OR = 1.31, P < .0001); organizational or personal values alignment (OR = 0.87, P = .0004); negative work impact on personal relationships (OR = 1.10, P = .0070); control over schedule (OR = 0.80, P = .0054); electronic health record experience (OR=1.03, P = .5392). DISCUSSION: Nearly all questions significantly predicted self-reported burnout, observed in over one-third of academic radiologists. Younger age and clinical focus were associated with burnout. Initiatives targeting individual factors (eg, sleep impairment, self-compassion) and institutional factors (eg, physician appreciation, leadership-faculty interactions) may reduce burnout.

Comparing Diagnostic Performance of Digital Breast Tomosynthesis and Full-Field Digital Mammography.

OBJECTIVE: Compare diagnostic performance of screening full-field digital mammography (FFDM), a hybrid FFDM and digital breast tomosynthesis (DBT) environment, and DBT only. MATERIALS AND METHODS: This institutional review board-approved, retrospective study consisted of all patients undergoing screening mammography at an urban academic medical center and outpatient imaging facility between January 1, 2011, and December 31, 2017. We used the electronic health record data warehouse to extract report data and patient demographics. A validated natural language processing algorithm extracted BI-RADS score from each report. An institutional cancer registry identified cancer diagnoses. Primary outcomes of recall rate, cancer detection rate (CDR), and positive predictive value 1 (PPV1) were calculated for three periods: FFDM-only environment, hybrid environment, and DBT-only environment. A χ2 test was used to compare recall rate, CDR, and PPV1. RESULTS: A total of 179,028 screening mammograms comprised the study cohort: 41,818 (23.3%) during the FFDM-only period, 83,125 (46.4%) during the hybrid period, and 54,084 (30.2%) during the DBT-only period. Recall rates were 10.4% (4,279 of 41,280) for the FFDM-only period, 10.6% (8,761 of 82,917) for the hybrid period, and 10.8% (5,850 of 54,020) for the DBT-only period (P = .96). CDR (cancers per 1,000 examinations) was 2.6 per 1,000, 4.9 per 1,000, and 6.0 per 1,000 for FFDM only, hybrid, and DBT only, respectively (P < .01). PPV1s (number of cancers per number of recalls) were 2.5% for the FFDM-only period, 4.6% for the hybrid period, and 5.6% for the DBT-only period (P < .01). CONCLUSION: Recall rates were not significantly different within the three periods in the breast imaging practice. However, PPV1 and CDR were significantly higher with DBT only.

Radiologists' Self-Assessment Versus Peer Assessment of Perceived Probability of Recommending Additional Imaging.

OBJECTIVE: Determine radiologist ability to accurately select the probability of recommendation of additional imaging (RAI) for themselves and colleagues when arrayed in a feedback report. METHODS: In this institutional review board-approved study, we analyzed 318,366 diagnostic imaging reports from examinations performed in the radiology department of a large quaternary teaching hospital during calendar year 2016. A validated machine learning algorithm identified reports containing RAI. A multivariable logistic regression model was then used to determine the probability of RAI. In 2018, an e-mailed survey asked radiologists to identify their own RAI probability and that of their colleagues from a report arrayed lowest to highest. Radiologists were grouped into quartiles based on their RAI probability. χ2 Analysis compared self-assessment and assessment of colleagues between quartiles. RESULTS: Forty-eight of 57 radiologists completed the survey (84.2%). Fourteen (29.2%) accurately self-identified their RAI probability (chose the correct quartile); 34 (70.8%) did not. There was no statistically significant difference between quartiles of radiologists and their ability to self-identify their RAI probability (ie, radiologists in the bottom or top quartile of RAI probabilities did not correctly predict their RAI probability). However, radiologists were better able to identify the RAI probability of their colleagues who were in the top and bottom quartiles. DISCUSSION: Radiologists were unable to estimate their own RAI probability but were better at predicting the RAI probability of colleagues. Given that radiologists, and physicians in general, may be poor evaluators of their own performance, objective assessment tools are likely needed to help reduce unwarranted variation.

Unscheduled Radiologic Examination Orders in the Electronic Health Record: A Novel Resource for Targeting Ambulatory Diagnostic Errors in Radiology.

PURPOSE: The aim of this study was to assess the prevalence of unscheduled radiologic examination orders in an electronic health record, and the proportion of unscheduled orders that are clinically necessary, by modality. METHODS: This retrospective study was conducted from January to October 2016 at an academic institution. All unscheduled radiologic examination orders were retrieved for seven modalities (CT, MR, ultrasound, obstetric ultrasound, bone densitometry, mammography, and fluoroscopy). After excluding duplicates, 100 randomly selected orders from each modality were assigned to two physician reviewers who classified their clinical necessity, with 10% overlap. Interannotator agreement was assessed using κ statistics, the percentage of clinically necessary unscheduled orders was compared, and χ2 analysis was used to assess differences by modality. RESULTS: A total 494,503 radiologic examination orders were placed during the study period. After exclusions, 33,546 unscheduled orders were identified, 7% of all radiologic examination orders. Among 700 reviewed unscheduled orders, agreement was substantial (κ = 0.63). Eighty-seven percent of bone densitometric examinations and sixty-five percent of mammographic studies were considered clinically necessary, primarily for follow-up management. The majority of orders in each modality were clinically necessary, except for CT, obstetric ultrasound, and fluoroscopy (P < .0001). CONCLUSIONS: Large numbers of radiologic examination orders remain unscheduled in the electronic health record. A substantial portion are clinically necessary, representing potential delays in executing documented provider care plans. Clinically unnecessary unscheduled orders may inadvertently be scheduled and performed. Identifying and performing clinically necessary unscheduled radiologic examination orders may help reduce diagnostic errors related to diagnosis and treatment delays and enhance patient safety, while eliminating clinically unnecessary unscheduled orders will help avoid unneeded testing.

Variation in Follow-up Imaging Recommendations in Radiology Reports: Patient, Modality, and Radiologist Predictors.

Background Variation between radiologists when making recommendations for additional imaging and associated factors are, to the knowledge of the authors, unknown. Clear identification of factors that account for variation in follow-up recommendations might prevent unnecessary tests for incidental or ambiguous image findings. Purpose To determine incidence and identify factors associated with follow-up recommendations in radiology reports from multiple modalities, patient care settings, and imaging divisions. Materials and Methods This retrospective study analyzed 318 366 reports obtained from diagnostic imaging examinations performed at a large urban quaternary care hospital from January 1 to December 31, 2016, excluding breast and US reports. A subset of 1000 reports were randomly selected and manually annotated to train and validate a machine learning algorithm to predict whether a report included a follow-up imaging recommendation (training-and-validation set consisted of 850 reports and test set of 150 reports). The trained algorithm was used to classify 318 366 reports. Multivariable logistic regression was used to determine the likelihood of follow-up recommendation. Additional analysis by imaging subspecialty division was performed, and intradivision and interradiologist variability was quantified. Results The machine learning algorithm classified 38 745 of 318 366 (12.2%) reports as containing follow-up recommendations. Average patient age was 59 years ± 17 (standard deviation); 45.2% (143 767 of 318 366) of reports were from male patients. Among 65 radiologists, 57% (37 of 65) were men. At multivariable analysis, older patients had higher rates of follow-up recommendations (odds ratio [OR], 1.01 [95% confidence interval {CI}: 1.01, 1.01] for each additional year), male patients had lower rates of follow-up recommendations (OR, 0.9; 95% CI: 0.9, 1.0), and follow-up recommendations were most common among CT studies (OR, 4.2 [95% CI: 4.0, 4.4] compared with radiography). Radiologist sex (P = .54), presence of a trainee (P = .45), and years in practice (P = .49) were not significant predictors overall. A division-level analysis showed 2.8-fold to 6.7-fold interradiologist variation. Conclusion Substantial interradiologist variation exists in the probability of recommending a follow-up examination in a radiology report, after adjusting for patient, examination, and radiologist factors. © RSNA, 2019 See also the editorial by Russell in this issue.

A Clinical Model for the Early Diagnosis of Acute Pancreatitis in the Emergency Department.

OBJECTIVE: This study aimed to develop a diagnostic model that predicts acute pancreatitis (AP) risk before imaging. METHODS: Emergency department patients with serum lipase elevated to 3 times the upper limit of normal or greater were identified retrospectively (September 1, 2013-August 31, 2015). An AP diagnosis was established by expert review of full hospitalization records. Candidate predictors included demographic and clinical characteristics at presentation. Using a derivation set, a multivariable logistic regression model and corresponding point-based scoring system was developed to predict AP. Discrimination accuracy and calibration were assessed in a separate validation set. RESULTS: In 319 eligible patients, 182 (57%) had AP. The final model (area under curve, 0.92) included 8 predictors: number of prior AP episodes; history of cholelithiasis; no abdominal surgery (prior 2 months); time elapsed from symptom onset; pain localized to epigastrium, of progressively worsening severity, and severity level at presentation; and extent of lipase elevation. At a diagnostic risk threshold of 8 points or higher (≥99%), the model identified AP with a sensitivity of 45%, and a specificity and a positive predictive value of 100%. CONCLUSIONS: In emergency department patients with lipase elevated to 3 times the upper limit of normal or greater, this model helps identify AP risk before imaging. Prospective validation studies are needed to confirm diagnostic accuracy.

Renal cancer at unenhanced CT: imaging features, detection rates, and outcomes.

PURPOSE: To describe and quantify the rate of detection of renal cancer on unenhanced CT. METHODS: This retrospective, HIPAA-compliant study was approved by the Institutional Review Board. Electronic health records for all patients who underwent unenhanced abdominal CT at our institution between 2000 and 2005 were reviewed to identify patients subsequently diagnosed with renal cancer during a follow-up period of up to 12 years. Images were reviewed to determine if the cancer was visible at index (first) unenhanced CT and their findings recorded. Original radiology reports were reviewed to determine whether the renal cancer was reported; Fisher's Exact Test compared imaging features of detected and missed cancers. Clinical outcomes including time until diagnosis and stage at diagnosis were used to assess the potential impact of missed cancers. RESULTS: Of 15,695 patients, 82 (0.52%) were diagnosed with renal cancer. Of these, 43/82 (52%) cancers were retrospectively detectable on index unenhanced CT. Among retrospectively detectable cancers, 63% (27/43) were originally detected and reported on index CT and 37% (16/43) were missed. Size was the only feature associated with detection; 83% (20/24) of cancers > 3.0 cm were detected versus 37% (7/19) of cancers ≤ 3.0 cm (p = 0.0036). Although none of the 16 missed renal cancers developed metastases between index CT and time of diagnosis (median 33.5 months), 4 (25%) progressed in stage. CONCLUSIONS: Renal cancer was rare in patients undergoing unenhanced abdominal CT. Over one-third of potentially detectable cancers were missed prospectively. However, missed cancers did not metastasize and infrequently progressed in stage before being diagnosed.