Radiology Patient Outcome Measures: Impact of a Departmental Pay-for-Performance Initiative on Key Quality and Safety Measures.

OBJECTIVE: Assess impact of a multifaceted pay-for-performance (PFP) initiative on radiologists' behavior regarding key quality and safety measures. METHODS: This institutional review board-approved prospective study was performed at a large, 12-division urban academic radiology department. Radiology patient outcome measures were implemented October 1, 2017, measuring report signature timeliness, critical results communication, and generation of peer-learning communications between radiologists. Subspecialty division-wide and individual radiologist targets were specified, performance was transparently communicated on an intranet dashboard updated daily, and performance was financially incentivized (5% of salary) quarterly. We compared outcomes 12 months pre- versus 12 months post-PFP implementation. Primary outcome was monthly 90th percentile time from scan completion to final report signature (CtoF). Secondary outcomes were percentage timely closed-loop communication of critical results and number of division-wide peer-learning communications. Statistical process control analysis and parallel coordinates charts were used to assess for temporal trends. RESULTS: In all, 144 radiologists generated 1,255,771 reports (613,273 pre-PFP) during the study period. Monthly 90th percentile CtoF exhibited an absolute decrease of 4.4 hours (from 21.1 to 16.7 hours) and a 20.9% relative decrease post-PFP. Statistical process control analysis demonstrated significant decreases in 90th percentile CtoF post-PFP, sustained throughout the study period (P < .003). Between 95% (119 of 125, July 1, 2018, to September 30, 2018) and 98.4% (126 of 128, October 1, 2017, to December 31, 2017) of radiologists achieved >90% timely closure of critical alerts; all divisions exceeded the target of 90 peer-learning communications each quarter (range: 97-472) after January 1, 2018. DISCUSSION: Implementation of a multifaceted PFP initiative using well-defined radiology patient outcome measures correlated with measurable improvements in radiologist behavior regarding key quality and safety parameters.

Breast Cancer Screening with Digital Breast Tomosynthesis: Are Initial Benefits Sustained?

Background Performance metrics with digital breast tomosynthesis (DBT) are based on early experiences. There is limited research on whether the benefits of DBT are sustained. Purpose To determine whether improved screening performance metrics with DBT are sustained over time at the population level and after the first screening round at the individual level. Materials and Methods A retrospective review was conducted of screening mammograms that had been obtained before DBT implementation (March 2008 to February 2011, two-dimensional digital mammography [DM] group) and for 5 years after implementation (January 2013 to December 2017, DBT1-DBT5 groups, respectively). Patients who underwent DBT were also categorized according to the number of previous DBT examinations they had undergone. Performance metrics were compared between DM and DBT groups and between patients with no previous DBT examinations and those with at least one prior DBT examination by using multivariable logistic regression models. Results The DM group consisted of 99 582 DM examinations in 55 086 women (mean age, 57.3 years ± 11.6 [standard deviation]). The DBT group consisted of 205 048 examinations in 76 276 women (mean age, 58.2 years ± 11.2). There were no differences in the cancer detection rate (CDR) between DM and DBT groups (4.6-5.8 per 1000 examinations, P = .08 to P = .95). The highest CDR was observed with a woman's first DBT examination (6.1 per 1000 examinations vs 4.4-5.7 per 1000 examinations with at least one prior DBT examination, P = .001 to P = .054). Compared with the DM group, the DBT1 group had a lower abnormal interpretation rate (AIR) (adjusted odds ratio [AOR], 0.85; P < .001), which remained reduced in the DBT2, DBT3, and DBT5 groups (P < .001 to P = .02). The reduction in AIR was also sustained after the first examination (P < .001 to P = .002). Compared with the DM group, the DBT1 group had a higher specificity (AOR, 1.20; P < .001), which remained increased in DBT2, DBT3, and DBT5 groups (P < .001 to P = .004). The increase in specificity was also sustained after the first examination (P < .001 to P = .01). Conclusion The benefits of reduced false-positive examinations and higher specificity with screening tomosynthesis were sustained after the first screening round at the individual level. © RSNA, 2020 See also the editorial by Taourel in this issue.

Impact of Background Parenchymal Enhancement on Diagnostic Performance in Screening Breast MRI.

RATIONALE AND OBJECTIVES: To evaluate the impact of background parenchymal enhancement (BPE) on diagnostic performance in screening breast magnetic resonance imaging (MRI). MATERIALS AND METHODS: Consecutive screening breast MRIs performed at our institution from 2011 to 2014 were reviewed in a HIPAA-compliant manner with institutional review board approval. BPE was extracted from radiology reports and examinations grouped into minimal/mild (lower) or moderate/marked (higher) BPE. Performance measures were compared between the two groups with Pearson's χ2 test and with logistic regression to adjust for possible confounders of age, screening indication, mammographic density, available prior MRI, and examination year, using lower BPE as the reference group. RESULTS: For 4686 screening MRIs performed in 2446 women, BPE was reported as minimal or mild for 3975 (85%) examinations and moderate or marked for 711(15%). Following logistic regression to adjust for multiple confounders, abnormal interpretation rate (AIR) significantly differed between the two BPE groups. AIR was 13% (89/711) in the higher BPE group versus 7% (295/3975) in the lower BPE group with an adjusted odds ratio of 1.37 (95% confidence interval: 1.03, 1.82). After adjustment, all other performance metrics, including cancer detection rate, positive predictive value, sensitivity, and specificity did not significantly differ between the two BPE groups (P > 0.05). CONCLUSION: Higher BPE on screening MRI is associated with higher abnormal interpretation rate, with no impact on cancer detection rate, sensitivity, or specificity.

Comparing Tumor Characteristics and Rates of Breast Cancers Detected by Screening Digital Breast Tomosynthesis and Full-Field Digital Mammography.

OBJECTIVE. The purpose of this study was to compare the cancer detection rates (CDRs), tumor types, and characteristics between screening digital breast tomosynthesis (DBT) and screening full-field digital mammography (FFDM) in a matched patient population in a large academic breast imaging practice with mixed DBT and FFDM technology. MATERIALS AND METHODS. In this retrospective study, we reviewed consecutive screening FFDM and DBT examinations performed between October 2012 and September 2014. To control for nonrandomized selection of FFDM versus DBT examinations, we applied propensity score matching on the basis of patient age, imaging site, and prior imaging findings. An institutional breast cancer registry identified cancer diagnoses. CDR and tumor type, grade, receptor, nodal status, and size were compared between matched FFDM and DBT groups. RESULTS. Sixty-one cancers were detected in the matched screening cohort of DBT (n = 9817) and FFDM (n = 14,180) examinations. CDR was higher with DBT than with FFDM for invasive cancers (2.8 vs 1.3, p = 0.01), minimal cancers (2.4 vs 1.2, p = 0.03), estrogen receptor-positive invasive cancers (2.6 vs 1.1, p = 0.01), and node-negative invasive cancers (2.3 vs 1.1, p = 0.02.), respectively. The ratio of screen-detected invasive cancers to ductal carcinoma in situ on DBT (3.0) was not significantly different from that on FFDM (2.6) (p = 0.79). CONCLUSION. DBT results in an overall increase in CDR irrespective of the tumor type, size, or grade of cancer.

Performance of Screening Breast MRI After Negative Full-Field Digital Mammography Versus After Negative Digital Breast Tomosynthesis in Women at Higher Than Average Risk for Breast Cancer.

OBJECTIVE: The objective of our study was to compare the supplemental cancer yield and performance of breast MRI in women at higher-than-average risk for breast cancer after negative 2D full-field digital mammography (FFDM) or negative digital breast tomosynthesis (DBT). MATERIALS AND METHODS: Retrospective review identified 4418 screening breast MRI examinations: 2291 were performed from January 2010 through January 2012 of patients with a negative FFDM examination in the 12 months before MRI (FFDM group), and 2127 were performed from January 2013 through January 2015 of patients with a negative DBT examination in the 12 months before MRI (DBT group). Screening indications included genetic predisposition, personal history of breast cancer or high-risk lesion, prior chest irradiation, family history, or other risk factors conferring a lifetime risk of greater than 20%. Supplemental cancer detection rate (CDR), abnormal interpretation rate (AIR), and positive predictive values (PPVs) were estimated with 95% exact CIs. Logistic regression analysis, adjusting for differences in patient demographics, was used to compare metrics. RESULTS: There was no significant difference in the CDR of MRI in the FFDM group versus the DBT group (11 vs 16 cancers per 1000 examinations, respectively; odds ratio, 1.4; 95% CI, 0.4-1.2; p = 0.23). The AIR, PPV1, PPV2, and PPV3 were 7.4%, 15%, 23%, and 28% for the FFDM group and 7.3%, 22%, 33%, and 35% for the DBT group, with no statistical differences. Of the cancers detected in both groups, the majority were invasive, less than 1 cm, and node-negative. CONCLUSION: In women at higher-than-average risk of breast cancer screened with DBT, the supplemental CDR of MRI is similar to that of MRI after FFDM screening, with most cancers being invasive, subcentimeter, and node-negative.

Breast Cancer Characteristics Associated with 2D Digital Mammography versus Digital Breast Tomosynthesis for Screening-detected and Interval Cancers.

Purpose To determine whether the rates and tumor characteristics of screening-detected and interval cancers differ for two-dimensional digital mammography (DM) versus digital breast tomosynthesis (DBT) mammography. Materials and Methods Consecutive screening mammograms from January 2009 to February 2011 (DM group, before DBT integration) and from January 2013 to February 2015 (DBT group, after complete DBT integration) were reviewed. Cancers were considered screening detected if diagnosed within 365 days of a positive screening examination and interval if diagnosed within 365 days of a negative screening examination. Z tests were used to compare cancers on DM versus DBT examinations. Results A total of 948 breast cancers were diagnosed after 78 385 DM and 76 896 DBT examinations. Although the overall rate of screening-detected cancers was similar with DM and DBT (5.0 vs 5.0 per 1000 examinations, P = .98), a higher proportion of screening-detected cancers were invasive rather than in situ with DBT (74.2% [287 of 387] vs 66.0% [260 of 394], P = .01). There were no significant differences in tumor characteristics, including size at pathologic examination, grade, hormone receptor status, and nodal status, between the screening-detected invasive cancers on DM versus DBT (P = .09-.99). The rate of interval cancers was similar with DM and DBT (1.1 vs 1.1 per 1000 examinations, P = .84). Compared with symptomatic interval cancers, magnetic resonance imaging-detected interval cancers were more likely to be minimal cancers. Conclusion The overall rates of screening-detected and interval cancers are similar with DM and DBT, but a higher proportion of screening-detected cancers are invasive rather than in situ with DBT. © RSNA, 2017.

Addition of tomosynthesis to conventional digital mammography: effect on image interpretation time of screening examinations.

PURPOSE: To determine the effect of implementing a screening tomosynthesis program on real-world clinical performance by quantifying differences between interpretation times for conventional screening mammography and combined tomosynthesis and mammography for multiple participating radiologists with a wide range of experience in a large academic center. MATERIALS AND METHODS: In this HIPAA-compliant, institutional review board-approved study, 10 radiologists prospectively read images from screening digital mammography or screening combined tomosynthesis and mammography examinations for 1-hour-long uninterrupted sessions. Images from 3665 examinations (1502 combined and 2163 digital mammography) from July 2012 to January 2013 were interpreted in at least five sessions per radiologist per modality. The number of cases reported during each session was recorded for each reader. The experience level for each radiologist was also correlated to the average number of cases reported per hour. Analysis of variance was used to assess the number of studies interpreted per hour. A linear regression model was used to evaluate correlation between breast imaging experience and time taken to interpret images from both modalities. RESULTS: The mean number of studies interpreted in hour was 23.8 ± 0.55 (standard deviation) (range, 14.4-40.4) for combined tomosynthesis and mammography and 34.0 ± 0.55 (range, 20.4-54.3) for digital mammography alone. A mean of 10.2 fewer studies were interpreted per hour during combined tomosynthesis and mammography compared with digital mammography sessions (P < .0001). The mean interpretation time was 2.8 minutes ± 0.9 (range, 1.5-4.2 minutes) for combined tomosynthesis and mammography and 1.9 minutes ± 0.6 (range, 1.1-3.0) for digital mammography; interpretation time with combined tomosynthesis and mammography was 0.9 minute longer (47% longer) compared with digital mammography alone (P < .0001). With the increase in years of breast imaging experience, the overall additional time required to read images from combined tomosynthesis and mammography examinations decreased (R(2) = 0.52, P = .03). CONCLUSION: Addition of tomosynthesis to mammography results in increased time to interpret images from screening examinations compared with time to interpret images from conventional digital mammography alone.

Informatics in radiology: Render: an online searchable radiology study repository.

Radiology departments are a rich source of information in the form of digital radiology reports and images obtained in patients with a wide spectrum of clinical conditions. A free text radiology report and image search application known as Render was created to allow users to find pertinent cases for a variety of purposes. Render is a radiology report and image repository that pools researchable information derived from multiple systems in near real time with use of (a) Health Level 7 links for radiology information system data, (b) periodic file transfers from the picture archiving and communication system, and (c) the results of natural language processing (NLP) analysis. Users can perform more structured and detailed searches with this application by combining different imaging and patient characteristics such as examination number; patient age, gender, and medical record number; and imaging modality. Use of NLP analysis allows a more effective search for reports with positive findings, resulting in the retrieval of more cases and terms having greater relevance. From the retrieved results, users can save images, bookmark examinations, and navigate to an external search engine such as Google. Render has applications in the fields of radiology education, research, and clinical decision support.

Effect of computerized order entry with integrated decision support on the growth of outpatient procedure volumes: seven-year time series analysis.

PURPOSE: To determine the effect of a computerized radiology order entry (ROE) and decision support (DS) system on growth rate of outpatient computed tomography (CT), magnetic resonance (MR) imaging, and ultrasonography (US) procedure volumes over time at a large metropolitan academic medical center. MATERIALS AND METHODS: Institutional review board approval was obtained for this study of deidentified aggregate administrative data. The research was compliant with HIPAA; informed consent was waived. This was a retrospective study of outpatient advanced imaging utilization before, during, and after implementation of a Web-based ROE and DS system. Dependent variables were the quarterly volumes of outpatient CT, MR imaging, and US examinations from quarter 4 of 2000 through quarter 4 of 2007. Outpatient visits during each quarter were included as control variables. These data were analyzed as three separate time series with piecewise linear regression for simultaneous estimation of quarterly examination volume trends before and after ROE and DS system implementation. This procedure was repeated with log-transformed quarterly volumes to estimate percentage growth rates. RESULTS: There was a significant decrease in CT volume growth (274 per quarter) and growth rate (2.75% per quarter) after ROE and DS system implementation (P < .001). For MR imaging, growth rate decreased significantly (1.2%, P = .016) after ROE and DS system implementation; however, there was no significant change in quarterly volume growth. With US, quarterly volume growth (n = 98, P = .014) and growth rate (1.3%, P = .001) decreased significantly after ROE implementation. These changes occurred during a steady growth in clinic visit volumes in the associated referral practices. CONCLUSION: Substantial decreases in the growth of outpatient CT and US procedure volume coincident with ROE implementation (supplemented by DS for CT) were observed. The utilization of outpatient MR imaging decreased less impressively, with only the rate of growth being significantly lower after interventions were in effect.

Use of Radcube for extraction of finding trends in a large radiology practice.

The purpose of our study was to demonstrate the use of Natural Language Processing (Leximer), along with Online Analytic Processing, (NLP-OLAP), for extraction of finding trends in a large radiology practice. Prior studies have validated the Natural Language Processing (NLP) program, Leximer for classifying unstructured radiology reports based on the presence of positive radiology findings (F (POS)) and negative radiology findings (F (NEG)). The F (POS) included new relevant radiology findings and any change in status from prior imaging. Electronic radiology reports from 1995-2002 and data from analysis of these reports with NLP-Leximer were saved in a data warehouse and exported to a multidimensional structure called the Radcube. Various relational queries on the data in the Radcube were performed using OLAP technique. Thus, NLP-OLAP was applied to determine trends of F (POS) in different radiology exams for different patient and examination attributes. Pivot tables were exported from NLP-OLAP interface to Microsoft Excel for statistical analysis. Radcube allowed rapid and comprehensive analysis of F (POS) and F (NEG) trends in a large radiology report database. Trends of F (POS) were extracted for different patient attributes such as age groups, gender, clinical indications, diseases with ICD codes, patient types (inpatient, ambulatory), imaging characteristics such as imaging modalities, referring physicians, radiology subspecialties, and body regions. Data analysis showed substantial differences between F (POS) rates for different imaging modalities ranging from 23.1% (mammography, 49,163/212,906) to 85.8% (nuclear medicine, 93,852/109,374; p < 0.0001). In conclusion, NLP-OLAP can help in analysis of yield of different radiology exams from a large radiology report database.

Extraction of recommendation features in radiology with natural language processing: exploratory study.

OBJECTIVE: The purposes of this study were to validate a natural language processing program for extraction of recommendation features, such as recommended time frames and imaging technique, from electronic radiology reports and to assess patterns of recommendation features in a large database of radiology reports. MATERIALS AND METHODS: This study was performed on a radiology reports database covering the years 1995-2004. From this database, 120 reports with and without recommendations were selected and randomized. Two radiologists independently classified these reports according to presence of recommendations, time frame, and imaging technique suggested for follow-up or repeated examinations. The natural language processing program then was used to classify the reports according to the same criteria used by the radiologists. The accuracy of classification of recommendation features was determined. The program then was used to determine the patterns of recommendation features for different patients and imaging features in the entire database of 4,211,503 reports. RESULTS: The natural language processing program had an accuracy of 93.2% (82/88) for identifying the imaging technique recommended by the radiologists for further evaluation. Categorization of recommended time frames in the reports with the 88 recommendations obtained with the program resulted in 83 (94.3%) accurate classifications and five (5.7%) inaccurate classifications. Recommendations of CT were most common (27.9%, 105,076 of 376,918 reports) followed by those for MRI (17.8%). In most (85.4%, 322,074/376,918) of the reports with imaging recommendations, however, radiologists did not specify the time frame. CONCLUSION: Accurate determination of recommended imaging techniques and time frames in a large database of radiology reports is possible with a natural language processing program. Most imaging recommendations are for high-cost but more accurate radiologic studies.

Natural language processing using online analytic processing for assessing recommendations in radiology reports.

PURPOSE: The study purpose was to describe the use of natural language processing (NLP) and online analytic processing (OLAP) for assessing patterns in recommendations in unstructured radiology reports on the basis of patient and imaging characteristics, such as age, gender, referring physicians, radiology subspecialty, modality, indications, diseases, and patient status (inpatient vs outpatient). MATERIALS AND METHODS: A database of 4,279,179 radiology reports from a single tertiary health care center during a 10-year period (1995-2004) was created. The database includes reports of computed tomography, magnetic resonance imaging, fluoroscopy, nuclear medicine, ultrasound, radiography, mammography, angiography, special procedures, and unclassified imaging tests with patient demographics. A clinical data mining and analysis NLP program (Leximer, Nuance Inc, Burlington, Massachusetts) in conjunction with OLAP was used for classifying reports into those with recommendations (I(REC)) and without recommendations (N(REC)) for imaging and determining I(REC) rates for different patient age groups, gender, imaging modalities, indications, diseases, subspecialties, and referring physicians. In addition, temporal trends for I(REC) were also determined. RESULTS: There was a significant difference in the I(REC) rates in different age groups, varying between 4.8% (10-19 years) and 9.5% (>70 years) (P <.0001). Significant variations in I(REC) rates were observed for different imaging modalities, with the highest rates for computed tomography (17.3%, 100,493/581,032). The I(REC) rates varied significantly for different subspecialties and among radiologists within a subspecialty (P < .0001). For most modalities, outpatients had a higher rate of recommendations when compared with inpatients. CONCLUSION: The radiology reports database analyzed with NLP in conjunction with OLAP revealed considerable differences between recommendation trends for different imaging modalities and other patient and imaging characteristics.