Prospective Validation of a Prediction Model for the Diagnosis of Acute Pancreatitis.

Importance: While most patients with acute pancreatitis (AP) fulfill diagnostic criteria with characteristic abdominal pain and serum lipase levels of at least 3 times the upper limit of normal (reference range) at presentation, early imaging is often used for confirmation. A prior prediction model and corresponding point-based score were developed using nonimaging parameters to diagnose AP in patients presenting to the emergency department (ED). Objective: To evaluate the performance of the prediction model to diagnose AP in a prospective patient cohort. Design, Setting, and Participants: This prospective diagnostic study included consecutive adult patients presenting to the ED between January 1, 2020, and March 9, 2021, at 2 large academic medical centers in the northeastern US with serum lipase levels at least 3 times the upper limit of normal. Patients transferred from outside institutions or with malignant disease and established intra-abdominal metastases, acute trauma, or altered mentation were excluded. Data were analyzed from October 15 to October 23, 2023. Exposures: Participants were assigned scores for initial serum lipase level, number of prior AP episodes, prior cholelithiasis, abdominal surgery within 2 months, presence of epigastric pain, pain of worsening severity, duration from pain onset to presentation, and pain level at ED presentation. Main Outcome and Measures: A final diagnosis of AP, established by expert review of hospitalization records. Results: Prospective scores in 349 participants (mean [SD] age, 53.0 [18.8] years; 184 women [52.7%]; 66 Black [18.9%]; 199 White [57.0%]) demonstrated an area under the receiver operating characteristics curve of 0.91. A score of at least 6 points achieved highest accuracy (F score, 82.0), corresponding to a sensitivity of 81.5%, specificity of 85.9%, positive predictive value of 82.6%, and negative predictive value of 85.1% for AP diagnosis. Early computed tomography or magnetic resonance imaging was performed more often in participants predicted to have AP (116 of 155 [74.8%] with a score ≥6 vs 111 of 194 [57.2%] with a score <6; P < .001). Early imaging revealed an alternative diagnosis in 8 of 116 participants (6.9%) with scores of at least 6 points, 1 of 93 (1.1%) with scores of at least 7 points, and 1 of 73 (1.4%) with scores of at least 8 points. Conclusions and Relevance: In this multicenter diagnostic study, the prediction model demonstrated excellent AP diagnostic accuracy. Its application may be used to avoid unnecessary confirmatory imaging.

A Model for Predicting Clinically Significant Prostate Cancer Using Prostate MRI and Risk Factors.

PURPOSE: The aim of this study was to develop and validate a predictive model for clinically significant prostate cancer (csPCa) using prostate MRI and patient risk factors. METHODS: In total, 960 men who underwent MRI from 2015 to 2019 and biopsy either 6 months before or 6 months after MRI were identified. Men diagnosed with csPCa were identified, and csPCa risk was modeled using known patient factors (age, race, and prostate-specific antigen [PSA] level) and prostate MRI findings (location, Prostate Imaging Reporting and Data System score, extraprostatic extension, dominant lesion size, and PSA density). csPCa was defined as Gleason score sum ≥ 7. Using a derivation cohort, a multivariable logistic regression model and a point-based scoring system were developed to predict csPCa. Discrimination and calibration were assessed in a separate independent validation cohort. RESULTS: Among 960 MRI reports, 552 (57.5%) were from men diagnosed with csPCa. Using the derivation cohort (n = 632), variables that predicted csPCa were Prostate Imaging Reporting and Data System scores of 4 and 5, the presence of extraprostatic extension, and elevated PSA density. Evaluation using the validation cohort (n = 328) resulted in an area under the curve of 0.77, with adequate calibration (Hosmer-Lemeshow P = .58). At a risk threshold of >2 points, the model identified csPCa with sensitivity of 98.4% and negative predictive value of 78.6% but prevented only 4.3% potential biopsies (0-2 points; 14 of 328). At a higher threshold of >5 points, the model identified csPCa with sensitivity of 89.5% and negative predictive value of 70.1% and avoided 20.4% of biopsies (0-5 points; 67 of 328). CONCLUSIONS: The point-based model reported here can potentially identify a vast majority of men at risk for csPCa, while avoiding biopsy in about 1 in 5 men with elevated PSA levels.

The Impact of Mammographic, Radiologist, and Patient Factors on the Likelihood of Probably Benign (BI-RADS 3) Assessment at Diagnostic Mammography.

OBJECTIVE: To evaluate the association of mammographic, radiologist, and patient factors on BI-RADS 3 assessment at diagnostic mammography in patients recalled from screening mammography. METHODS: This Institutional Review Board-approved retrospective study of consecutive unique diagnostic mammography examinations in asymptomatic patients recalled from screening mammography March 5, 2014, to December 31, 2019, was conducted in a single large United States health care institution. Mammographic features (mass, calcification, distortion, asymmetry), breast density, prior examination, and BI-RADS assessment were extracted from reports by natural language processing. Patient age, race, and ethnicity were extracted from the electronic health record. Radiologist years in practice, recall rate, and number of interpreted diagnostic mammograms were calculated. A mixed effect logistic regression model evaluated factors associated with likelihood of BI-RADS 3 compared with other BI-RADS assessments. RESULTS: A total of 12 080 diagnostic mammography examinations were performed during the study period, yielding 2010 (16.6%) BI-RADS 3 and 10 070 (83.4%) other BI-RADS assessments. Asymmetry (odds ratio [OR] = 6.49, P <.001) and calcification (OR = 5.59, P <.001) were associated with increased likelihood of BI-RADS 3 assessment; distortion (OR = 0.20, P <.001), dense breast parenchyma (OR = 0.82, P <.001), prior examination (OR = 0.63, P = .01), and increasing patient age (OR = 0.99, P <.001) were associated with decreased likelihood. Mass, patient race or ethnicity, and radiologist factors were not significantly associated with BI-RADS 3 assessment. Malignancy rate for BI-RADS 3 lesions was 1.6%. CONCLUSION: Asymmetry and calcifications had an increased likelihood of BI-RADS 3 assessment at diagnostic evaluation with low likelihood of malignancy, while radiologist features had no association.

Risk of Malignancy in Incidentally Detected Lung Nodules in Patients Aged Younger Than 35 Years.

BACKGROUND: The risk of malignancy in pulmonary nodules incidentally detected on computed tomography (CT) in patients who are aged younger than 35 years is unclear. OBJECTIVE: The aim of this study was to evaluate the incidence of lung cancer in incidental pulmonary nodules in patients who are 15-34 years old. METHODS: This retrospective study included patients aged 15-34 years who had an incidental pulmonary nodule on chest CT from 2010 to 2018 at our hospital. Patients with prior, current, or suspected malignancy were excluded. A chart review identified patients with diagnosis of malignancy. Incidental pulmonary nodule was deemed benign if stable or resolved on a follow-up CT at least 2 years after initial or if there was a medical visit in our health care network at least 2 years after initial CT without diagnosis of malignancy.Receiver operating characteristic curve analysis was performed with nodule size. Association of categorical variables with lung cancer diagnosis was performed with Fisher exact test, and association of continuous variables was performed with logistic regression. RESULTS: Five thousand three hundred fifty-five chest CTs performed on patients aged 15-34 years between January 2010 and December 2018. After excluding patients without a reported pulmonary nodule and prior or current malignancy, there were a total of 779 patients. Of these, 690 (89%) had clinical or imaging follow-up after initial imaging. Of these, 545 (70% of total patients) patients had imaging or clinical follow-up greater than 2 years after their initial imaging.A malignant diagnosis was established in 2/779 patients (0.3%; 95% confidence interval, 0.1%-0.9%). Nodule size was strongly associated with malignancy (P = 0.007), with area under the receiver operating characteristic curve of 0.97. There were no malignant nodules that were less than 10 mm in size. Smoking history, number of nodules, and nodule density were not associated with malignancy. CONCLUSIONS: Risk of malignancy for incidentally detected pulmonary nodules in patients aged 15-34 years is extremely small (0.3%). There were no malignant nodules that were less than 10 mm in size. Routine follow-up of subcentimeter pulmonary nodules should be carefully weighed against the risks.

Magnetic Resonance Imaging, Clinical, and Biopsy Findings in Suspected Prostate Cancer: A Systematic Review and Meta-Analysis.

Importance: Multiple strategies integrating magnetic resonance imaging (MRI) and clinical data have been proposed to determine the need for a prostate biopsy in men with suspected clinically significant prostate cancer (csPCa) (Gleason score ≥3 + 4). However, inconsistencies across different strategies create challenges for drawing a definitive conclusion. Objective: To determine the optimal prostate biopsy decision-making strategy for avoiding unnecessary biopsies and minimizing the risk of missing csPCa by combining MRI Prostate Imaging Reporting & Data System (PI-RADS) and clinical data. Data Sources: PubMed, Ovid MEDLINE, Embase, Web of Science, and Cochrane Library from inception to July 1, 2022. Study Selection: English-language studies that evaluated men with suspected but not confirmed csPCa who underwent MRI PI-RADS followed by prostate biopsy were included. Each study had proposed a biopsy plan by combining PI-RADS and clinical data. Data Extraction and Synthesis: Studies were independently assessed for eligibility for inclusion. Quality of studies was appraised using the Quality Assessment of Diagnostic Accuracy Studies 2 tool and the Newcastle-Ottawa Scale. Mixed-effects meta-analyses and meta-regression models with multimodel inference were performed. Reporting of this study followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guideline. Main Outcomes and Measures: Independent risk factors of csPCa were determined by performing meta-regression between the rate of csPCa and PI-RADS and clinical parameters. Yields of different biopsy strategies were assessed by performing diagnostic meta-analysis. Results: The analyses included 72 studies comprising 36 366 patients. Univariable meta-regression showed that PI-RADS 4 (ß-coefficient [SE], 7.82 [3.85]; P = .045) and PI-RADS 5 (ß-coefficient [SE], 23.18 [4.46]; P < .001) lesions, but not PI-RADS 3 lesions (ß-coefficient [SE], -4.08 [3.06]; P = .19), were significantly associated with a higher risk of csPCa. When considered jointly in a multivariable model, prostate-specific antigen density (PSAD) was the only clinical variable significantly associated with csPCa (ß-coefficient [SE], 15.50 [5.14]; P < .001) besides PI-RADS 5 (ß-coefficient [SE], 9.19 [3.33]; P < .001). Avoiding biopsy in patients with lesions with PI-RADS category of 3 or less and PSAD less than 0.10 (vs <0.15) ng/mL2 resulted in reducing 30% (vs 48%) of unnecessary biopsies (compared with performing biopsy in all suspected patients), with an estimated sensitivity of 97% (vs 95%) and number needed to harm of 17 (vs 15). Conclusions and Relevance: These findings suggest that in patients with suspected csPCa, patient-tailored prostate biopsy decisions based on PI-RADS and PSAD could prevent unnecessary procedures while maintaining high sensitivity.

Recommendations for Additional Imaging on Head and Neck Imaging Examinations: Interradiologist Variation and Associated Factors.

BACKGROUND. A paucity of relevant guidelines may lead to pronounced variation among radiologists in issuing recommendations for additional imaging (RAI) for head and neck imaging. OBJECTIVE. The purpose of this article was to explore associations of RAI for head and neck imaging examinations with examination, patient, and radiologist factors and to assess the role of individual radiologist-specific behavior in issuing such RAI. METHODS. This retrospective study included 39,200 patients (median age, 58 years; 21,855 women, 17,315 men, 30 with missing sex information) who underwent 39,200 head and neck CT or MRI examinations, interpreted by 61 radiologists, from June 1, 2021, through May 31, 2022. A natural language processing (NLP) tool with manual review of NLP results was used to identify RAI in report impressions. Interradiologist variation in RAI rates was assessed. A generalized mixed-effects model was used to assess associations between RAI and examination, patient, and radiologist factors. RESULTS. A total of 2943 (7.5%) reports contained RAI. Individual radiologist RAI rates ranged from 0.8% to 22.0% (median, 7.1%; IQR, 5.2-10.2%), representing a 27.5-fold difference between minimum and a maximum values and 1.8-fold difference between 25th and 75th percentiles. In multivariable analysis, RAI likelihood was higher for CTA than for CT examinations (OR, 1.32), for examinations that included a trainee in report generation (OR, 1.23), and for patients with self-identified race of Black or African American versus White (OR, 1.25); was lower for male than female patients (OR, 0.90); and was associated with increasing patient age (OR, 1.09 per decade) and inversely associated with radiologist years since training (OR, 0.90 per 5 years). The model accounted for 10.9% of the likelihood of RAI. Of explainable likelihood of RAI, 25.7% was attributable to examination, patient, and radiologist factors; 74.3% was attributable to radiologist-specific behavior. CONCLUSION. Interradiologist variation in RAI rates for head and neck imaging was substantial. RAI appear to be more substantially associated with individual radiologist-specific behavior than with measurable systemic factors. CLINICAL IMPACT. Quality improvement initiatives, incorporating best practices for incidental findings management, may help reduce radiologist preference-sensitive decision-making in issuing RAI for head and neck imaging and associated care variation.

Factors Associated With Timeliness and Equity of Access to Outpatient MRI Examinations.

OBJECTIVE: The role of MRI in guiding patients' diagnosis and treatment is increasing. Therefore, timely MRI performance prevents delays that can impact patient care. We assessed the timeliness of performing outpatient MRIs using the socio-ecological model approach and evaluated multilevel factors associated with delays. METHODS: This institutional review board-approved study included outpatient MRI examinations ordered between October 1, 2021, and December 31, 2022, for performance at a large quaternary care health system. Mean order-to-performed (OtoP) interval (in days) and prolonged OtoP interval (defined as >10 days) for MRI orders with an expected date of 1 day to examination performance were measured. Logistic regression was used to assess patient-level (demographic and social determinants of health), radiology practice-level, and community-level factors associated with prolonged OtoP interval. RESULTS: There were 126,079 MRI examination orders with expected performance within 1 day placed during the study period (56% of all MRI orders placed). After excluding duplicates, there were 97,160 orders for unique patients. Of the MRI orders, 48% had a prolonged OtoP interval, and mean OtoP interval was 18.5 days. Factors significantly associated with delay in MRI performance included public insurance (odds ratio [OR] = 1.11, P < .001), female gender (OR = 1.11, P < .001), radiology subspecialty (ie, cardiac, OR = 1.71, P < .001), and patients from areas that are most deprived (ie, highest Area Deprivation Index quintile, OR = 1.70, P < .001). DISCUSSION: Nearly half of outpatient MRI orders were delayed, performed >10 days from the expected date selected by the ordering provider. Addressing multilevel factors associated with such delays may help enhance timeliness and equity of access to MRI examinations, potentially reducing diagnostic errors and treatment delays.

Actionability of Recommendations for Additional Imaging in Head and Neck Radiology.

PURPOSE: The aims of this study were to measure the actionability of recommendations for additional imaging (RAIs) in head and neck CT and MRI, for which there is a near complete absence of best practices or guidelines; to identify the most common recommendations; and to assess radiologist factors associated with actionability. METHODS: All head and neck CT and MRI radiology reports across a multi-institution, multipractice health care system from June 1, 2021, to May 31, 2022, were retrospectively reviewed. The actionability of RAIs was scored using a validated taxonomy. The most common RAIs were identified. Actionability association with radiologist factors (gender, years out of training, fellowship training, practice type) and with trainees was measured using a mixed-effects model. RESULTS: Two hundred nine radiologists generated 60,543 reports, of which 7.2% (n = 4,382) contained RAIs. Only 3.9% of RAIs (170 of 4,382) were actionable. More than 60% of RAIs were for eight examinations: thyroid ultrasound (14.1%), neck CT (12.6%), brain MRI (6.9%), chest CT (6.5%), neck CT angiography (5.5%), temporal bone CT (5.3%), temporal bone MRI (5.2%), and pituitary MRI (4.6%). Radiologists >23 years out of training (odds ratio, 0.39; 95% confidence interval, 0.15-1.02; P = .05) and community radiologists (odds ratio, 0.53; 95% confidence interval, 0.22-1.31; P = .17) had substantially lower estimated odds of making actionable RAIs than radiologists <7 years out of training and academic radiologists, respectively. CONCLUSIONS: The studied radiologists rarely made actionable RAIs, which makes it difficult to identify and track clinically necessary RAIs to timely performance. Multifaceted quality improvement initiatives including peer comparisons, clinical decision support at the time of reporting, and the development of evidence-based best practices, may help improve tracking and timely performance of clinically necessary RAIs.

Imaging characteristics of interval cancers detected on Full Field Digital Mammography (FFDM) versus Digital Breast Tomosynthesis (DBT).

OBJECTIVE: To compare imaging features of interval cancers detected in patients screened with full field digital mammography (FFDM) versus digital breast tomosynthesis (DBT). MATERIALS/METHODS: This retrospective observational study consisted of female patients undergoing screening DM or FFDM at an academic medical center and two outpatient imaging facilities between January 2012 and June 2017. A natural language processing algorithm queried breast imaging reports for breast density and BI-RADS category. This was cross-referenced to an institutional breast cancer registry to identify interval cancers. Retrospective consensus review of the cases was done to categorize imaging features of interval cancers on FFDM vs DBT. RESULTS: The rate of interval cancers was comparable in patients screened with FFDM (30/39793) and DBT (29/32180) (p = 0.58). There was no significant difference in the rate, histopathology, or imaging features of interval cancers in patients screened with FFDM versus DBT. The most common mammographic features on diagnostic imaging across both groups was the presence of a mass (13/47). Almost equally common was negative diagnostic mammogram with mass detected only on ultrasound (11/47). The rate of interval cancers detected by high-risk surveillance breast MRI was increased in patients who previously had screening with DBT relative to those who had screening with FFDM (p = 0.0419). CONCLUSION: There is no significant difference in rate of detection, histopathology, or imaging features of interval cancers in patients screened with FFDM versus DBT. However, across both cohorts, the most common features on diagnostic mammogram were either the presence of a mass or a negative mammogram.

MRI-Targeted, Systematic, or Combined Biopsy for Detecting Clinically Significant Prostate Cancer.

PURPOSE: The aim of this study was to assess MRI-targeted, systematic, or combined prostate biopsy for diagnosing prostate cancer to identify opportunities for diagnostic accuracy improvement. METHODS: This institutional review board-approved, retrospective study, performed at a large, quaternary hospital, included all men undergoing prostate multiparametric MRI (mpMRI) from January 1, 2015, to December 31, 2019, with prostate-specific antigen ≥ 4 ng/mL, biopsy target on mpMRI (Prostate Imaging Reporting and Data System [PI-RADS] 3-5 lesion), and combined targeted and systematic biopsy ≤6 months after MRI. Analysis included the highest grade lesion per patient. The primary outcome was prostate cancer diagnosis by grade group (GG; 1, 2, and ≥3). Secondary outcomes were rates of cancer upgrading by biopsy type and cancer proximity to the targeted biopsy site in patients upgraded by systematic biopsy. RESULTS: Two hundred sixty-seven biopsies (267 patients) were included; 94.4% (252 of 267) were biopsy naive. The most suspicious mpMRI lesion was PI-RADS 3 in 18.7% (50 of 267), PI-RADS 4 in 52.4% (140 of 267), and PI-RADS 5 in 28.8% (77 of 267). Prostate cancer was diagnosed in 68.5% (183 of 267): 22.1% (59 of 267) GG 1, 16.1% (43 of 267) GG 2, and 30.3% (81 of 267) GG ≥ 3. Combined biopsy (124 of 267) yielded more GG ≥ 2 prostate cancer diagnoses than systematic (87 of 267) or targeted (110 of 267) biopsy alone. More GG ≥ 2 cancers were upgraded by targeted biopsy than by systematic biopsy (P = .0062). Systematic biopsy upgrades were in close proximity to the targeted biopsy site in 42.1% (24 of 57); GG ≥ 3 cancers 62.5% (15 of 24) constituted most proximal misses. CONCLUSIONS: In men with prostate-specific antigen ≥ 4 ng/mL and PI-RADS 3, 4, or 5 lesion on mpMRI, combined biopsy led to more prostate cancer diagnoses than targeted or systematic biopsy alone. Cancers upgraded by systematic biopsy proximal and distant from the targeted biopsy site may indicate opportunities for biopsy and mpMRI improvement, respectively.

Impact of an Automated Closed-Loop Communication and Tracking Tool on the Rate of Recommendations for Additional Imaging in Thoracic Radiology Reports.

OBJECTIVE: Assess the effects of feedback reports and implementing a closed-loop communication system on rates of recommendations for additional imaging (RAIs) in thoracic radiology reports. METHODS: In this retrospective, institutional review board-approved study at an academic quaternary care hospital, we analyzed 176,498 thoracic radiology reports during a pre-intervention (baseline) period from April 1, 2018, to November 30, 2018; a feedback report only period from December 1, 2018, to September 30, 2019; and a closed-loop communication system plus feedback report (IT intervention) period from October 1, 2019, to December 31, 2020, promoting explicit documentation of rationale, time frame, and imaging modality for RAI, defined as complete RAI. A previously validated natural language processing tool was used to classify reports with an RAI. Primary outcome of rate of RAI was compared using a control chart. Multivariable logistic regression determined factors associated with likelihood of RAI. We also estimated the completeness of RAI in reports comparing IT intervention to baseline using χ2 statistic. RESULTS: The natural language processing tool classified 3.2% (5,682 of 176,498) reports as having an RAI; 3.5% (1,783 of 51,323) during the pre-intervention period, 3.8% (2,147 of 56,722) during the feedback report only period (odds ratio: 1.1, P = .03), and 2.6% (1,752 of 68,453) during the IT intervention period (odds ratio: 0.60, P < .001). In subanalysis, the proportion of incomplete RAI decreased from 84.0% (79 of 94) during the pre-intervention period to 48.5% (47 of 97) during the IT intervention period (P < .001). DISCUSSION: Feedback reports alone increased RAI rates, and an IT intervention promoting documentation of complete RAI in addition to feedback reports led to significant reductions in RAI rate, incomplete RAI, and improved overall completeness of the radiology recommendations.

Patient and System Factors Associated With Completed Follow-Up of Probably Benign (BI-RADS 3) Breast Imaging Findings.

OBJECTIVE: Evaluate patient factors and health system test ordering and scheduling processes associated with completed BI-RADS 3 breast imaging follow-up. METHODS: Retrospective review of reports from January 1, 2021, to July 31, 2021, identified BI-RADS 3 findings corresponding to unique patient encounters (index examinations). The electronic health record was queried for patient, examination, and health system ordering or scheduling data including follow-up order status (order placed, performed; order placed, scheduled, but not performed; order placed, unscheduled; no order placed); ordering provider specialty and health system affiliation (primary care versus other, internal versus external to health system); and ordering department (radiology staff versus referring physician staff). Patient home addresses were categorized by area deprivation index (University of Wisconsin's Neighborhood Atlas). Univariable and multivariable analysis identified patient, examination, and ordering or scheduling factors associated with completed follow-up imaging within 15 months of BI-RADS 3 assessment. RESULTS: There were 3,104 unique BI-RADS 3 assessments, 2,561 (82.5%) with completed BI-RADS 3 follow-up within 15 months of study examination. In multivariable analysis, factors associated with incomplete follow-up included ultrasound (odds ratio [OR] 0.48; 95% confidence interval [95% CI] 0.38-0.60; P < .001) and MRI (OR 0.71; 95% CI 0.50-1.00; P = .049) versus mammogram; patients living in the highest disadvantaged neighborhoods (OR 0.70; 95% CI 0.50-0.98; P = .04); patients <40 years (OR 0.14; 95% CI 0.11-0.19; P < .001); Asian race (OR 0.55; 95% CI 0.37-0.81; P = .003); order placement >3 months (OR, 0.05; 95% CI 0.02-0.16; P < .001) after index examination or scheduling >6 months after order placement (OR, 0.35; 95% CI 0.14-0.87; P = .02); order placement by breast oncology or breast surgery departments (OR 0.35; 95% CI 0.17-0.73; P = .01) versus radiology department. DISCUSSION: Incomplete BI-RADS 3 follow-up is associated with ultrasound or MRI, most socioeconomically disadvantaged patients, younger patients, Asian race, delayed order entry, and follow-up examination ordering and scheduling by non-radiology departments.

Frequency and Outcomes of BI-RADS Category 3 Assessments in Patients With a Personal History of Breast Cancer: Full-Field Digital Mammography Versus Digital Breast Tomosynthesis.

BACKGROUND. Studies establishing the validity of BI-RADS category 3 excluded patients with personal history of breast cancer (PHBC). Use of category 3 in patients with PHBC may be impacted not only by this population's increased breast cancer risk, but also by adoption of digital breast tomosynthesis (DBT) over full-field digital mammography (FFDM). OBJECTIVE. The purpose of this article was to compare the frequency, outcomes, and additional characteristics of BI-RADS category 3 assessments between FFDM and DBT in patients with PHBC. METHODS. This retrospective study included 14,845 mammograms in 10,118 patients (mean age, 63 years) with PHBC who had undergone mastectomy and/or lumpectomy. Of these, 8422 examinations were performed by FFDM from October 2014 to September 2016, and 6423 examinations by FFDM with DBT from February 2017 to December 2018, after interval conversion of the center's mammography units. Information was extracted from the EHR and radiology reports. FFDM and DBT groups were compared in the entire sample and among index category 3 lesions (i.e., earliest category 3 assessment per lesion). RESULTS. The frequency of category 3 assessment was lower for DBT than FFDM (5.6% vs 6.4%; p = .05). DBT, compared with FFDM, showed a lower malignancy rate for category 3 lesions (1.8% vs 5.0%; p = .04), higher malignancy rate for category 4 lesions (32.0% vs 23.2%; p = .03), and no difference in malignancy rate for category 5 lesions (100.0% vs 75.0%; p = .24). Analysis of index category 3 lesions included 438 and 274 lesions for FFDM and DBT, respectively. For category 3 lesions, DBT, compared with FFDM, showed lower PPV3 (13.9% vs 36.1%; p = .02) and a more frequent mammographic finding of mass (33.2% vs 23.1%; p = .003). CONCLUSION. The malignancy rate for category 3 lesions in patients with PHBC was less than the accepted limit (2%) for DBT (1.8%), but not FFDM (5.0%). A lower malignancy rate for category 3 lesions but higher malignancy rate for category 4 lesions for DBT supports more appropriate application of category 3 assessment in patients with PHBC through use of DBT. CLINICAL IMPACT. These insights may help establish whether category 3 assessments in patients with PHBC are within benchmarks for early detection of second cancers and reduction of benign biopsies.

Development and External Validation of an Artificial Intelligence Model for Identifying Radiology Reports Containing Recommendations for Additional Imaging.

BACKGROUND. Reported rates of recommendations for additional imaging (RAIs) in radiology reports are low. Bidirectional encoder representations from transformers (BERT), a deep learning model pretrained to understand language context and ambiguity, has potential for identifying RAIs and thereby assisting large-scale quality improvement efforts. OBJECTIVE. The purpose of this study was to develop and externally validate an artificial intelligence (AI)-based model for identifying radiology reports containing RAIs. METHODS. This retrospective study was performed at a multisite health center. A total of 6300 radiology reports generated at one site from January 1, 2015, to June 30, 2021, were randomly selected and split by 4:1 ratio to create training (n = 5040) and test (n = 1260) sets. A total of 1260 reports generated at the center's other sites (including academic and community hospitals) from April 1 to April 30, 2022, were randomly selected as an external validation group. Referring practitioners and radiologists of varying sub-specialties manually reviewed report impressions for presence of RAIs. A BERT-based technique for identifying RAIs was developed by use of the training set. Performance of the BERT-based model and a previously developed traditional machine learning (TML) model was assessed in the test set. Finally, performance was assessed in the external validation set. The code for the BERT-based RAI model is publicly available. RESULTS. Among a total of 7419 unique patients (4133 women, 3286 men; mean age, 58.8 years), 10.0% of 7560 reports contained RAI. In the test set, the BERT-based model had 94.4% precision, 98.5% recall, and an F1 score of 96.4%. In the test set, the TML model had 69.0% precision, 65.4% recall, and an F1 score of 67.2%. In the test set, accuracy was greater for the BERT-based than for the TML model (99.2% vs 93.1%, p < .001). In the external validation set, the BERT-based model had 99.2% precision, 91.6% recall, an F1 score of 95.2%, and 99.0% accuracy. CONCLUSION. The BERT-based AI model accurately identified reports with RAIs, outperforming the TML model. High performance in the external validation set suggests the potential for other health systems to adapt the model without requiring institution-specific training. CLINICAL IMPACT. The model could potentially be used for real-time EHR monitoring for RAIs and other improvement initiatives to help ensure timely performance of clinically necessary recommended follow-up.

Development and Assessment of an Information Technology Intervention to Improve the Clarity of Radiologist Follow-up Recommendations.

Importance: It is challenging to ensure timely performance of radiologist-recommended additional imaging when radiologist recommendation language is incomplete or ambiguous. Objective: To evaluate whether voluntary use of an information technology tool with forced structured entry of recommendation attributes was associated with improved completeness of recommendations for additional imaging over time. Design, Setting, and Participants: This cohort study of imaging report data was performed at an academic quaternary care center in Boston, Massachusetts, and included consecutive adults with radiology examinations performed from September 12 to 13, 2019 (taxonomy validation), October 14 to 17, 2019 (before intervention), April 5 to 7, 2021 (1 week after intervention), and April 4 to 7, 2022 (1 year after intervention), with reports containing recommendations for additional imaging. A radiologist scored the 3 groups (preintervention group, 1-week postintervention group, and 1-year postintervention group) of 336 consecutive radiology reports (n = 1008) with recommendations for additional imaging. Intervention: Final implementation on March 27, 2021, of a voluntary closed-loop communication tool embedded in radiologist clinical workflow that required structured entry of recommendation attributes. Main Outcomes and Measures: The a priori primary outcome was completeness of recommendations for additional imaging, defined in a taxonomy created by a multidisciplinary expert panel. To validate the taxonomy, 2 radiologists independently reviewed and scored language attributes as present or absent in 247 consecutive radiology reports containing recommendations for additional imaging. Agreement was assessed with Cohen κ. Recommendation completeness over time was compared with with 1-sided Fisher exact tests and significance set at P < .05. Results: Radiology-related information for consecutive radiology reports from the 4 time periods was collected from the radiology department data warehouse, which does not include data on patient demographic characteristics or other nonimaging patient medical information. The panel defined 5 recommendation language attributes: complete (contains imaging modality, time frame, and rationale), ambiguous (equivocal, vague language), conditional (qualifying language), multiplicity (multiple options), and alternate (language favoring a different examination to that ordered). Two radiologists had more than 90% agreement (κ > 0.8) for these attributes. Completeness with use of the tool increased more than 3-fold, from 14% (46 of 336) before the intervention to 46% (153 of 336) (P < .001) 1 year after intervention; completeness in the corresponding free-text report language increased from 14% (46 of 336) before the intervention to 25% (85 of 336) (P < .001) 1 year after the intervention. Conclusions and Relevance: This study suggests that supplementing free-text dictation with voluntary use of a structured entry tool was associated with improved completeness of radiologist recommendations for additional imaging as assessed by an internally validated taxonomy. Future research is needed to assess the association with timely performance of clinically necessary recommendations and diagnostic errors. The taxonomy can be used to evaluate and build interventions to modify radiologist reporting behaviors.

Patterns of Screening Recall Behavior Among Subspecialty Breast Radiologists.

RATIONALE AND OBJECTIVES: Determine whether there are patterns of lesion recall among breast imaging subspecialists interpreting screening mammography, and if so, whether recall patterns correlate to morphologies of screen-detected cancers. MATERIALS AND METHODS: This Institutional Review Board-approved, retrospective review included all screening examinations January 3, 2012-October 1, 2018 interpreted by fifteen breast imaging subspecialists at a large academic medical center and two outpatient imaging centers. Natural language processing identified radiologist recalls by lesion type (mass, calcifications, asymmetry, architectural distortion); proportions of callbacks by lesion types were calculated per radiologist. Hierarchical cluster analysis grouped radiologists based on recall patterns. Groups were compared to overall practice and each other by proportions of lesion types recalled, and overall and lesion-specific positive predictive value-1 (PPV1). RESULTS: Among 161,859 screening mammograms with 13,086 (8.1%) recalls, Hierarchical cluster analysis grouped 15 radiologists into five groups. There was substantial variation in proportions of lesions recalled: calcifications 13%-18% (Chi-square 45.69, p < 0.00001); mass 16%-44% (Chi-square 498.42, p < 0.00001); asymmetry 13%-47% (Chi-square 660.93, p < 0.00001) architectural distortion 6%-20% (Chi-square 283.81, p < 0.00001). Radiologist groups differed significantly in overall PPV1 (range 5.6%-8.8%; Chi-square 17.065, p = 0.0019). PPV1 by lesion type varied among groups: calcifications 9.2%-15.4% (Chi-square 2.56, p = 0.6339); mass 5.6%-8.5% (Chi-square 1.31, p = 0.8597); asymmetry 3.4%-5.9% (Chi-square 2.225, p = 0.6945); architectural distortion 5.6%-10.8% (Chi-square 5.810, p = 0.2138). Proportions of recalled lesions did not consistently correlate to proportions of screen-detected cancer. CONCLUSION: Breast imaging subspecialists have patterns for screening mammography recalls, suggesting differential weighting of imaging findings for perceived malignant potential. Radiologist recall patterns are not always predictive of screen-detected cancers nor lesion-specific PPV1s.

Live Mammographic Screening Interpretation Versus Offline Same-Day Screening Interpretation at a Tertiary Cancer Center.

OBJECTIVES: The aim of this study was to compare screening mammography performance metrics for immediate (live) interpretation versus offline interpretation at a cancer center. METHODS: An institutional review board-approved, retrospective comparison of screening mammography metrics at a cancer center for January 1, 2018, to December 31, 2019 (live period), and September 1, 2020, to March 31, 2022 (offline period), was performed. Before July 2020, screening examinations were interpreted while patients waited (live period), and diagnostic workup was performed concurrently. After the coronavirus disease 2019 shutdown from March to mid-June 2020, offline same-day interpretation was instituted. Patients with abnormal screening results returned for separate diagnostic evaluation. Screening metrics of positive predictive value 1 (PPV1), cancer detection rate (CDR), and abnormal interpretation rate (AIR) were compared for 17 radiologists who interpreted during both periods. Statistical significance was assessed using χ2 analysis. RESULTS: In the live period, there were 7,105 screenings, 635 recalls, and 51 screen-detected cancers. In the offline period, there were 7,512 screenings, 586 recalls, and 47 screen-detected cancers. Comparison of live screening metrics versus offline metrics produced the following results: AIR, 8.9% (635 of 7,105) versus 7.8% (586 of 7,512) (P = .01); PPV1, 8.0% (51 of 635) versus 8.0% (47 of 586); and CDR, 7.2/1,000 versus 6.3/1,000 (P = .50). When grouped by >10% AIR or <10% AIR for the live period, the >10% AIR group showed a significant decrease in AIR for offline interpretation (from 12.7% to 9.7%, P < .001), whereas the <10% AIR group showed no significant change (from 7.4% to 6.7%, P = .17). CONCLUSIONS: Conversion to offline screening interpretation from immediate interpretation at a cancer center was associated with lower AIR and similar CDR and PPV1. This effect was seen largely in radiologists with AIR > 10% in the live setting.

Predictors of Completion of Clinically Necessary Radiologist-Recommended Follow-Up Imaging: Assessment Using an Automated Closed-Loop Communication and Tracking Tool.

BACKGROUND. Patients with adverse social determinants of health may be at increased risk of not completing clinically necessary follow-up imaging. OBJECTIVE. The purpose of this study was to use an automated closed-loop communication and tracking tool to identify patient-, referrer-, and imaging-related factors associated with lack of completion of radiologist-recommended follow-up imaging. METHODS. This retrospective study was performed at a single academic health system. A tool for automated communication and tracking of radiologist-recommended follow-up imaging was embedded in the PACS and electronic health record. The tool prompted referrers to record whether they deemed recommendations to be clinically necessary and assessed whether clinically necessary follow-up imaging was pursued. If imaging was not performed within 1 month after the intended completion date, the tool prompted a safety net team to conduct further patient and referrer follow-up. The study included patients for whom a follow-up imaging recommendation deemed clinically necessary by the referrer was entered with the tool from October 21, 2019, through June 30, 2021. The electronic health record was reviewed for documentation of eventual completion of the recommended imaging at the study institution or an outside institution. Multivariable logistic regression analysis was performed to identify factors associated with completion of follow-up imaging. RESULTS. Of 5856 recommendations entered during the study period, the referrer agreed with 4881 recommendations in 4599 patients (2929 women, 1670 men; mean age, 61.3 ± 15.6 years), who formed the study sample. Follow-up was completed for 74.8% (3651/4881) of recommendations. Independent predictors of lower likelihood of completing follow-up imaging included living in a socioeconomically disadvantaged neighborhood according to the area deprivation index (odds ratio [OR], 0.67 [95% CI, 0.54-0.84]), inpatient (OR, 0.25 [95% CI, 0.20-0.32]) or emergency department (OR, 0.09 [95% CI, 0.05-0.15]) care setting, and referrer surgical specialty (OR, 0.70 [95% CI, 0.58-0.84]). Patient age, race and ethnicity, primary language, and insurance status were not independent predictors of completing follow-up (p > .05). CONCLUSION. Socioeconomically disadvantaged patients are at increased risk of not completing recommended follow-up imaging that referrers deem clinically necessary. CLINICAL IMPACT. Initiatives for ensuring completion of follow-up imaging should be aimed at the identified patient groups to reduce disparities in missed and delayed diagnoses.

Radiologists' Contribution to Variation in Detecting Clinically Significant Prostate Cancer in Men With Prostate MRI.

OBJECTIVE: Assess radiologists' contribution to variation in clinically significant prostate cancer (csPCa) detection in patients with elevated prostate-specific antigen (PSA) and multiparametric MRI (mpMRI). METHODS: This institutional review board-approved, retrospective cohort study was performed at a tertiary, academic, National Cancer Institute-designated Comprehensive Cancer Center with a multidisciplinary prostate cancer program. Men undergoing mpMRI examinations from January 1, 2015, to December 31, 2019, with elevated PSA (≥4 ng/mL) and biopsy within 6 months pre- or post-MRI or prostatectomy within 6 months post-mpMRI were included. Univariate and multivariable hierarchical logistic regression assessed impact of patient, provider, mpMRI examination, mpMRI report, and pathology factors on the diagnosis of Grade Group ≥ 2 csPCa. RESULTS: Study cohort included 960 MRIs in 928 men, mean age 64.0 years (SD ± 7.4), and 59.8% (555 of 928) had csPCa. Interpreting radiologist was not significant individually (P > .999) or combined with mpMRI ordering physician and physician performing biopsy or prostatectomy (P = .41). Prostate Imaging Reporting and Data System (PI-RADS) category 2 (odds ratio [OR] 0.18, P = .04), PI-RADS category 4 (OR 2.52, P < .001), and PI-RADS category 5 (OR 4.99, P < .001) assessment compared with no focal lesion; PSA density of 0.1 to 0.15 ng/mL/cc (OR 2.46, P < .001), 0.15 to 0.2 ng/mL/cc (OR 2.77, P < .001), or ≥0.2 ng/mL/cc (OR 4.52, P < .001); private insurance (reference = Medicare, OR 0.52, P = .001), and unambiguous extraprostatic extension on mpMRI (OR 2.94, P = .01) were independently associated with csPCa. PI-RADS 3 assessment (OR 1.18, P = .56), age (OR 0.99, P = .39), and African American race (OR 0.90, P = .75) were not. DISCUSSION: Although there is known in-practice variation in radiologists' interpretation of mpMRI, in our multidisciplinary prostate cancer program we found no significant radiologist-attributable variation in csPCa detection.