Variation in Follow-up Imaging Recommendations in Radiology Reports: Patient, Modality, and Radiologist Predictors.

Background Variation between radiologists when making recommendations for additional imaging and associated factors are, to the knowledge of the authors, unknown. Clear identification of factors that account for variation in follow-up recommendations might prevent unnecessary tests for incidental or ambiguous image findings. Purpose To determine incidence and identify factors associated with follow-up recommendations in radiology reports from multiple modalities, patient care settings, and imaging divisions. Materials and Methods This retrospective study analyzed 318 366 reports obtained from diagnostic imaging examinations performed at a large urban quaternary care hospital from January 1 to December 31, 2016, excluding breast and US reports. A subset of 1000 reports were randomly selected and manually annotated to train and validate a machine learning algorithm to predict whether a report included a follow-up imaging recommendation (training-and-validation set consisted of 850 reports and test set of 150 reports). The trained algorithm was used to classify 318 366 reports. Multivariable logistic regression was used to determine the likelihood of follow-up recommendation. Additional analysis by imaging subspecialty division was performed, and intradivision and interradiologist variability was quantified. Results The machine learning algorithm classified 38 745 of 318 366 (12.2%) reports as containing follow-up recommendations. Average patient age was 59 years ± 17 (standard deviation); 45.2% (143 767 of 318 366) of reports were from male patients. Among 65 radiologists, 57% (37 of 65) were men. At multivariable analysis, older patients had higher rates of follow-up recommendations (odds ratio [OR], 1.01 [95% confidence interval {CI}: 1.01, 1.01] for each additional year), male patients had lower rates of follow-up recommendations (OR, 0.9; 95% CI: 0.9, 1.0), and follow-up recommendations were most common among CT studies (OR, 4.2 [95% CI: 4.0, 4.4] compared with radiography). Radiologist sex (P = .54), presence of a trainee (P = .45), and years in practice (P = .49) were not significant predictors overall. A division-level analysis showed 2.8-fold to 6.7-fold interradiologist variation. Conclusion Substantial interradiologist variation exists in the probability of recommending a follow-up examination in a radiology report, after adjusting for patient, examination, and radiologist factors. © RSNA, 2019 See also the editorial by Russell in this issue.

Use of Machine Learning to Identify Follow-Up Recommendations in Radiology Reports.

PURPOSE: The aims of this study were to assess follow-up recommendations in radiology reports, develop and assess traditional machine learning (TML) and deep learning (DL) models in identifying follow-up, and benchmark them against a natural language processing (NLP) system. METHODS: This HIPAA-compliant, institutional review board-approved study was performed at an academic medical center generating >500,000 radiology reports annually. One thousand randomly selected ultrasound, radiography, CT, and MRI reports generated in 2016 were manually reviewed and annotated for follow-up recommendations. TML (support vector machines, random forest, logistic regression) and DL (recurrent neural nets) algorithms were constructed and trained on 850 reports (training data), with subsequent optimization of model architectures and parameters. Precision, recall, and F1 score were calculated on the remaining 150 reports (test data). A previously developed and validated NLP system (iSCOUT) was also applied to the test data, with equivalent metrics calculated. RESULTS: Follow-up recommendations were present in 12.7% of reports. The TML algorithms achieved F1 scores of 0.75 (random forest), 0.83 (logistic regression), and 0.85 (support vector machine) on the test data. DL recurrent neural nets had an F1 score of 0.71; iSCOUT also had an F1 score of 0.71. Performance of both TML and DL methods by F1 scores appeared to plateau after 500 to 700 samples while training. CONCLUSIONS: TML and DL are feasible methods to identify follow-up recommendations. These methods have great potential for near real-time monitoring of follow-up recommendations in radiology reports.

Impact of Delayed Time to Advanced Imaging on Missed Appointments Across Different Demographic and Socioeconomic Factors.

PURPOSE: The aim of this study was to investigate the impact of wait days (WDs) on missed outpatient MRI appointments across different demographic and socioeconomic factors. METHODS: An institutional review board-approved retrospective study was conducted among adult patients scheduled for outpatient MRI during a 12-month period. Scheduling data and demographic information were obtained. Imaging missed appointments were defined as missed scheduled imaging encounters. WDs were defined as the number of days from study order to appointment. Multivariate logistic regression was applied to assess the contribution of race and socioeconomic factors to missed appointments. Linear regression was performed to assess the relationship between missed appointment rates and WDs stratified by race, income, and patient insurance groups with analysis of covariance statistics. RESULTS: A total of 42,727 patients met the inclusion criteria. Mean WDs were 7.95 days. Multivariate regression showed increased odds ratio for missed appointments for patients with increased WDs (7-21 days: odds ratio [OR], 1.39; >21 days: OR, 1.77), African American patients (OR, 1.71), Hispanic patients (OR, 1.30), patients with noncommercial insurance (OR, 2.00-2.55), and those with imaging performed at the main hospital campus (OR, 1.51). Missed appointment rate linearly increased with WDs, with analysis of covariance revealing underrepresented minorities and Medicaid insurance as significant effect modifiers. CONCLUSIONS: Increased WDs for advanced imaging significantly increases the likelihood of missed appointments. This effect is most pronounced among underrepresented minorities and patients with lower socioeconomic status. Efforts to reduce WDs may improve equity in access to and utilization of advanced diagnostic imaging for all patients.

Mathematical framework for activity-based cancer biomarkers.

Advances in nanomedicine are providing sophisticated functions to precisely control the behavior of nanoscale drugs and diagnostics. Strategies that coopt protease activity as molecular triggers are increasingly important in nanoparticle design, yet the pharmacokinetics of these systems are challenging to understand without a quantitative framework to reveal nonintuitive associations. We describe a multicompartment mathematical model to predict strategies for ultrasensitive detection of cancer using synthetic biomarkers, a class of activity-based probes that amplify cancer-derived signals into urine as a noninvasive diagnostic. Using a model formulation made of a PEG core conjugated with protease-cleavable peptides, we explore a vast design space and identify guidelines for increasing sensitivity that depend on critical parameters such as enzyme kinetics, dosage, and probe stability. According to this model, synthetic biomarkers that circulate in stealth but then activate at sites of disease have the theoretical capacity to discriminate tumors as small as 5 mm in diameter-a threshold sensitivity that is otherwise challenging for medical imaging and blood biomarkers to achieve. This model may be adapted to describe the behavior of additional activity-based approaches to allow cross-platform comparisons, and to predict allometric scaling across species.