Theory of radiologist interaction with instant messaging decision support tools: A sequential-explanatory study.

Radiology specific clinical decision support systems (CDSS) and artificial intelligence are poorly integrated into the radiologist workflow. Current research and development efforts of radiology CDSS focus on 4 main interventions, based around exam centric time points-after image acquisition, intra-report support, post-report analysis, and radiology workflow adjacent. We review the literature surrounding CDSS tools in these time points, requirements for CDSS workflow augmentation, and technologies that support clinician to computer workflow augmentation. We develop a theory of radiologist-decision tool interaction using a sequential explanatory study design. The study consists of 2 phases, the first a quantitative survey and the second a qualitative interview study. The phase 1 survey identifies differences between average users and radiologist users in software interventions using the User Acceptance of Information Technology: Toward a Unified View (UTAUT) framework. Phase 2 semi-structured interviews provide narratives on why these differences are found. To build this theory, we propose a novel solution called Radibot-a conversational agent capable of engaging clinicians with CDSS as an assistant using existing instant messaging systems supporting hospital communications. This work contributes an understanding of how radiologist-users differ from the average user and can be utilized by software developers to increase satisfaction of CDSS tools within radiology.

Improving the Safety of Computed Tomography Through Automated Quality Measurement: A Radiologist Reader Study of Radiation Dose, Image Noise, and Image Quality.

OBJECTIVES: The Centers for Medicare and Medicaid Services funded the development of a computed tomography (CT) quality measure for use in pay-for-performance programs, which balances automated assessments of radiation dose with image quality to incentivize dose reduction without compromising the diagnostic utility of the tests. However, no existing quantitative method for assessing CT image quality has been validated against radiologists' image quality assessments on a large number of CT examinations. Thus to develop an automated measure of image quality, we tested the relationship between radiologists' subjective ratings of image quality with measurements of radiation dose and image noise. MATERIALS AND METHODS: Board-certified, posttraining, clinically active radiologists rated the image quality of 200 diagnostic CT examinations from a set of 734, representing 14 CT categories. Examinations with significant distractions, motion, or artifact were excluded. Radiologists rated diagnostic image quality as excellent, adequate, marginally acceptable, or poor; the latter 2 were considered unacceptable for rendering diagnoses. We quantified the relationship between ratings and image noise and radiation dose, by category, by analyzing the odds of an acceptable rating per standard deviation (SD) increase in noise or geometric SD (gSD) in dose. RESULTS: One hundred twenty-five radiologists contributed 24,800 ratings. Most (89%) were acceptable. The odds of an examination being rated acceptable statistically significantly increased per gSD increase in dose and decreased per SD increase in noise for most categories, including routine dose head, chest, and abdomen-pelvis, which together comprise 60% of examinations performed in routine practice. For routine dose abdomen-pelvis, the most common category, each gSD increase in dose raised the odds of an acceptable rating (2.33; 95% confidence interval, 1.98-3.24), whereas each SD increase in noise decreased the odds (0.90; 0.79-0.99). For only 2 CT categories, high-dose head and neck/cervical spine, neither dose nor noise was associated with ratings. CONCLUSIONS: Radiation dose and image noise correlate with radiologists' image quality assessments for most CT categories, making them suitable as automated metrics in quality programs incentivizing reduction of excessive radiation doses.

The Evidence for Using Artificial Intelligence to Enhance Prostate Cancer MR Imaging.

PURPOSE OF REVIEW: The purpose of this review is to summarize the current status of artificial intelligence applied to prostate cancer MR imaging. RECENT FINDINGS: Artificial intelligence has been applied to prostate cancer MR imaging to improve its diagnostic accuracy and reproducibility of interpretation. Multiple models have been tested for gland segmentation and volume calculation, automated lesion detection, localization, and characterization, as well as prediction of tumor aggressiveness and tumor recurrence. Studies show, for example, that very robust automated gland segmentation and volume calculations can be achieved and that lesions can be detected and accurately characterized. Although results are promising, we should view these with caution. Most studies included a small sample of patients from a single institution and most models did not undergo proper external validation. More research is needed with larger and well-design studies for the development of reliable artificial intelligence tools.

Artificial Intelligence and Radiology Education.

Implementation of artificial intelligence (AI) applications into clinical practice requires AI-savvy radiologists to ensure the safe, ethical, and effective use of these systems for patient care. Increasing demand for AI education reflects recognition of the translation of AI applications from research to clinical practice, with positive trainee attitudes regarding the influence of AI on radiology. However, barriers to AI education, such as limited access to resources, predispose to insufficient preparation for the effective use of AI in practice. In response, national organizations have sponsored formal and self-directed learning courses to provide introductory content on imaging informatics and AI. Foundational courses, such as the National Imaging Informatics Course - Radiology and the Radiological Society of North America Imaging AI Certificate, lay a framework for trainees to explore the creation, deployment, and critical evaluation of AI applications. This report includes additional resources for formal programming courses, video series from leading organizations, and blogs from AI and informatics communities. Furthermore, the scope of "AI and radiology education" includes AI-augmented radiology education, with emphasis on the potential for "precision education" that creates personalized experiences for trainees by accounting for varying learning styles and inconsistent, possibly deficient, clinical case volume. © RSNA, 2022 Keywords: Use of AI in Education, Impact of AI on Education, Artificial Intelligence, Medical Education, Imaging Informatics, Natural Language Processing, Precision Education.

Automated detection of IVC filters on radiographs with deep convolutional neural networks.

PURPOSE: To create an algorithm able to accurately detect IVC filters on radiographs without human assistance, capable of being used to screen radiographs to identify patients needing IVC filter retrieval. METHODS: A primary dataset of 5225 images, 30% of which included IVC filters, was assembled and annotated. 85% of the data was used to train a Cascade R-CNN (Region Based Convolutional Neural Network) object detection network incorporating a pre-trained ResNet-50 backbone. The remaining 15% of the data, independently annotated by three radiologists, was used as a test set to assess performance. The algorithm was also assessed on an independently constructed 1424-image dataset, drawn from a different institution than the primary dataset. RESULTS: On the primary test set, the algorithm achieved a sensitivity of 96.2% (95% CI 92.7-98.1%) and a specificity of 98.9% (95% CI 97.4-99.5%). Results were similar on the external test set: sensitivity 97.9% (95% CI 96.2-98.9%), specificity 99.6 (95% CI 98.9-99.9%). CONCLUSION: Fully automated detection of IVC filters on radiographs with high sensitivity and excellent specificity required for an automated screening system can be achieved using object detection neural networks. Further work will develop a system for identifying patients for IVC filter retrieval based on this algorithm.

Impact of patient portal-based self-scheduling of diagnostic imaging studies on health disparities.

While many case studies have described the implementation of self-scheduling tools, which allow patients to schedule visits and imaging studies asynchronously online, none have explored the impact of self-scheduling on equitable access to care.1 Using an electronic health record patient portal, University of California San Francisco deployed a self-scheduling tool that allowed patients to self-schedule diagnostic imaging studies. We analyzed electronic health record data for the imaging modalities with the option to be self-scheduled from January 1, 2021 to September 1, 2021. We used descriptive statistics to compare demographic characteristics and created a multivariable logistic regression model to identify predictors of patient self-scheduling utilization. Among all active patient portal users, Latinx, Black/African American, and non-English speaking patients were less likely to self-schedule studies. Patients with Medi-Cal, California's Medicaid program, and Medicare insurance were also less likely to self-schedule when compared with commercially insured patients. Efforts to facilitate use of patient portal-based applications are necessary to increase equitability and decrease disparities in access.

Multi-institutional Experience with Patient Image Access Through Electronic Health Record Patient Portals.

The objective is to determine patients' utilization rate of radiology image viewing through an online patient portal and to understand its impact on radiologists. IRB approval was waived. In this two-part, multi-institutional study, patients' image viewing rate was retrospectively assessed, and radiologists were anonymously surveyed for the impact of patient imaging access on their workflow. Patient access to web-based image viewing via electronic patient portals was enabled at 3 institutions (all had open radiology reports) within the past 5 years. The number of exams viewed online was compared against the total number of viewable imaging studies. An anonymized survey was distributed to radiologists at the 3 institutions, and responses were collected over 2 months. Patients viewed 14.2% of available exams - monthly open rate varied from 7.3 to 41.0%. A total of 254 radiologists responded to the survey (response rate 32.8%); 204 were aware that patients could view images. The majority (155/204; 76.0%) felt no impact on their role as radiologists; 11.8% felt negative and 9.3% positive. The majority (63.8%) were never approached by patients. Of the 86 who were contacted, 46.5% were contacted once or twice, 46.5% 3-4 times a year, and 4.7% 3-4 times a month. Free text comments included support for healthcare transparency (71), concern for patient confusion and anxiety (45), and need for attention to radiology reports and image annotations (15). A small proportion of patients viewed their radiology images. Overall, patients' image viewing had minimal impact on radiologists. Radiologists were seldom contacted by patients. While many radiologists feel supportive, some are concerned about causing patient confusion and suggest minor workflow modifications.

An Image Quality-informed Framework for CT Characterization.

Background Lack of standardization in CT protocol choice contributes to radiation dose variation. Purpose To create a framework to assess radiation doses within broad CT categories defined according to body region and clinical imaging indication and to cluster indications according to the dose required for sufficient image quality. Materials and Methods This was a retrospective study using Digital Imaging and Communications in Medicine metadata. CT examinations in adults from January 1, 2016 to December 31, 2019 from the University of California San Francisco International CT Dose Registry were grouped into 19 categories according to body region and required radiation dose levels. Five body regions had a single dose range (ie, extremities, neck, thoracolumbar spine, combined chest and abdomen, and combined thoracolumbar spine). Five additional regions were subdivided according to dose. Head, chest, cardiac, and abdomen each had low, routine, and high dose categories; combined head and neck had routine and high dose categories. For each category, the median and 75th percentile (ie, diagnostic reference level [DRL]) were determined for dose-length product, and the variation in dose within categories versus across categories was calculated and compared using an analysis of variance. Relative median and DRL (95% CI) doses comparing high dose versus low dose categories were calculated. Results Among 4.5 million examinations, the median and DRL doses varied approximately 10 times between categories compared with between indications within categories. For head, chest, abdomen, and cardiac (3 266 546 examinations [72%]), the relative median doses were higher in examinations assigned to the high dose categories than in examinations assigned to the low dose categories, suggesting the assignment of indications to the broad categories is valid (head, 3.4-fold higher [95% CI: 3.4, 3.5]; chest, 9.6 [95% CI: 9.3, 10.0]; abdomen, 2.4 [95% CI: 2.4, 2.5]; and cardiac, 18.1 [95% CI: 17.7, 18.6]). Results were similar for DRL doses (all P < .001). Conclusion Broad categories based on image quality requirements are a suitable framework for simplifying radiation dose assessment, according to expected variation between and within categories. © RSNA, 2021 See also the editorial by Mahesh in this issue.

Imaging AI in Practice: A Demonstration of Future Workflow Using Integration Standards.

Artificial intelligence (AI) tools are rapidly being developed for radiology and other clinical areas. These tools have the potential to dramatically change clinical practice; however, for these tools to be usable and function as intended, they must be integrated into existing radiology systems. In a collaborative effort between the Radiological Society of North America, radiologists, and imaging-focused vendors, the Imaging AI in Practice (IAIP) demonstrations were developed to show how AI tools can generate, consume, and present results throughout the radiology workflow in a simulated clinical environment. The IAIP demonstrations highlight the critical importance of semantic and interoperability standards, as well as orchestration profiles for successful clinical integration of radiology AI tools. Keywords: Computer Applications-General (Informatics), Technology Assessment © RSNA, 2021.

Evaluating Artificial Intelligence Systems to Guide Purchasing Decisions.

Many radiologists are considering investments in artificial intelligence (AI) to improve the quality of care for our patients. This article outlines considerations for the purchasing process beginning with performance evaluation. Practices should decide whether there is a need to independently verify performance or accept vendor-provided data. Successful implementations will consider who will receive AI results, how results will be presented, and the impact on efficiency. The article provides education on infrastructure considerations including the benefits and drawbacks of best-of-breed and platform approaches in addition to highly specialized server requirements like graphical processing unit availability. Finally, the article presents financial and quality and safety considerations, some of which are unique to AI. Examples include whether additional revenue could be obtained, as in the case of mammography, and whether an AI model unintentionally leads to reinforcing healthcare disparities.

Between Always and Never: Evaluating Uncertainty in Radiology Reports Using Natural Language Processing.

The ideal radiology report reduces diagnostic uncertainty, while avoiding ambiguity whenever possible. The purpose of this study was to characterize the use of uncertainty terms in radiology reports at a single institution and compare the use of these terms across imaging modalities, anatomic sections, patient characteristics, and radiologist characteristics. We hypothesized that there would be variability among radiologists and between subspecialities within radiology regarding the use of uncertainty terms and that the length of the impression of a report would be a predictor of use of uncertainty terms. Finally, we hypothesized that use of uncertainty terms would often be interpreted by human readers as "hedging." To test these hypotheses, we applied a natural language processing (NLP) algorithm to assess and count the number of uncertainty terms within radiology reports. An algorithm was created to detect usage of a published set of uncertainty terms. All 642,569 radiology report impressions from 171 reporting radiologists were collected from 2011 through 2015. For validation, two radiologists without knowledge of the software algorithm reviewed report impressions and were asked to determine whether the report was "uncertain" or "hedging." The relationship between the presence of 1 or more uncertainty terms and the human readers' assessment was compared. There were significant differences in the proportion of reports containing uncertainty terms across patient admission status and across anatomic imaging subsections. Reports with uncertainty were significantly longer than those without, although report length was not significantly different between subspecialities or modalities. There were no significant differences in rates of uncertainty when comparing the experience of the attending radiologist. When compared with reader 1 as a gold standard, accuracy was 0.91, sensitivity was 0.92, specificity was 0.9, and precision was 0.88, with an F1-score of 0.9. When compared with reader 2, accuracy was 0.84, sensitivity was 0.88, specificity was 0.82, and precision was 0.68, with an F1-score of 0.77. Substantial variability exists among radiologists and subspecialities regarding the use of uncertainty terms, and this variability cannot be explained by years of radiologist experience or differences in proportions of specific modalities. Furthermore, detection of uncertainty terms demonstrates good test characteristics for predicting human readers' assessment of uncertainty.

Effect of shelter-in-place on emergency department radiology volumes during the COVID-19 pandemic.

PURPOSE: The coronavirus disease 2019 (COVID-19) pandemic has led to significant disruptions in the healthcare system including surges of infected patients exceeding local capacity, closures of primary care offices, and delays of non-emergent medical care. Government-initiated measures to decrease healthcare utilization (i.e., "flattening the curve") have included shelter-in-place mandates and social distancing, which have taken effect across most of the USA. We evaluate the immediate impact of the Public Health Messaging and shelter-in-place mandates on Emergency Department (ED) demand for radiology services. METHODS: We analyzed ED radiology volumes from the five University of California health systems during a 2-week time period following the shelter-in-place mandate and compared those volumes with March 2019 and early April 2019 volumes. RESULTS: ED radiology volumes declined from the 2019 baseline by 32 to 40% (p < 0.001) across the five health systems with a total decrease in volumes across all 5 systems by 35% (p < 0.001). Stratifying by subspecialty, the smallest declines were seen in non-trauma thoracic imaging, which decreased 18% (p value < 0.001), while all other non-trauma studies decreased by 48% (p < 0.001). CONCLUSION: Total ED radiology demand may be a marker for public adherence to shelter-in-place mandates, though ED chest radiology demand may increase with an increase in COVID-19 cases.

Implementation and design of artificial intelligence in abdominal imaging.

Artificial intelligence is a technique that holds promise for helping radiologists improve the care of our patients. At the same time, implementation decisions we make now can have a long-lasting effect on patient outcomes. In the following article, we discuss four areas with unique considerations for implementation of AI: bias, trust, risk, and design. In each section, we highlight applications of AI to abdominal imaging and prostate cancer specifically.

Ethics of Artificial Intelligence in Radiology: Summary of the Joint European and North American Multisociety Statement.

This is a condensed summary of an international multisociety statement on ethics of artificial intelligence (AI) in radiology produced by the ACR, European Society of Radiology, RSNA, Society for Imaging Informatics in Medicine, European Society of Medical Imaging Informatics, Canadian Association of Radiologists, and American Association of Physicists in Medicine. AI has great potential to increase efficiency and accuracy throughout radiology, but it also carries inherent pitfalls and biases. Widespread use of AI-based intelligent and autonomous systems in radiology can increase the risk of systemic errors with high consequence and highlights complex ethical and societal issues. Currently, there is little experience using AI for patient care in diverse clinical settings. Extensive research is needed to understand how to best deploy AI in clinical practice. This statement highlights our consensus that ethical use of AI in radiology should promote well-being, minimize harm, and ensure that the benefits and harms are distributed among stakeholders in a just manner. We believe AI should respect human rights and freedoms, including dignity and privacy. It should be designed for maximum transparency and dependability. Ultimate responsibility and accountability for AI remains with its human designers and operators for the foreseeable future. The radiology community should start now to develop codes of ethics and practice for AI that promote any use that helps patients and the common good and should block use of radiology data and algorithms for financial gain without those two attributes. This article is a simultaneous joint publication in Radiology, Journal of the American College of Radiology, Canadian Association of Radiologists Journal, and Insights into Imaging. Published under a CC BY-NC-ND 4.0 license. Online supplemental material is available for this article.

Ethics of artificial intelligence in radiology: summary of the joint European and North American multisociety statement.

This is a condensed summary of an international multisociety statement on ethics of artificial intelligence (AI) in radiology produced by the ACR, European Society of Radiology, RSNA, Society for Imaging Informatics in Medicine, European Society of Medical Imaging Informatics, Canadian Association of Radiologists, and American Association of Physicists in Medicine.AI has great potential to increase efficiency and accuracy throughout radiology, but also carries inherent pitfalls and biases. Widespread use of AI-based intelligent and autonomous systems in radiology can increase the risk of systemic errors with high consequence, and highlights complex ethical and societal issues. Currently, there is little experience using AI for patient care in diverse clinical settings. Extensive research is needed to understand how to best deploy AI in clinical practice.This statement highlights our consensus that ethical use of AI in radiology should promote well-being, minimize harm, and ensure that the benefits and harms are distributed among stakeholders in a just manner. We believe AI should respect human rights and freedoms, including dignity and privacy. It should be designed for maximum transparency and dependability. Ultimate responsibility and accountability for AI remains with its human designers and operators for the foreseeable future.The radiology community should start now to develop codes of ethics and practice for AI which promote any use that helps patients and the common good and should block use of radiology data and algorithms for financial gain without those two attributes.

Ethics of Artificial Intelligence in Radiology: Summary of the Joint European and North American Multisociety Statement.

This is a condensed summary of an international multisociety statement on ethics of artificial intelligence (AI) in radiology produced by the ACR, European Society of Radiology, RSNA, Society for Imaging Informatics in Medicine, European Society of Medical Imaging Informatics, Canadian Association of Radiologists, and American Association of Physicists in Medicine. AI has great potential to increase efficiency and accuracy throughout radiology, but it also carries inherent pitfalls and biases. Widespread use of AI-based intelligent and autonomous systems in radiology can increase the risk of systemic errors with high consequence and highlights complex ethical and societal issues. Currently, there is little experience using AI for patient care in diverse clinical settings. Extensive research is needed to understand how to best deploy AI in clinical practice. This statement highlights our consensus that ethical use of AI in radiology should promote well-being, minimize harm, and ensure that the benefits and harms are distributed among stakeholders in a just manner. We believe AI should respect human rights and freedoms, including dignity and privacy. It should be designed for maximum transparency and dependability. Ultimate responsibility and accountability for AI remains with its human designers and operators for the foreseeable future. The radiology community should start now to develop codes of ethics and practice for AI that promote any use that helps patients and the common good and should block use of radiology data and algorithms for financial gain without those two attributes.

Ethics of Artificial Intelligence in Radiology: Summary of the Joint European and North American Multisociety Statement.

This is a condensed summary of an international multisociety statement on ethics of artificial intelligence (AI) in radiology produced by the ACR, European Society of Radiology, RSNA, Society for Imaging Informatics in Medicine, European Society of Medical Imaging Informatics, Canadian Association of Radiologists, and American Association of Physicists in Medicine. AI has great potential to increase efficiency and accuracy throughout radiology, but it also carries inherent pitfalls and biases. Widespread use of AI-based intelligent and autonomous systems in radiology can increase the risk of systemic errors with high consequence and highlights complex ethical and societal issues. Currently, there is little experience using AI for patient care in diverse clinical settings. Extensive research is needed to understand how to best deploy AI in clinical practice. This statement highlights our consensus that ethical use of AI in radiology should promote well-being, minimize harm, and ensure that the benefits and harms are distributed among stakeholders in a just manner. We believe AI should respect human rights and freedoms, including dignity and privacy. It should be designed for maximum transparency and dependability. Ultimate responsibility and accountability for AI remains with its human designers and operators for the foreseeable future. The radiology community should start now to develop codes of ethics and practice for AI that promote any use that helps patients and the common good and should block use of radiology data and algorithms for financial gain without those two attributes.