Leveraging Artificial Intelligence to Enhance Peer Review: Missed Liver Lesions on Computed Tomographic Pulmonary Angiography.

PURPOSE: The aim of this study was to use artificial intelligence (AI) to facilitate peer review for detection of missed suspicious liver lesions (SLLs) on CT pulmonary angiographic (CTPA) examinations. METHODS: This retrospective study included 1 month of consecutive CTPA examinations from a multisite teleradiology practice. Visual classification (VC) software analyzed images for the presence (+) or absence (-) of SLLs (>1 cm, >20 Hounsfield units). Separately, a natural language processing (NLP) algorithm evaluated corresponding reports for description (+) of an SLL or lack thereof (-). Studies containing possible missed SLLs (VC+/NLP-) were reviewed by three abdominal radiologists in a two-step adjudication process to confirm if an SLL was missed by the interpreting radiologist. The number of VC+/NLP- cases, the number of images needing radiologist review, and the number of cases with confirmed missed SLLs were recorded. Interobserver agreement for SLLs was calculated for the radiologist readers. RESULTS: A total of 2,573 CTPA examinations were assessed, and 136 were classified as potentially containing missed SLLs (VC+/NLP-). After radiologist review, 13 cases with missed SLLs were confirmed, representing 0.5% of analyzed CT studies. Using AI, the ratio of CT studies requiring review to missed SLLs identified was 10:1; the ratio without the help of AI would be at least 66:1. Among the 136 cases reviewed by radiologists, interobserver agreement for SLLs was excellent (κ = 0.91). CONCLUSIONS: AI can accelerate meaningful peer review by rapidly assessing thousands of examinations to identify potentially clinically significant errors. Although radiologist involvement is necessary, the amount of effort required after initial AI screening is dramatically reduced.

Missed Incidental Pulmonary Embolism: Harnessing Artificial Intelligence to Assess Prevalence and Improve Quality Improvement Opportunities.

PURPOSE: Incidental pulmonary embolism (IPE) can be found on body CT. The aim of this study was to evaluate the feasibility of using artificial intelligence to identify missed IPE on a large number of CT examinations. METHODS: This retrospective analysis included all single-phase chest, abdominal, and pelvic (CAP) and abdominal and pelvic (AP) CT examinations performed at a single center over 1 year, for indications other than identification of PE. Proprietary visual classification and natural language processing software was used to analyze images and reports from all CT examinations, followed by a two-step human adjudication process to classify cases as true positive, false positive, true negative, or false negative. Descriptive statistics were assessed for prevalence of IPE and features (subsegmental versus central, unifocal versus multifocal, right heart strain or not) of missed IPE. Interrater agreement for radiologist readers was also calculated. RESULTS: A total of 11,913 CT examinations (6,398 CAP, 5,515 AP) were included. Thirty false-negative examinations were identified on CAP (0.47%; 95% confidence interval [CI], 0.32%-0.67%) and nineteen false-negative studies on AP (0.34%; 95% CI, 0.21%-0.54%) studies. During manual review, readers showed substantial agreement for identification of IPE on CAP (κ = 0.76; 95% CI, 0.66-0.86) and nearly perfect agreement for identification of IPE on AP (κ = 0.86; 95% CI, 0.76-0.97). Forty-nine missed IPEs (0.41%; 95% CI, 0.30%-0.54%) were ultimately identified, compared with seventy-nine IPEs (0.66%; 95% CI, 0.53%-0.83%) identified at initial clinical interpretation. CONCLUSIONS: Artificial intelligence can efficiently analyze CT examinations to identify potential missed IPE. These results can inform peer-review efforts and quality control and could potentially be implemented in a prospective fashion.

Automated Structured Reporting for Thyroid Ultrasound: Effect on Reporting Errors and Efficiency.

PURPOSE: To compare the effectiveness of different reporting templates using the ACR Thyroid Imaging Reporting and Data System (TI-RADS) for thyroid ultrasound. METHODS: In this retrospective study, four radiologists implemented ACR TI-RADS while dictating 20 thyroid ultrasounds for each of four different templates: free text, minimally structured, fully structured, fully structured and automated (embedded software automatically sums TI-RADS points, correlates with nodule size, and inserts appropriate recommendation into report impression). In total, 80 reports were constructed per template type. Frequencies of different errors related to ACR TI-RADS were recorded: errors in point assignment, point addition, risk-level assignment, and recommendation. Reporting times were recorded, and a survey about using the template was administered. Differences in error rates were compared using χ2 and Fisher's exact tests, and differences in reporting times were compared using Kruskal-Wallis tests. RESULTS: Across all readers, errors were identified in 27.5% of reports (22 of 80) for the free text template, 28.8% (23 of 80) for the minimally structured template, 18.8% (15 of 80) for the fully structured template, and 0% (0 of 80) for the fully structured and automated template (P < .0001). Frequency of each error type (number assignment, addition, TR categorization, recommendation) decreased across the four templates (P < .0005 to P < .005). Median reporting times for the less complex templates were 210 to 240 seconds, whereas the median automated template reporting time was 180 seconds (P = .41). Radiologists subjectively preferred using the automated template. CONCLUSION: A structured reporting template for thyroid ultrasound that automatically executed steps of ACR TI-RADS resulted in fewer reporting errors for radiologists.

A Simulation Paradigm for Evaluation of Subtle Liver Lesions at Pediatric CT: Performance and Confidence.

Purpose: To create and validate a systematic observer performance platform for evaluation of simulated liver lesions at pediatric CT and to test this paradigm to measure the effect of radiation dose reduction on detection performance and reader confidence. Materials and Methods: Thirty normal pediatric (from patients aged 0-10 years) contrast material-enhanced, de-identified abdominal CT scans obtained from July 1, 2012, through July 1, 2016, were retrospectively collected from the clinical database. The study was exempt from institutional review board approval. Zero to three simulated, low-contrast liver lesions (≤6 mm) were digitally inserted by using software, and noise was added to simulate reductions in volume CT dose index (representing radiation dose estimation) of 25% and 50%. Pediatric, abdominal, and resident radiologists (three of each) reviewed 90 data sets in three sessions using an online interface, marking each lesion location and rating confidence (scale, 0-100). Statistical analysis was performed by using software. Results: Mixed-effects models revealed a significant decrease in detection sensitivity as radiation dose decreased (P < .001). The mean confidence of the full-dose and 25% dose reduction examinations was significantly higher than that of the 50% dose reduction examinations (P = .011 and .012, respectively) but not different from one another (P = .866). Dose was not a significant predictor of time to complete each case, and subspecialty was not a significant predictor of sensitivity or false-positive results. Conclusion: Sensitivity for lesion detection significantly decreased as dose decreased; however, confidence did not change between the full-dose and 25% reduced-dose scans. This suggests that readers are unaware of this decrease in performance, which should be accounted for in clinical dose reduction efforts.Keywords: Abdomen/GI, CT, Liver, Observer Performance, Pediatrics, Perception Image© RSNA, 2019.

Two Distinct Moral Mechanisms for Ascribing and Denying Intentionality.

Philosophers and legal scholars have long theorized about how intentionality serves as a critical input for morality and culpability, but the emerging field of experimental philosophy has revealed a puzzling asymmetry. People judge actions leading to negative consequences as being more intentional than those leading to positive ones. The implications of this asymmetry remain unclear because there is no consensus regarding the underlying mechanism. Based on converging behavioral and neural evidence, we demonstrate that there is no single underlying mechanism. Instead, two distinct mechanisms together generate the asymmetry. Emotion drives ascriptions of intentionality for negative consequences, while the consideration of statistical norms leads to the denial of intentionality for positive consequences. We employ this novel two-mechanism model to illustrate that morality can paradoxically shape judgments of intentionality. This is consequential for mens rea in legal practice and arguments in moral philosophy pertaining to terror bombing, abortion, and euthanasia among others.

The functional significance of shyness in anorexia nervosa.

The defining features of anorexia nervosa (AN) include disordered eating and disturbance in the experience of their bodies; however, many women with AN also demonstrate higher harm avoidance (HA), lower novelty seeking, and challenges with interpersonal functioning. The current study explored whether HA and novelty seeking could explain variation in disordered eating and social functioning in healthy control women ( n = 18), weight-restored women with a history of AN (n = 17), and women currently-ill with AN (AN; n = 17). Our results indicated that clinical participants (AN + weight-restored women) reported poorer social skills than healthy control participants. Moreover, the relationship between eating disorder symptoms and social skill deficits was mediated by HA. Follow-up analyses indicated that only the 'shyness with strangers' factor of HA independently mediated this relationship. Collectively, our results suggest a better understanding of shyness in many individuals with eating disorders could inform models of interpersonal functioning in AN.