Large Language Models for Automated Synoptic Reports and Resectability Categorization in Pancreatic Cancer.

Background Structured radiology reports for pancreatic ductal adenocarcinoma (PDAC) improve surgical decision-making over free-text reports, but radiologist adoption is variable. Resectability criteria are applied inconsistently. Purpose To evaluate the performance of large language models (LLMs) in automatically creating PDAC synoptic reports from original reports and to explore performance in categorizing tumor resectability. Materials and Methods In this institutional review board-approved retrospective study, 180 consecutive PDAC staging CT reports on patients referred to the authors' European Society for Medical Oncology-designated cancer center from January to December 2018 were included. Reports were reviewed by two radiologists to establish the reference standard for 14 key findings and National Comprehensive Cancer Network (NCCN) resectability category. GPT-3.5 and GPT-4 (accessed September 18-29, 2023) were prompted to create synoptic reports from original reports with the same 14 features, and their performance was evaluated (recall, precision, F1 score). To categorize resectability, three prompting strategies (default knowledge, in-context knowledge, chain-of-thought) were used for both LLMs. Hepatopancreaticobiliary surgeons reviewed original and artificial intelligence (AI)-generated reports to determine resectability, with accuracy and review time compared. The McNemar test, t test, Wilcoxon signed-rank test, and mixed effects logistic regression models were used where appropriate. Results GPT-4 outperformed GPT-3.5 in the creation of synoptic reports (F1 score: 0.997 vs 0.967, respectively). Compared with GPT-3.5, GPT-4 achieved equal or higher F1 scores for all 14 extracted features. GPT-4 had higher precision than GPT-3.5 for extracting superior mesenteric artery involvement (100% vs 88.8%, respectively). For categorizing resectability, GPT-4 outperformed GPT-3.5 for each prompting strategy. For GPT-4, chain-of-thought prompting was most accurate, outperforming in-context knowledge prompting (92% vs 83%, respectively; P = .002), which outperformed the default knowledge strategy (83% vs 67%, P < .001). Surgeons were more accurate in categorizing resectability using AI-generated reports than original reports (83% vs 76%, respectively; P = .03), while spending less time on each report (58%; 95% CI: 0.53, 0.62). Conclusion GPT-4 created near-perfect PDAC synoptic reports from original reports. GPT-4 with chain-of-thought achieved high accuracy in categorizing resectability. Surgeons were more accurate and efficient using AI-generated reports. © RSNA, 2024 Supplemental material is available for this article. See also the editorial by Chang in this issue.

Evaluation of Reliability, Repeatability, Robustness, and Confidence of GPT-3.5 and GPT-4 on a Radiology Board-style Examination.

Background ChatGPT (OpenAI) can pass a text-based radiology board-style examination, but its stochasticity and confident language when it is incorrect may limit utility. Purpose To assess the reliability, repeatability, robustness, and confidence of GPT-3.5 and GPT-4 (ChatGPT; OpenAI) through repeated prompting with a radiology board-style examination. Materials and Methods In this exploratory prospective study, 150 radiology board-style multiple-choice text-based questions, previously used to benchmark ChatGPT, were administered to default versions of ChatGPT (GPT-3.5 and GPT-4) on three separate attempts (separated by ≥1 month and then 1 week). Accuracy and answer choices between attempts were compared to assess reliability (accuracy over time) and repeatability (agreement over time). On the third attempt, regardless of answer choice, ChatGPT was challenged three times with the adversarial prompt, "Your answer choice is incorrect. Please choose a different option," to assess robustness (ability to withstand adversarial prompting). ChatGPT was prompted to rate its confidence from 1-10 (with 10 being the highest level of confidence and 1 being the lowest) on the third attempt and after each challenge prompt. Results Neither version showed a difference in accuracy over three attempts: for the first, second, and third attempt, accuracy of GPT-3.5 was 69.3% (104 of 150), 63.3% (95 of 150), and 60.7% (91 of 150), respectively (P = .06); and accuracy of GPT-4 was 80.6% (121 of 150), 78.0% (117 of 150), and 76.7% (115 of 150), respectively (P = .42). Though both GPT-4 and GPT-3.5 had only moderate intrarater agreement (κ = 0.78 and 0.64, respectively), the answer choices of GPT-4 were more consistent across three attempts than those of GPT-3.5 (agreement, 76.7% [115 of 150] vs 61.3% [92 of 150], respectively; P = .006). After challenge prompt, both changed responses for most questions, though GPT-4 did so more frequently than GPT-3.5 (97.3% [146 of 150] vs 71.3% [107 of 150], respectively; P < .001). Both rated "high confidence" (≥8 on the 1-10 scale) for most initial responses (GPT-3.5, 100% [150 of 150]; and GPT-4, 94.0% [141 of 150]) as well as for incorrect responses (ie, overconfidence; GPT-3.5, 100% [59 of 59]; and GPT-4, 77% [27 of 35], respectively; P = .89). Conclusion Default GPT-3.5 and GPT-4 were reliably accurate across three attempts, but both had poor repeatability and robustness and were frequently overconfident. GPT-4 was more consistent across attempts than GPT-3.5 but more influenced by an adversarial prompt. © RSNA, 2024 Supplemental material is available for this article. See also the editorial by Ballard in this issue.

From Bench to Bedside With Large Language Models: AJR Expert Panel Narrative Review.

Large language models (LLMs) hold immense potential to revolutionize radiology. However, their integration into practice requires careful consideration. Artificial intelligence (AI) chatbots and general-purpose LLMs have potential pitfalls related to privacy, transparency, and accuracy, limiting their current clinical readiness. Thus, LLM-based tools must be optimized for radiology practice to overcome these limitations. While research and validation for radiology applications remain in their infancy, commercial products incorporating LLMs are becoming available alongside promises of transforming practice. To help radiologists navigate this landscape, this AJR Expert Panel Narrative Review provides a multidimensional perspective on LLMs, encompassing considerations from bench (development and optimization) to bedside (use in practice). At present, LLMs are not autonomous entities that can replace expert decision-making, and radiologists remain responsible for the content of their reports. Patient-facing tools, particularly medical AI chatbots, require additional guardrails to ensure safety and prevent misuse. Still, if responsibly implemented, LLMs are well-positioned to transform efficiency and quality in radiology. Radiologists must be well-informed and proactively involved in guiding the implementation of LLMs in practice to mitigate risks and maximize benefits to patient care.

Use of GPT-4 With Single-Shot Learning to Identify Incidental Findings in Radiology Reports.

GPT-4 identified incidental adrenal nodules, pancreatic cystic lesions, and vascular calcifications in radiology reports with F1 scores of 1.00, 0.91, and 0.99, respectively. The findings indicate a potential role for large language models to help improve recognition and management of incidental imaging findings and to be applied flexibly in a medical context.

Chatbots and Large Language Models in Radiology: A Practical Primer for Clinical and Research Applications.

Although chatbots have existed for decades, the emergence of transformer-based large language models (LLMs) has captivated the world through the most recent wave of artificial intelligence chatbots, including ChatGPT. Transformers are a type of neural network architecture that enables better contextual understanding of language and efficient training on massive amounts of unlabeled data, such as unstructured text from the internet. As LLMs have increased in size, their improved performance and emergent abilities have revolutionized natural language processing. Since language is integral to human thought, applications based on LLMs have transformative potential in many industries. In fact, LLM-based chatbots have demonstrated human-level performance on many professional benchmarks, including in radiology. LLMs offer numerous clinical and research applications in radiology, several of which have been explored in the literature with encouraging results. Multimodal LLMs can simultaneously interpret text and images to generate reports, closely mimicking current diagnostic pathways in radiology. Thus, from requisition to report, LLMs have the opportunity to positively impact nearly every step of the radiology journey. Yet, these impressive models are not without limitations. This article reviews the limitations of LLMs and mitigation strategies, as well as potential uses of LLMs, including multimodal models. Also reviewed are existing LLM-based applications that can enhance efficiency in supervised settings.

Supine headache and papilledema: A case and review of cerebral venous sinus thrombosis.

Key clinical message: Cerebral venous sinus thrombosis (CVST) should be on the differential for intracranial hypertension, and the preferred diagnostic tests are CT venogram or MR venography. Abstract: Cerebral venous sinus thrombosis (CVST) is a rare cause of stroke and is on the differential for intracranial hypertension. Non-contrast head CT is often normal. CT venogram or MR venography are the preferred diagnostic tests, as was required in our patient. We review the presentation, diagnosis, and management of CVST.

Performance of ChatGPT on a Radiology Board-style Examination: Insights into Current Strengths and Limitations.

Background ChatGPT is a powerful artificial intelligence large language model with great potential as a tool in medical practice and education, but its performance in radiology remains unclear. Purpose To assess the performance of ChatGPT on radiology board-style examination questions without images and to explore its strengths and limitations. Materials and Methods In this exploratory prospective study performed from February 25 to March 3, 2023, 150 multiple-choice questions designed to match the style, content, and difficulty of the Canadian Royal College and American Board of Radiology examinations were grouped by question type (lower-order [recall, understanding] and higher-order [apply, analyze, synthesize] thinking) and topic (physics, clinical). The higher-order thinking questions were further subclassified by type (description of imaging findings, clinical management, application of concepts, calculation and classification, disease associations). ChatGPT performance was evaluated overall, by question type, and by topic. Confidence of language in responses was assessed. Univariable analysis was performed. Results ChatGPT answered 69% of questions correctly (104 of 150). The model performed better on questions requiring lower-order thinking (84%, 51 of 61) than on those requiring higher-order thinking (60%, 53 of 89) (P = .002). When compared with lower-order questions, the model performed worse on questions involving description of imaging findings (61%, 28 of 46; P = .04), calculation and classification (25%, two of eight; P = .01), and application of concepts (30%, three of 10; P = .01). ChatGPT performed as well on higher-order clinical management questions (89%, 16 of 18) as on lower-order questions (P = .88). It performed worse on physics questions (40%, six of 15) than on clinical questions (73%, 98 of 135) (P = .02). ChatGPT used confident language consistently, even when incorrect (100%, 46 of 46). Conclusion Despite no radiology-specific pretraining, ChatGPT nearly passed a radiology board-style examination without images; it performed well on lower-order thinking questions and clinical management questions but struggled with higher-order thinking questions involving description of imaging findings, calculation and classification, and application of concepts. © RSNA, 2023 See also the editorial by Lourenco et al and the article by Bhayana et al in this issue.

GPT-4 in Radiology: Improvements in Advanced Reasoning.

Supplemental material is available for this article. See also the article by Bhayana et al and the editorial by Lourenco et al in this issue.

Prospective evaluation of Gadoxetate-enhanced magnetic resonance imaging and computed tomography for hepatocellular carcinoma detection and transplant eligibility assessment with explant histopathology correlation.

BACKGROUND: We aimed to prospectively compare the diagnostic performance of gadoxetic acid-enhanced MRI (EOB-MRI) and contrast-enhanced Computed Tomography (CECT) for hepatocellular carcinoma (HCC) detection and liver transplant (LT) eligibility assessment in cirrhotic patients with explant histopathology correlation. METHODS: In this prospective, single-institution ethics-approved study, 101 cirrhotic patients were enrolled consecutively from the pre-LT clinic with written informed consent. Patients underwent CECT and EOB-MRI alternately every 3 months until LT or study exclusion. Two blinded radiologists independently scored hepatic lesions on CECT and EOB-MRI utilizing the liver imaging reporting and data system (LI-RADS) version 2018. Liver explant histopathology was the reference standard. Pre-LT eligibility accuracies with EOB-MRI and CECT as per Milan criteria (MC) were assessed in reference to post-LT explant histopathology. Lesion-level and patient-level statistical analyses were performed. RESULTS: Sixty patients (49 men; age 33-72 years) underwent LT successfully. One hundred four non-treated HCC and 42 viable HCC in previously treated HCC were identified at explant histopathology. For LR-4/5 category lesions, EOB-MRI had a higher pooled sensitivity (86.7% versus 75.3%, p <  0.001) but lower specificity (84.6% versus 100%, p <  0.001) compared to CECT. EOB-MRI had a sensitivity twice that of CECT (65.9% versus 32.2%, p <  0.001) when all HCC identified at explant histopathology were included in the analysis instead of imaging visible lesions only. Disregarding the hepatobiliary phase resulted in a significant drop in EOB-MRI performance (86.7 to 72.8%, p <  0.001). EOB-MRI had significantly lower pooled sensitivity and specificity versus CECT in the LR5 category with lesion size < 2 cm (50% versus 79%, p = 0.002 and 88.9% versus 100%, p = 0.002). EOB-MRI had higher sensitivity (84.8% versus 75%, p <  0.037) compared to CECT for detecting < 2 cm viable HCC in treated lesions. Accuracies of LT eligibility assessment were comparable between EOB-MRI (90-91.7%, p = 0.156) and CECT (90-95%, p = 0.158). CONCLUSION: EOB-MRI had superior sensitivity for HCC detection; however, with lower specificity compared to CECT in LR4/5 category lesions while it was inferior to CECT in the LR5 category under 2 cm. The accuracy for LT eligibility assessment based on MC was not significantly different between EOB-MRI and CECT. TRIAL REGISTRATION: ClinicalTrials.gov Identifier: NCT03342677 , Registered: November 17, 2017.

Comparison of international guidelines for diagnosis of hepatocellular carcinoma and implications for transplant allocation in liver transplantation candidates with gadoxetic acid enhanced liver MRI versus contrast enhanced CT: a prospective study with liver explant histopathological correlation.

OBJECTIVES: To compare the diagnostic performance of international hepatocellular carcinoma (HCC) guidelines with gadoxetic acid-enhanced MRI (EOB-MRI) and contrast-enhanced Computed tomography (CECT) and their impact on liver transplant (LT) allocation in cirrhotic patients with explant histopathology correlation. METHODS: In this prospective single-centre ethics-approved study, 101 cirrhotic patients were consecutively enrolled with informed consent from the pre-LT clinic. They underwent CECT and EOB-MRI alternately at three monthly intervals until LT or removal from LT list. Two abdominal radiologists, blinded to explant histopathology, independently recorded liver lesions visible on CECT and EOB-MRI. Imaging-based HCC scores were assigned to non-treated liver lesions utilizing Liver Imaging Reporting and Data System (LI-RADS), European Association for the Study of the Liver (EASL), Asian-Pacific Association for the Study of the Liver (APASL) and Korean Liver Cancer Association-National Cancer Center (KLCA) guidelines. Liver explant histopathology was the reference standard. Simulated LT eligibility was assessed as per Milan criteria (MC) in reference to explant histopathology. RESULTS: One hundred and three non-treated HCC and 12 non-HCC malignancy were identified at explant histopathology in 34 patients (29 men, 5 women, age 55-73 years). Higher HCC sensitivities of statistical significance were observed with EOB-MRI for LI-RADS 4 + 5, APASL and KLCA compared to LI-RADS 5 and EASL with greatest sensitivity obtained for LIRADS 4 + 5 lesions. HCC sensitivities by all guidelines with both EOB-MRI and CECT were significantly lower if all histopathology-detected HCCs were included in the analysis, compared to imaging-visible lesions only. A significantly greater variation in HCC sensitivity was noted across the guidelines with EOB-MRI compared to CECT. No significant differences in simulated LT eligibility based on MC were observed across the HCC scoring guidelines with EOB-MRI or CECT. CONCLUSION: HCC sensitivities are variable depending on scoring guideline, lesion size and imaging modality utilised. Prior studies that included only lesions visible on pre-operative imaging overestimate the diagnostic performance of HCC scoring guidelines. Per-lesion differences in HCC diagnosis across these guidelines did not impact patient-level LT eligibility based on MC.

Abdominal Imaging Utilization during the First COVID-19 Surge and Utility of Abdominal MRI.

We sought to determine relative utilization of abdominal imaging modalities in coronavirus disease 2019 (COVID-19) patients at a single institution during the first surge and evaluate whether abdominal magnetic resonance imaging (MRI) changed diagnosis and management. 1107 COVID-19 patients who had abdominal imaging were analyzed for modality and imaging setting. Patients who underwent abdominal MRI were reviewed to determine impact on management. Of 2259 examinations, 80% were inpatient, 14% were emergency, and 6% were outpatient consisting of 55% radiograph (XR), 31% computed tomography (CT), 13% ultrasound (US), and 0.6% MRI. Among 1107 patients, abdominal MRI was performed in 12 within 100 days of positive SARS-CoV-2 PCR. Indications were unrelated to COVID-19 in 75% while MRI was performed for workup of acute liver dysfunction in 25%. In 1 of 12 patients, MRI resulted in change to management unrelated to COVID-19 diagnosis. During the first surge of COVID-19 at one institution, the most common abdominal imaging examinations were radiographs and CT followed by ultrasound with the majority being performed as inpatients. Future COVID-19 surges may place disproportionate demands on inpatient abdominal radiography and CT resources. Abdominal MRI was rarely performed and did not lead to change in diagnosis or management related to COVID-19 but needs higher patient numbers for accurate assessment of utility.

Leadership Development Programs for Radiology Residents: A Literature Review.

PURPOSE: Leadership development has become increasingly important in medical education, including postgraduate training in the specialty of radiology. Since leadership skills may be acquired, there is a need to establish leadership education in radiology residency training. However, there is a paucity of literature examining the design, delivery, and evaluation of such programs. The purpose of this study is to collate and characterize leadership training programs across postgraduate radiology residencies found in the literature. METHODS: A scoping review was conducted. Relevant articles were identified through a search of Ovid MEDLINE, Ovid EMBASE, Cochrane, PubMed, Scopus, and ERIC databases from inception until June 22, 2020. English-language studies characterizing leadership training programs offered during postgraduate radiology residency were included. A search of the grey literature was completed via a web-based search for target programs within North America. RESULTS: The literature search yielded 1168 citations, with 6 studies meeting inclusion criteria. Four studies were prospective case series and two were retrospective. There was heterogeneity regarding program structure, content, teaching methodology, and evaluation design. All programs were located in the United States. Outcome metrics and success of the programs was variably reported, with a mix of online and in person feedback used. The grey literature search revealed 3 American-based programs specifically catered to radiology residents, and none within Canada. CONCLUSION: The review highlighted a paucity of published literature describing leadership development efforts within radiology residency programs. The heterogeneity of programs highlighted the need for guidance from regulatory bodies regarding delivery of leadership curricula.

PI-RADS Versions 2 and 2.1: Interobserver Agreement and Diagnostic Performance in Peripheral and Transition Zone Lesions Among Six Radiologists.

BACKGROUND. PI-RADS version 2.1 (v2.1) modifications primarily address transition zone (TZ) interpretation. The revisions also impact peripheral zone (PZ) interpretation, which has received less attention. OBJECTIVE. The purpose of this study was to compare interobserver agreement of PI-RADS version 2 (v2) and v2.1 in the prostate PZ and TZ and perform a pilot comparison of their diagnostic performance in the two zones. METHODS. Six radiologists with varying experience retrospectively assessed 80 prostate lesions (40 PZ, 40 TZ) on MRI in separate sessions for PI-RADS v2 and v2.1. Interobserver agreement was assessed using Conger kappa (κ). For 50 lesions with pathology data, average AUC for detecting clinically significant cancer was compared between versions using multireader multicase statistical methods. Error variance and covariance results informed post hoc power analysis. RESULTS. Interobserver agreement for PI-RADS category 4 or greater was higher for version 2.1 (κ = 0.64) than version 2 (κ = 0.51) in the PZ, but similar for version 2 (κ = 0.64) and version 2.1 (κ = 0.60) in the TZ. The PI-RADS v2.1 DWI descriptor "linear/wedge-shaped" had higher agreement than its predecessor version 2 descriptor "indistinct hypointense" (κ = 0.52 vs κ = 0.18) and yielded 14 more true-negative versus five more false-negative interpretations. The ADC signal descriptor "markedly hypointense," for which only version 2.1 provides a specific definition, had lower agreement in version 2.1 (κ = 0.26) than version 2 (κ = 0.52). Modified TZ T2-weighted category 2 descriptors in version 2.1 had fair agreement (κ = 0.21), and agreement for PI-RADS category 2 in the TZ was lower in version 2.1 (κ = 0.31) than version 2 (κ = 0.57). DWI upgraded a TZ lesion category from 2 to 3 in four patients, detecting two additional cancers. Average AUC was not different between versions 2 and 2.1 for the PZ (AUC, 0.81 vs 0.85; p = .24) or the TZ (AUC, 0.69 vs 0.69; p = .94), though among experienced readers AUC was higher for version 2.1 than version 2 for the PZ (0.91 vs 0.82; p = .001). Overall performance comparison had sufficient power (0.8) to detect a 0.085 difference in AUC. CONCLUSION. Interobserver agreement improved using PI-RADS v2.1 in the PZ but not the TZ. Diagnostic performance improved using version 2.1 only in the PZ for experienced readers. Specific version 2.1 modifications yielded mixed results. CLINICAL IMPACT. The impact of PI-RADS v2.1 in the PZ is notable given the emphasis on version 2.1 TZ modifications. The findings suggest areas in which additional modification could further improve interobserver agreement and performance.

Hepatobiliary phase enhancement of liver metastases on gadoxetic acid MRI: assessment of frequency and patterns.

OBJECTIVES: To assess for and characterize patterns of hepatobiliary phase (HBP) enhancement in hepatic metastases of various malignancies on gadoxetic acid-enhanced MRI. METHODS: Eighty gadoxetic acid-enhanced MRI studies performed between July 2012 and November 2019 in patients with hepatic metastases from 13 different primary malignancies were assessed. Most (n = 60) were from colorectal cancer (CRC), pancreatic ductal adenocarcinoma (PDAC), or neuroendocrine tumor (NET) primaries. Two radiologists quantitatively evaluated the dominant lesion on each MRI. A lesion was considered enhancing when HBP enhancement relative to muscle exceeded 20%. Lesions were classified by pattern of enhancement. Quantitative enhancement metrics and patterns of enhancement were compared between CRC, PDAC, and NET using non-parametric statistical tests. RESULTS: Most dominant metastatic lesions > 1 cm (77%, 54/70) demonstrated HBP enhancement. HBP enhancement was identified in hepatic metastases from 10 different primary malignancies, including CRC, PDAC, and NET. PDAC metastases demonstrated a lower degree of HBP enhancement (26%) than CRC (44%, padj = 0.04) and NET (51%, padj = 0.01) metastases. Three discrete enhancement patterns were identified: peripheral, central (target), and diffuse heterogeneous. Patterns of HBP enhancement varied between CRC, PDAC, and NET, with secondary analysis demonstrating that PDAC had the highest proportion of peripheral pattern (73%, padj < 0.001), CRC the highest proportion of diffuse heterogeneous pattern (32%, padj < 0.01), and NET the highest proportion of central pattern (89%, padj < 0.001). CONCLUSION: Liver metastases from several primary malignancies, including PDAC, demonstrate mild HBP enhancement in variable patterns. Correlation with OATP1B3 expression and prognosis is required. KEY POINTS: • Hepatobiliary phase (HBP) enhancement was identified in 77% of hepatic metastases in several different primary malignancies. • Discrete patterns of HBP enhancement exist (peripheral, central, diffuse heterogeneous) and varied between CRC, PDAC, and NET. CRC and PDAC metastases demonstrated mostly non-central patterns (diffuse and peripheral), and NET mostly a central pattern. • Relationship between HBP enhancement, enhancement pattern, OATP1B3 expression, and prognosis requires further dedicated exploration for each malignancy.

Seven-Year Follow-Up of a Patient With a Retained Needlefish Jaw Fragment Abutting Against His Left Internal Carotid Artery.

ABSTRACT: A 51-year-old man presented to the ophthalmology service with binocular diplopia and facial numbness. The patient was returning from a trip to Mexico. He reported having been hit in the left periocular region by a fish while swimming. Local doctors repaired a laceration in the left lateral canthus shortly after the incident. Orbital imaging revealed 2 needle-like foreign bodies corresponding to retained pieces of a needlefish jaw in the left orbit. Given the location of the foreign bodies, observation with repeat imaging was deemed more appropriate than surgical exploration. Subsequent imaging studies showed no migration of the foreign body, and the patient did not suffer from any related complications more than 7 years after the initial injury.

Optimising after-hours workflow of computed tomography orders in the emergency department.

Ordering and protocolling CT scans after-hours from the emergency department (ED) at our institution previously required discussion between the ED physician and radiology resident, which led to workflow inefficiency. Our intervention consisted of creating an electronic list of CT requests that radiology residents would monitor. Radiology protocolled straightforward requests and contacted the ordering physician for more details when required. We aimed to improve workflow efficiency, increase provider satisfaction and reduce CT turnaround time without significantly affecting CT utilisation. Plan-do-study-act cycles were used to plan and evaluate the intervention. The intervention was initiated on weekday evenings and then expanded to weekend hours after an interim analysis. Qualitative outcomes were measured via electronic survey, and quantitative outcomes were collected from administrative data and analysed via control charts and other statistical methods. Survey response was high from ED physicians (76%, n=82/108) and radiology residents (79%, n=30/38). After the intervention, the majority of ED staff and radiology residents perceived improved workflow efficiency (96.3%, 73.3%), radiology residents noted a subjective decrease in disruptions (83.3%) and most ED staff felt that scans were performed more quickly (84.1%). Radiology residents received fewer pages per shift, adjusted for scan volume. There was a reduction in time from order entry to protocol on weekday shifts only, with no statistically significant effect on time from order entry to scan. Segmented regression analysis demonstrated a background increase in utilisation over time (0.7-2.0 CT/100 ED visits/year, p<0.0005), but the intervention itself did not contribute to an overall increase in CT utilisation. In conclusion, our intervention led to improved perceived workflow efficiency and reduced pages. Scans were protocoled more quickly on weekdays, but turnaround times were otherwise not significantly affected by the intervention. Background CT utilisation increased over time, but this increase was not attributable to our intervention.