A review of self-supervised, generative, and few-shot deep learning methods for data-limited magnetic resonance imaging segmentation.

Magnetic resonance imaging (MRI) is a ubiquitous medical imaging technology with applications in disease diagnostics, intervention, and treatment planning. Accurate MRI segmentation is critical for diagnosing abnormalities, monitoring diseases, and deciding on a course of treatment. With the advent of advanced deep learning frameworks, fully automated and accurate MRI segmentation is advancing. Traditional supervised deep learning techniques have advanced tremendously, reaching clinical-level accuracy in the field of segmentation. However, these algorithms still require a large amount of annotated data, which is oftentimes unavailable or impractical. One way to circumvent this issue is to utilize algorithms that exploit a limited amount of labeled data. This paper aims to review such state-of-the-art algorithms that use a limited number of annotated samples. We explain the fundamental principles of self-supervised learning, generative models, few-shot learning, and semi-supervised learning and summarize their applications in cardiac, abdomen, and brain MRI segmentation. Throughout this review, we highlight algorithms that can be employed based on the quantity of annotated data available. We also present a comprehensive list of notable publicly available MRI segmentation datasets. To conclude, we discuss possible future directions of the field-including emerging algorithms, such as contrastive language-image pretraining, and potential combinations across the methods discussed-that can further increase the efficacy of image segmentation with limited labels.

Discovery Viewer (DV): Web-Based Medical AI Model Development Platform and Deployment Hub.

The rapid rise of artificial intelligence (AI) in medicine in the last few years highlights the importance of developing bigger and better systems for data and model sharing. However, the presence of Protected Health Information (PHI) in medical data poses a challenge when it comes to sharing. One potential solution to mitigate the risk of PHI breaches is to exclusively share pre-trained models developed using private datasets. Despite the availability of these pre-trained networks, there remains a need for an adaptable environment to test and fine-tune specific models tailored for clinical tasks. This environment should be open for peer testing, feedback, and continuous model refinement, allowing dynamic model updates that are especially important in the medical field, where diseases and scanning techniques evolve rapidly. In this context, the Discovery Viewer (DV) platform was developed in-house at the Biomedical Engineering and Imaging Institute at Mount Sinai (BMEII) to facilitate the creation and distribution of cutting-edge medical AI models that remain accessible after their development. The all-in-one platform offers a unique environment for non-AI experts to learn, develop, and share their own deep learning (DL) concepts. This paper presents various use cases of the platform, with its primary goal being to demonstrate how DV holds the potential to empower individuals without expertise in AI to create high-performing DL models. We tasked three non-AI experts to develop different musculoskeletal AI projects that encompassed segmentation, regression, and classification tasks. In each project, 80% of the samples were provided with a subset of these samples annotated to aid the volunteers in understanding the expected annotation task. Subsequently, they were responsible for annotating the remaining samples and training their models through the platform's "Training Module". The resulting models were then tested on the separate 20% hold-off dataset to assess their performance. The classification model achieved an accuracy of 0.94, a sensitivity of 0.92, and a specificity of 1. The regression model yielded a mean absolute error of 14.27 pixels. And the segmentation model attained a Dice Score of 0.93, with a sensitivity of 0.9 and a specificity of 0.99. This initiative seeks to broaden the community of medical AI model developers and democratize the access of this technology to all stakeholders. The ultimate goal is to facilitate the transition of medical AI models from research to clinical settings.

Deep Learning for Automated Measurement of Patellofemoral Anatomic Landmarks.

BACKGROUND: Patellofemoral anatomy has not been well characterized. Applying deep learning to automatically measure knee anatomy can provide a better understanding of anatomy, which can be a key factor in improving outcomes. METHODS: 483 total patients with knee CT imaging (April 2017-May 2022) from 6 centers were selected from a cohort scheduled for knee arthroplasty and a cohort with healthy knee anatomy. A total of 7 patellofemoral landmarks were annotated on 14,652 images and approved by a senior musculoskeletal radiologist. A two-stage deep learning model was trained to predict landmark coordinates using a modified ResNet50 architecture initialized with self-supervised learning pretrained weights on RadImageNet. Landmark predictions were evaluated with mean absolute error, and derived patellofemoral measurements were analyzed with Bland-Altman plots. Statistical significance of measurements was assessed by paired t-tests. RESULTS: Mean absolute error between predicted and ground truth landmark coordinates was 0.20/0.26 cm in the healthy/arthroplasty cohort. Four knee parameters were calculated, including transepicondylar axis length, transepicondylar-posterior femur axis angle, trochlear medial asymmetry, and sulcus angle. There were no statistically significant parameter differences (p > 0.05) between predicted and ground truth measurements in both cohorts, except for the healthy cohort sulcus angle. CONCLUSION: Our model accurately identifies key trochlear landmarks with ~0.20-0.26 cm accuracy and produces human-comparable measurements on both healthy and pathological knees. This work represents the first deep learning regression model for automated patellofemoral annotation trained on both physiologic and pathologic CT imaging at this scale. This novel model can enhance our ability to analyze the anatomy of the patellofemoral compartment at scale.

Interstitial lung disease diagnosis and prognosis using an AI system integrating longitudinal data.

For accurate diagnosis of interstitial lung disease (ILD), a consensus of radiologic, pathological, and clinical findings is vital. Management of ILD also requires thorough follow-up with computed tomography (CT) studies and lung function tests to assess disease progression, severity, and response to treatment. However, accurate classification of ILD subtypes can be challenging, especially for those not accustomed to reading chest CTs regularly. Dynamic models to predict patient survival rates based on longitudinal data are challenging to create due to disease complexity, variation, and irregular visit intervals. Here, we utilize RadImageNet pretrained models to diagnose five types of ILD with multimodal data and a transformer model to determine a patient's 3-year survival rate. When clinical history and associated CT scans are available, the proposed deep learning system can help clinicians diagnose and classify ILD patients and, importantly, dynamically predict disease progression and prognosis.

Influence of thoracic radiology training on classification of interstitial lung diseases.

INTRODUCTION: Interpretation of high-resolution CT images plays an important role in the diagnosis and management of interstitial lung diseases. However, interreader variation may exist due to varying levels of training and expertise. This study aims to evaluate interreader variation and the role of thoracic radiology training in classifying interstitial lung disease (ILD). METHODS: This is a retrospective study where seven physicians (radiologists, thoracic radiologists, and a pulmonologist) classified the subtypes of ILD of 128 patients from a tertiary referral center, all selected from the Interstitial Lung Disease Registry which consists of patients from November 2014 to January 2021. Each patient was diagnosed with a subtype of interstitial lung disease by a consensus diagnosis from pathology, radiology, and pulmonology. Each reader was provided with only clinical history, only CT images, or both. Reader sensitivity and specificity and interreader agreements using Cohen's κ were calculated. RESULTS: Interreader agreement based only on clinical history, only on radiologic information, or combination of both was most consistent amongst readers with thoracic radiology training, ranging from fair (Cohen's κ: 0.2-0.46), moderate to almost perfect (Cohen's κ: 0.55-0.92), and moderate to almost perfect (Cohen's κ: 0.53-0.91) respectively. Radiologists with any thoracic training showed both increased sensitivity and specificity for NSIP as compared to other radiologists and the pulmonologist when using only clinical history, only CT information, or combination of both (p < 0.05). CONCLUSIONS: Readers with thoracic radiology training showed the least interreader variation and were more sensitive and specific at classifying certain subtypes of ILD. SUMMARY SENTENCE: Thoracic radiology training may improve sensitivity and specificity in classifying ILD based on HRCT images and clinical history.

RadImageNet: An Open Radiologic Deep Learning Research Dataset for Effective Transfer Learning.

Purpose: To demonstrate the value of pretraining with millions of radiologic images compared with ImageNet photographic images on downstream medical applications when using transfer learning. Materials and Methods: This retrospective study included patients who underwent a radiologic study between 2005 and 2020 at an outpatient imaging facility. Key images and associated labels from the studies were retrospectively extracted from the original study interpretation. These images were used for RadImageNet model training with random weight initiation. The RadImageNet models were compared with ImageNet models using the area under the receiver operating characteristic curve (AUC) for eight classification tasks and using Dice scores for two segmentation problems. Results: The RadImageNet database consists of 1.35 million annotated medical images in 131 872 patients who underwent CT, MRI, and US for musculoskeletal, neurologic, oncologic, gastrointestinal, endocrine, abdominal, and pulmonary pathologic conditions. For transfer learning tasks on small datasets-thyroid nodules (US), breast masses (US), anterior cruciate ligament injuries (MRI), and meniscal tears (MRI)-the RadImageNet models demonstrated a significant advantage (P < .001) to ImageNet models (9.4%, 4.0%, 4.8%, and 4.5% AUC improvements, respectively). For larger datasets-pneumonia (chest radiography), COVID-19 (CT), SARS-CoV-2 (CT), and intracranial hemorrhage (CT)-the RadImageNet models also illustrated improved AUC (P < .001) by 1.9%, 6.1%, 1.7%, and 0.9%, respectively. Additionally, lesion localizations of the RadImageNet models were improved by 64.6% and 16.4% on thyroid and breast US datasets, respectively. Conclusion: RadImageNet pretrained models demonstrated better interpretability compared with ImageNet models, especially for smaller radiologic datasets.Keywords: CT, MR Imaging, US, Head/Neck, Thorax, Brain/Brain Stem, Evidence-based Medicine, Computer Applications-General (Informatics) Supplemental material is available for this article. Published under a CC BY 4.0 license.See also the commentary by Cadrin-Chênevert in this issue.

U-Net Based Segmentation and Characterization of Gliomas.

(1) Background: Gliomas are the most common primary brain neoplasms accounting for roughly 40−50% of all malignant primary central nervous system tumors. We aim to develop a deep learning-based framework for automated segmentation and prediction of biomarkers and prognosis in patients with gliomas. (2) Methods: In this retrospective two center study, patients were included if they (1) had a diagnosis of glioma with known surgical histopathology and (2) had preoperative MRI with FLAIR sequence. The entire tumor volume including FLAIR hyperintense infiltrative component and necrotic and cystic components was segmented. Deep learning-based U-Net framework was developed based on symmetric architecture from the 512 × 512 segmented maps from FLAIR as the ground truth mask. (3) Results: The final cohort consisted of 208 patients with mean ± standard deviation of age (years) of 56 ± 15 with M/F of 130/78. DSC of the generated mask was 0.93. Prediction for IDH-1 and MGMT status had a performance of AUC 0.88 and 0.62, respectively. Survival prediction of <18 months demonstrated AUC of 0.75. (4) Conclusions: Our deep learning-based framework can detect and segment gliomas with excellent performance for the prediction of IDH-1 biomarker status and survival.

Clonally expanded CD8 T cells characterize amyotrophic lateral sclerosis-4.

Amyotrophic lateral sclerosis (ALS) is a heterogenous neurodegenerative disorder that affects motor neurons and voluntary muscle control1. ALS heterogeneity includes the age of manifestation, the rate of progression and the anatomical sites of symptom onset. Disease-causing mutations in specific genes have been identified and define different subtypes of ALS1. Although several ALS-associated genes have been shown to affect immune functions2, whether specific immune features account for ALS heterogeneity is poorly understood. Amyotrophic lateral sclerosis-4 (ALS4) is characterized by juvenile onset and slow progression3. Patients with ALS4 show motor difficulties by the time that they are in their thirties, and most of them require devices to assist with walking by their fifties. ALS4 is caused by mutations in the senataxin gene (SETX). Here, using Setx knock-in mice that carry the ALS4-causative L389S mutation, we describe an immunological signature that consists of clonally expanded, terminally differentiated effector memory (TEMRA) CD8 T cells in the central nervous system and the blood of knock-in mice. Increased frequencies of antigen-specific CD8 T cells in knock-in mice mirror the progression of motor neuron disease and correlate with anti-glioma immunity. Furthermore, bone marrow transplantation experiments indicate that the immune system has a key role in ALS4 neurodegeneration. In patients with ALS4, clonally expanded TEMRA CD8 T cells circulate in the peripheral blood. Our results provide evidence of an antigen-specific CD8 T cell response in ALS4, which could be used to unravel disease mechanisms and as a potential biomarker of disease state.

Radiology Implementation Considerations for Artificial Intelligence (AI) Applied to COVID-19, From the AJR Special Series on AI Applications.

Hundreds of imaging-based artificial intelligence (AI) models have been developed in response to the COVID-19 pandemic. AI systems that incorporate imaging have shown promise in primary detection, severity grading, and prognostication of outcomes in COVID-19, and have enabled integration of imaging with a broad range of additional clinical and epidemiologic data. However, systematic reviews of AI models applied to COVID-19 medical imaging have highlighted problems in the field, including methodologic issues and problems in real-world deployment. Clinical use of such models should be informed by both the promise and potential pitfalls of implementation. How does a practicing radiologist make sense of this complex topic, and what factors should be considered in the implementation of AI tools for imaging of COVID-19? This critical review aims to help the radiologist understand the nuances that impact the clinical deployment of AI for imaging of COVID-19. We review imaging use cases for AI models in COVID-19 (e.g., diagnosis, severity assessment, and prognostication) and explore considerations for AI model development and testing, deployment infrastructure, clinical user interfaces, quality control, and institutional review board and regulatory approvals, with a practical focus on what a radiologist should consider when implementing an AI tool for COVID-19.

Prediction of arrhythmia susceptibility through mathematical modeling and machine learning.

At present, the QT interval on the electrocardiographic (ECG) waveform is the most common metric for assessing an individual's susceptibility to ventricular arrhythmias, with a long QT, or, at the cellular level, a long action potential duration (APD) considered high risk. However, the limitations of this simple approach have long been recognized. Here, we sought to improve prediction of arrhythmia susceptibility by combining mechanistic mathematical modeling with machine learning (ML). Simulations with a model of the ventricular myocyte were performed to develop a large heterogenous population of cardiomyocytes (n = 10,586), and we tested each variant's ability to withstand three arrhythmogenic triggers: 1) block of the rapid delayed rectifier potassium current (IKr Block), 2) augmentation of the L-type calcium current (ICaL Increase), and 3) injection of inward current (Current Injection). Eight ML algorithms were trained to predict, based on simulated AP features in preperturbed cells, whether each cell would develop arrhythmic dynamics in response to each trigger. We found that APD can accurately predict how cells respond to the simple Current Injection trigger but cannot effectively predict the response to IKr Block or ICaL Increase. ML predictive performance could be improved by incorporating additional AP features and simulations of additional experimental protocols. Importantly, we discovered that the most relevant features and experimental protocols were trigger specific, which shed light on the mechanisms that promoted arrhythmia formation in response to the triggers. Overall, our quantitative approach provides a means to understand and predict differences between individuals in arrhythmia susceptibility.

Artificial intelligence-enabled rapid diagnosis of COVID-19 patients.

For diagnosis of COVID-19, a SARS-CoV-2 virus-specific reverse transcriptase polymerase chain reaction (RT-PCR) test is routinely used. However, this test can take up to two days to complete, serial testing may be required to rule out the possibility of false negative results, and there is currently a shortage of RT-PCR test kits, underscoring the urgent need for alternative methods for rapid and accurate diagnosis of COVID-19 patients. Chest computed tomography (CT) is a valuable component in the evaluation of patients with suspected SARS-CoV-2 infection. Nevertheless, CT alone may have limited negative predictive value for ruling out SARS-CoV-2 infection, as some patients may have normal radiologic findings at early stages of the disease. In this study, we used artificial intelligence (AI) algorithms to integrate chest CT findings with clinical symptoms, exposure history, and laboratory testing to rapidly diagnose COVID-19 positive patients. Among a total of 905 patients tested by real-time RT-PCR assay and next-generation sequencing RT-PCR, 419 (46.3%) tested positive for SARS-CoV-2. In a test set of 279 patients, the AI system achieved an AUC of 0.92 and had equal sensitivity as compared to a senior thoracic radiologist. The AI system also improved the detection of RT-PCR positive COVID-19 patients who presented with normal CT scans, correctly identifying 17 of 25 (68%) patients, whereas radiologists classified all of these patients as COVID-19 negative. When CT scans and associated clinical history are available, the proposed AI system can help to rapidly diagnose COVID-19 patients.

Computed Tomography Features of Coronavirus Disease 2019 (COVID-19): A Review for Radiologists.

Coronavirus Disease 2019 (COVID-19) pneumonia has become a global pandemic. Although the rate of new infections in China has decreased, currently, 169 countries report confirmed cases, with many nations showing increasing numbers daily. Testing for COVID-19 infection is performed via reverse transcriptase polymerase chain reaction, but availability is limited in many parts of the world. The role of chest computed tomography is yet to be determined and may vary depending on the local prevalence of disease and availability of laboratory testing. A common but nonspecific pattern of disease with a somewhat predictable progression is seen in patients with COVID-19. Specifically, patchy ground-glass opacities in the periphery of the lower lungs may be present initially, eventually undergoing coalescence, consolidation, and organization, and ultimately showing features of fibrosis. In this article, we review the computed tomography features of COVID-19 infection. Familiarity with these findings and their evolution will help radiologists recognize potential COVID-19 and recognize the significant overlap with other causes of acute lung injury.

Artificial intelligence-enabled rapid diagnosis of patients with COVID-19.

For diagnosis of coronavirus disease 2019 (COVID-19), a SARS-CoV-2 virus-specific reverse transcriptase polymerase chain reaction (RT-PCR) test is routinely used. However, this test can take up to 2 d to complete, serial testing may be required to rule out the possibility of false negative results and there is currently a shortage of RT-PCR test kits, underscoring the urgent need for alternative methods for rapid and accurate diagnosis of patients with COVID-19. Chest computed tomography (CT) is a valuable component in the evaluation of patients with suspected SARS-CoV-2 infection. Nevertheless, CT alone may have limited negative predictive value for ruling out SARS-CoV-2 infection, as some patients may have normal radiological findings at early stages of the disease. In this study, we used artificial intelligence (AI) algorithms to integrate chest CT findings with clinical symptoms, exposure history and laboratory testing to rapidly diagnose patients who are positive for COVID-19. Among a total of 905 patients tested by real-time RT-PCR assay and next-generation sequencing RT-PCR, 419 (46.3%) tested positive for SARS-CoV-2. In a test set of 279 patients, the AI system achieved an area under the curve of 0.92 and had equal sensitivity as compared to a senior thoracic radiologist. The AI system also improved the detection of patients who were positive for COVID-19 via RT-PCR who presented with normal CT scans, correctly identifying 17 of 25 (68%) patients, whereas radiologists classified all of these patients as COVID-19 negative. When CT scans and associated clinical history are available, the proposed AI system can help to rapidly diagnose COVID-19 patients.

CT Features of Coronavirus Disease (COVID-19) in 30 Pediatric Patients.

OBJECTIVE. The purpose of this study is to characterize the CT findings of 30 children from mainland China who had laboratory-confirmed coronavirus disease (COVID-19). Although recent American College of Radiology recommendations assert that CT should not be used as a screening or diagnostic tool for patients with suspected COVID-19, radiologists should be familiar with the imaging appearance of this disease to identify its presence in patients undergoing CT for other reasons. MATERIALS AND METHODS. We retrospectively reviewed the CT findings and clinical symptoms of 30 pediatric patients with laboratory-confirmed COVID-19 who were seen at six centers in China from January 23, 2020, to February 8, 2020. Patient age ranged from 10 months to 18 years. Patients older than 18 years of age or those without chest CT examinations were excluded. Two cardiothoracic radiologists and a cardiothoracic imaging fellow characterized and scored the extent of lung involvement. Cohen kappa coefficient was used to calculate interobserver agreement between the readers. RESULTS. Among children, CT findings were often negative (77%). Positive CT findings seen in children included ground-glass opacities with a peripheral lung distribution, a crazy paving pattern, and the halo and reverse halo signs. There was a correlation between increasing age and increasing severity of findings, consistent with reported symptomatology in children. Eleven of 30 patients (37%) underwent follow-up chest CT, with 10 of 11 examinations (91%) showing no change, raising questions about the utility of CT in the diagnosis and management of COVID-19 in children. CONCLUSION. The present study describes the chest CT findings encountered in children with COVID-19 and questions the utility of CT in the diagnosis and management of pediatric patients.

Chest CT Findings in Coronavirus Disease-19 (COVID-19): Relationship to Duration of Infection.

In this retrospective study, chest CTs of 121 symptomatic patients infected with coronavirus disease-19 (COVID-19) from four centers in China from January 18, 2020 to February 2, 2020 were reviewed for common CT findings in relationship to the time between symptom onset and the initial CT scan (i.e. early, 0-2 days (36 patients), intermediate 3-5 days (33 patients), late 6-12 days (25 patients)). The hallmarks of COVID-19 infection on imaging were bilateral and peripheral ground-glass and consolidative pulmonary opacities. Notably, 20/36 (56%) of early patients had a normal CT. With a longer time after the onset of symptoms, CT findings were more frequent, including consolidation, bilateral and peripheral disease, greater total lung involvement, linear opacities, "crazy-paving" pattern and the "reverse halo" sign. Bilateral lung involvement was observed in 10/36 early patients (28%), 25/33 intermediate patients (76%), and 22/25 late patients (88%).

CT Imaging Features of 2019 Novel Coronavirus (2019-nCoV).

In this retrospective case series, chest CT scans of 21 symptomatic patients from China infected with the 2019 novel coronavirus (2019-nCoV) were reviewed, with emphasis on identifying and characterizing the most common findings. Typical CT findings included bilateral pulmonary parenchymal ground-glass and consolidative pulmonary opacities, sometimes with a rounded morphology and a peripheral lung distribution. Notably, lung cavitation, discrete pulmonary nodules, pleural effusions, and lymphadenopathy were absent. Follow-up imaging in a subset of patients during the study time window often demonstrated mild or moderate progression of disease, as manifested by increasing extent and density of lung opacities.

Predicting malignancy of pulmonary ground-glass nodules and their invasiveness by random forest.

BACKGROUND: The purpose of this study was to develop a predictive model that could accurately predict the malignancy of the pulmonary ground-glass nodules (GGNs) and the invasiveness of the malignant GGNs. METHODS: The authors built two binary classification models that could predict the malignancy of the pulmonary GGNs and the invasiveness of the malignant GGNs. RESULTS: Results of our developed model showed random forest could achieve 95.1% accuracy to predict the malignancy of GGNs and 83.0% accuracy to predict the invasiveness of the malignant GGNs. CONCLUSIONS: The malignancy and invasiveness of pulmonary GGNs could be predicted by random forest.