Artificial intelligence with deep learning in nuclear medicine and radiology.

The use of deep learning in medical imaging has increased rapidly over the past few years, finding applications throughout the entire radiology pipeline, from improved scanner performance to automatic disease detection and diagnosis. These advancements have resulted in a wide variety of deep learning approaches being developed, solving unique challenges for various imaging modalities. This paper provides a review on these developments from a technical point of view, categorizing the different methodologies and summarizing their implementation. We provide an introduction to the design of neural networks and their training procedure, after which we take an extended look at their uses in medical imaging. We cover the different sections of the radiology pipeline, highlighting some influential works and discussing the merits and limitations of deep learning approaches compared to other traditional methods. As such, this review is intended to provide a broad yet concise overview for the interested reader, facilitating adoption and interdisciplinary research of deep learning in the field of medical imaging.

High-resolution monolithic LYSO detector with 6-layer depth-of-interaction for clinical PET.

The system spatial resolution of whole-body positron emission tomography (PET) is limited to around 2 mm due to positron physics and the large diameter of the bore. To stay below this 'physics'-limit a scintillation detector with an intrinsic spatial resolution of around 1.3 mm is needed. Currently used detector technology consists of arrays of 2.6-5 mm segmented scintillator pixels which are the dominant factor contributing to the system resolution. Pixelated detectors using smaller pixels exist but face major drawbacks in sensitivity, timing, energy resolution and cost. Monolithic continuous detectors, where the spatial resolution is determined by the shape of the light distribution on the photodetector array, are a promising alternative. Without having the drawbacks of pixelated detectors, monolithic ones can also provide depth-of-interaction (DOI) information. In this work we present a monolithic detector design aiming to serve high-resolution clinical PET systems while maintaining high sensitivity. A 50 × 50 × 16 mm3Lutetium-Yttrium oxyorthosilicate scintillation crystal with silicon photomultiplier (SiPM) backside readout is calibrated in singles mode by a collimated beam obtaining a reference dataset for the event positioning. A mean nearest neighbour (MNN) algorithm and an artificial neural network for positioning are compared. The targeted intrinsic detector resolution of 1.3 mm needed to reach a 2 mm resolution on system level was accomplished with both algorithms. The neural network achieved a mean spatial resolution of 1.14 mm FWHM for the whole detector and 1.02 mm in the centre (30 × 30 mm2). The MNN algorithm performed slightly worse with 1.17 mm for the whole detector and 1.13 mm in the centre. The intrinsic DOI information will also result in uniform system spatial resolution over the full field of view.

Artificial neural networks for positioning of gamma interactions in monolithic PET detectors.

To detect gamma rays with good spatial, timing and energy resolution while maintaining high sensitivity we need accurate and efficient algorithms to estimate the first gamma interaction position from the measured light distribution. Furthermore, monolithic detectors are investigated as an alternative to pixelated detectors due to increased sensitivity, resolution and intrinsic DOI encoding. Monolithic detectors, however, are challenging because of complicated calibration setups and edge effects. In this work, we evaluate the use of neural networks to estimate the 3D first (Compton or photoelectric) interaction position. Using optical simulation data of a 50 × 50 × 16 mm3LYSO crystal, performance is evaluated as a function of network complexity (two to five hidden layers with 64 to 1024 neurons) and amount of training data (1000-8000 training events per calibration position). We identify and address the potential pitfall of overfitting on the training grid through evaluation on intermediate positions that are not in the training set. Additionally, the performance of neural networks is directly compared with nearest neighbour positioning. Optimal performance was achieved with a network containing three hidden layers of 256 neurons trained on 1000 events/position. For more complex networks, the performance degrades at intermediate positions and overfitting starts to occur. A median 3D positioning error of 0.77 mm and a 2D FWHM of 0.46 mm is obtained. This is a 17% improvement in terms of FWHM compared to the nearest neighbour algorithm. Evaluation only on events that are not Compton scattered results in a 3D positioning error of 0.40 mm and 2D FWHM of 0.42 mm. This reveals that Compton scatter results in a considerable increase of 93% in positioning error. This study demonstrates that very good spatial resolutions can be achieved with neural networks, superior to nearest neighbour positioning. However, potential overfitting on the training grid should be carefully evaluated.

Automated MRI based pipeline for segmentation and prediction of grade, IDH mutation and 1p19q co-deletion in glioma.

In the WHO glioma classification guidelines grade (glioblastoma versus lower-grade glioma), IDH mutation and 1p/19q co-deletion status play a central role as they are important markers for prognosis and optimal therapy planning. Currently, diagnosis requires invasive surgical procedures. Therefore, we propose an automatic segmentation and classification pipeline based on routinely acquired pre-operative MRI (T1, T1 postcontrast, T2 and/or FLAIR). A 3D U-Net was designed for segmentation and trained on the BraTS 2019 training dataset. After segmentation, the 3D tumor region of interest is extracted from the MRI and fed into a CNN to simultaneously predict grade, IDH mutation and 1p19q co-deletion. Multi-task learning allowed to handle missing labels and train one network on a large dataset of 628 patients, collected from The Cancer Imaging Archive and BraTS databases. Additionally, the network was validated on an independent dataset of 110 patients retrospectively acquired at the Ghent University Hospital (GUH). Segmentation performance calculated on the BraTS validation set shows an average whole tumor dice score of 90% and increased robustness to missing image modalities by randomly excluding input MRI during training. Classification area under the curve scores are 93%, 94% and 82% on the TCIA test data and 94%, 86% and 87% on the GUH data for grade, IDH and 1p19q status respectively. We developed a fast, automatic pipeline to segment glioma and accurately predict important (molecular) markers based on pre-therapy MRI.