Robust Deformable Image Registration Using Cycle-Consistent Implicit Representations.

Recent works in medical image registration have proposed the use of Implicit Neural Representations, demonstrating performance that rivals state-of-the-art learning-based methods. However, these implicit representations need to be optimized for each new image pair, which is a stochastic process that may fail to converge to a global minimum. To improve robustness, we propose a deformable registration method using pairs of cycle-consistent Implicit Neural Representations: each implicit representation is linked to a second implicit representation that estimates the opposite transformation, causing each network to act as a regularizer for its paired opposite. During inference, we generate multiple deformation estimates by numerically inverting the paired backward transformation and evaluating the consensus of the optimized pair. This consensus improves registration accuracy over using a single representation and results in a robust uncertainty metric that can be used for automatic quality control. We evaluate our method with a 4D lung CT dataset. The proposed cycle-consistent optimization method reduces the optimization failure rate from 2.4% to 0.0% compared to the current state-of-the-art. The proposed inference method improves landmark accuracy by 4.5% and the proposed uncertainty metric detects all instances where the registration method fails to converge to a correct solution. We verify the generalizability of these results to other data using a centerline propagation task in abdominal 4D MRI, where our method achieves a 46% improvement in propagation consistency compared with single-INR registration and demonstrates a strong correlation between the proposed uncertainty metric and registration accuracy.

Knowledge distillation with ensembles of convolutional neural networks for medical image segmentation.

Purpose: Ensembles of convolutional neural networks (CNNs) often outperform a single CNN in medical image segmentation tasks, but inference is computationally more expensive and makes ensembles unattractive for some applications. We compared the performance of differently constructed ensembles with the performance of CNNs derived from these ensembles using knowledge distillation, a technique for reducing the footprint of large models such as ensembles. Approach: We investigated two different types of ensembles, namely, diverse ensembles of networks with three different architectures and two different loss-functions, and uniform ensembles of networks with the same architecture but initialized with different random seeds. For each ensemble, additionally, a single student network was trained to mimic the class probabilities predicted by the teacher model, the ensemble. We evaluated the performance of each network, the ensembles, and the corresponding distilled networks across three different publicly available datasets. These included chest computed tomography scans with four annotated organs of interest, brain magnetic resonance imaging (MRI) with six annotated brain structures, and cardiac cine-MRI with three annotated heart structures. Results: Both uniform and diverse ensembles obtained better results than any of the individual networks in the ensemble. Furthermore, applying knowledge distillation resulted in a single network that was smaller and faster without compromising performance compared with the ensemble it learned from. The distilled networks significantly outperformed the same network trained with reference segmentation instead of knowledge distillation. Conclusion: Knowledge distillation can compress segmentation ensembles of uniform or diverse composition into a single CNN while maintaining the performance of the ensemble.

Untangling and segmenting the small intestine in 3D cine-MRI using deep learning.

Cine-MRI of the abdomen is a non-invasive imaging technique allowing assessment of small intestinal motility. This is valuable for the evaluation of gastrointestinal disorders. While 2D cine-MRI is increasingly used for this purpose in both clinical practice and in research settings, the potential of 3D cine-MRI has been largely underexplored. In the absence of image analysis tools enabling investigation of the intestines as 3D structures, the assessment of motility in 3D cine-images is generally limited to the evaluation of movement in separate 2D slices. Furthermore, while a segmentation map of the small intestine would be required for a number of automatic analysis tasks, deep learning based segmentation of the small intestine generally performs poorly due to the large variety in shapes, sizes and locations in the abdomen among different patients. Using a data set of 3D cine-MRI scans from 14 healthy volunteers, we developed a multi-task method that automatically tracks individual segments of the small intestine in a time-point from 3D cine-MRI scans, using a stochastic tracker built on top of a CNN-based orientation classifier. The method additionally performs segmentation, conditioned on the locations of intestinal centerlines. We demonstrate the benefit of our stochastic tracking strategy and we show that our proposed segmentation method performs significantly better than an identical network without centerline conditioning. Furthermore, we assess the robustness of the method through evaluation on a set of patients with severe bowel disease. In terms of centerline tracking, our method achieves a recall of 0.74±0.07, a precision of 0.80±0.06 and an F1 score of 0.77±0.05 in the set of healthy volunteers. In the set of patients, it achieves a recall of 0.76±0.12, a precision of 0.86±0.11 and an F1 score of 0.80±0.08. Segmentation achieves a Dice coefficient of 0.88±0.03 in the set of healthy volunteers and 0.79±0.09 in the set of patients. By extracting a structural representation of the small intestine, the presented method provides a major first step towards automatic detailed quantitative assessment of small intestinal motility in abdominal 3D cine-MRI.