The Big Bang of Deep Learning in Ultrasound-Guided Surgery: A Review.

Ultrasound (US) imaging is a paramount modality in many image-guided surgeries and percutaneous interventions, thanks to its high portability, temporal resolution, and cost-efficiency. However, due to its imaging principles, the US is often noisy and difficult to interpret. Appropriate image processing can greatly enhance the applicability of the imaging modality in clinical practice. Compared with the classic iterative optimization and machine learning (ML) approach, deep learning (DL) algorithms have shown great performance in terms of accuracy and efficiency for US processing. In this work, we conduct a comprehensive review on deep-learning algorithms in the applications of US-guided interventions, summarize the current trends, and suggest future directions on the topic.

DiffeoRaptor: diffeomorphic inter-modal image registration using RaPTOR.

PURPOSE: Diffeomorphic image registration is essential in many medical imaging applications. Several registration algorithms of such type have been proposed, but primarily for intra-contrast alignment. Currently, efficient inter-modal/contrast diffeomorphic registration, which is vital in numerous applications, remains a challenging task. METHODS: We proposed a novel inter-modal/contrast registration algorithm that leverages Robust PaTch-based cOrrelation Ratio metric to allow inter-modal/contrast image alignment and bandlimited geodesic shooting demonstrated in Fourier-Approximated Lie Algebras (FLASH) algorithm for fast diffeomorphic registration. RESULTS: The proposed algorithm, named DiffeoRaptor, was validated with three public databases for the tasks of brain and abdominal image registration while comparing the results against three state-of-the-art techniques, including FLASH, NiftyReg, and Symmetric image Normalization (SyN). CONCLUSIONS: Our results demonstrated that DiffeoRaptor offered comparable or better registration performance in terms of registration accuracy. Moreover, DiffeoRaptor produces smoother deformations than SyN in inter-modal and contrast registration. The code for DiffeoRaptor is publicly available at https://github.com/nimamasoumi/DiffeoRaptor .

Multimodal 3D ultrasound and CT in image-guided spinal surgery: public database and new registration algorithms.

PURPOSE: Accurate multimodal registration of intraoperative ultrasound (US) and preoperative computed tomography (CT) is a challenging problem. Construction of public datasets of US and CT images can accelerate the development of such image registration techniques. This can help ensure the accuracy and safety of spinal surgeries using image-guided surgery systems where an image registration is employed. In addition, we present two algorithms to register US and CT images. METHODS: We present three different datasets of vertebrae with corresponding CT, US, and simulated US images. For each of the two latter datasets, we also provide 16 landmark pairs of matching structures between the CT and US images and performed fiducial registration to acquire a silver standard for assessing image registration. Besides, we proposed two patch-based rigid image registration algorithms, one based on normalized cross-correlation (NCC) and the other based on correlation ratio (CR) to register misaligned CT and US images. RESULTS: The CT and corresponding US images of the proposed database were pre-processed and misaligned with different error intervals, resulting in 6000 registration problems solved using both NCC and CR methods. Our results show that the methods were successful in aligning the pre-processed CT and US images by decreasing the warping index. CONCLUSIONS: The database provides a resource for evaluating image registration techniques. The simulated data have two applications. First, they provide the gold standard ground-truth which is difficult to obtain with ex vivo and in vivo data for validating US-CT registration methods. Second, the simulated US images can be used to validate real-time US simulation methods. Besides, the proposed image registration techniques can be useful for developing methods in clinical application.

ARENA: Inter-modality affine registration using evolutionary strategy.

PURPOSE: Image fusion of different imaging modalities renders valuable information to clinicians. In this paper, we proposed an automatic multimodal registration method to register intra-operative ultrasound images (US) to preoperative magnetic resonance images (MRI) in the context of image-guided neurosurgery. METHODS: We employed refined correlation ratio as a similarity metric for our intensity-based image registration method. We deem MRI as the fixed image ([Formula: see text]) and US as the moving image ([Formula: see text]) and then transform [Formula: see text] to align with [Formula: see text]. We utilized the covariance matrix adaptation evolutionary strategy to find the optimal affine transformation in registration of [Formula: see text] to [Formula: see text]. RESULTS: We applied our method on the publicly available retrospective evaluation of cerebral tumors (RESECT) database and Montreal Neurological Institute's brain images of tumors for evaluation (BITE) database. We validated the results qualitatively and quantitatively. Qualitative validation is conducted (by the three authors) through overlaying pre- and post-registration US and MRI to allow visual assessment of the alignment. Quantitative validation is performed by utilizing the corresponding landmarks in the databases for the preoperative MRI and the intra-operative US. Average mean target registration error (mTRE) has been reduced from [Formula: see text] to [Formula: see text] in 22 patients in the RESECT database and from [Formula: see text] to [Formula: see text] in the BITE database. A nonparametric statistical analysis performed using the Wilcoxon rank sum test shows that there is a significant difference between pre- and post-registration mTREs with a p value of [Formula: see text] for the RESECT database and [Formula: see text] for the BITE database. CONCLUSIONS: The proposed fully automatic registration method significantly improved the alignment of MRI and US images and can therefore be used to reduce the misalignment of US and MRI caused by brain shift, calibration errors, and patient to MRI transformation matrix.