Navigating the nuances: comparative analysis and hyperparameter optimisation of neural architectures on contrast-enhanced MRI for liver and liver tumour segmentation.

In medical imaging, accurate segmentation is crucial to improving diagnosis, treatment, or both. However, navigating the multitude of available architectures for automatic segmentation can be overwhelming, making it challenging to determine the appropriate type of architecture and tune the most crucial parameters during dataset optimisation. To address this problem, we examined and refined seven distinct architectures for segmenting the liver, as well as liver tumours, with a restricted training collection of 60 3D contrast-enhanced magnetic resonance images (CE-MRI) from the ATLAS dataset. Included in these architectures are convolutional neural networks (CNNs), transformers, and hybrid CNN/transformer architectures. Bayesian search techniques were used for hyperparameter tuning to hasten convergence to the optimal parameter mixes while also minimising the number of trained models. It was unexpected that hybrid models, which typically exhibit superior performance on larger datasets, would exhibit comparable performance to CNNs. The optimisation of parameters contributed to better segmentations, resulting in an average increase of 1.7% and 5.0% in liver and tumour segmentation Dice coefficients, respectively. In conclusion, the findings of this study indicate that hybrid CNN/transformer architectures may serve as a practical substitute for CNNs even in small datasets. This underscores the significance of hyperparameter optimisation.

Registration of preoperative temporal bone CT-scan to otoendoscopic video for augmented-reality based on convolutional neural networks.

PURPOSE: Patient-to-image registration is a preliminary step required in surgical navigation based on preoperative images. Human intervention and fiducial markers hamper this task as they are time-consuming and introduce potential errors. We aimed to develop a fully automatic 2D registration system for augmented reality in ear surgery. METHODS: CT-scans and corresponding oto-endoscopic videos were collected from 41 patients (58 ears) undergoing ear examination (vestibular schwannoma before surgery, profound hearing loss requiring cochlear implant, suspicion of perilymphatic fistula, contralateral ears in cases of unilateral chronic otitis media). Two to four images were selected from each case. For the training phase, data from patients (75% of the dataset) and 11 cadaveric specimens were used. Tympanic membranes and malleus handles were contoured on both video images and CT-scans by expert surgeons. The algorithm used a U-Net network for detecting the contours of the tympanic membrane and the malleus on both preoperative CT-scans and endoscopic video frames. Then, contours were processed and registered through an iterative closest point algorithm. Validation was performed on 4 cases and testing on 6 cases. Registration error was measured by overlaying both images and measuring the average and Hausdorff distances. RESULTS: The proposed registration method yielded a precision compatible with ear surgery with a 2D mean overlay error of 0.65 ± 0.60 mm for the incus and 0.48 ± 0.32 mm for the round window. The average Hausdorff distance for these 2 targets was 0.98 ± 0.60 mm and 0.78 ± 0.34 mm respectively. An outlier case with higher errors (2.3 mm and 1.5 mm average Hausdorff distance for incus and round window respectively) was observed in relation to a high discrepancy between the projection angle of the reconstructed CT-scan and the video image. The maximum duration for the overall process was 18 s. CONCLUSIONS: A fully automatic 2D registration method based on a convolutional neural network and applied to ear surgery was developed. The method did not rely on any external fiducial markers nor human intervention for landmark recognition. The method was fast and its precision was compatible with ear surgery.

Image-to-Patient Registration in Computer-Assisted Surgery of Head and Neck: State-of-the-Art, Perspectives, and Challenges.

Today, image-guided systems play a significant role in improving the outcome of diagnostic and therapeutic interventions. They provide crucial anatomical information during the procedure to decrease the size and the extent of the approach, to reduce intraoperative complications, and to increase accuracy, repeatability, and safety. Image-to-patient registration is the first step in image-guided procedures. It establishes a correspondence between the patient's preoperative imaging and the intraoperative data. When it comes to the head-and-neck region, the presence of many sensitive structures such as the central nervous system or the neurosensory organs requires a millimetric precision. This review allows evaluating the characteristics and the performances of different registration methods in the head-and-neck region used in the operation room from the perspectives of accuracy, invasiveness, and processing times. Our work led to the conclusion that invasive marker-based methods are still considered as the gold standard of image-to-patient registration. The surface-based methods are recommended for faster procedures and applied on the surface tissues especially around the eyes. In the near future, computer vision technology is expected to enhance these systems by reducing human errors and cognitive load in the operating room.

Segmentation of 4D Flow MRI: Comparison between 3D Deep Learning and Velocity-Based Level Sets.

A thoracic aortic aneurysm is an abnormal dilatation of the aorta that can progress and lead to rupture. The decision to conduct surgery is made by considering the maximum diameter, but it is now well known that this metric alone is not completely reliable. The advent of 4D flow magnetic resonance imaging has allowed for the calculation of new biomarkers for the study of aortic diseases, such as wall shear stress. However, the calculation of these biomarkers requires the precise segmentation of the aorta during all phases of the cardiac cycle. The objective of this work was to compare two different methods for automatically segmenting the thoracic aorta in the systolic phase using 4D flow MRI. The first method is based on a level set framework and uses the velocity field in addition to 3D phase contrast magnetic resonance imaging. The second method is a U-Net-like approach that is only applied to magnitude images from 4D flow MRI. The used dataset was composed of 36 exams from different patients, with ground truth data for the systolic phase of the cardiac cycle. The comparison was performed based on selected metrics, such as the Dice similarity coefficient (DSC) and Hausdorf distance (HD), for the whole aorta and also three aortic regions. Wall shear stress was also assessed and the maximum wall shear stress values were used for comparison. The U-Net-based approach provided statistically better results for the 3D segmentation of the aorta, with a DSC of 0.92 ± 0.02 vs. 0.86 ± 0.5 and an HD of 21.49 ± 24.8 mm vs. 35.79 ± 31.33 mm for the whole aorta. The absolute difference between the wall shear stress and ground truth slightly favored the level set method, but not significantly (0.754 ± 1.07 Pa vs. 0.737 ± 0.79 Pa). The results showed that the deep learning-based method should be considered for the segmentation of all time steps in order to evaluate biomarkers based on 4D flow MRI.

Segmentation of the aorta in systolic phase from 4D flow MRI: multi-atlas vs. deep learning.

OBJECTIVE: In the management of the aortic aneurysm, 4D flow magnetic resonance Imaging provides valuable information for the computation of new biomarkers using computational fluid dynamics (CFD). However, accurate segmentation of the aorta is required. Thus, our objective is to evaluate the performance of two automatic segmentation methods on the calculation of aortic wall pressure. METHODS: Automatic segmentation of the aorta was performed with methods based on deep learning and multi-atlas using the systolic phase in the 4D flow MRI magnitude image of 36 patients. Using mesh morphing, isotopological meshes were generated, and CFD was performed to calculate the aortic wall pressure. Node-to-node comparisons of the pressure results were made to identify the most robust automatic method respect to the pressures obtained with a manually segmented model. RESULTS: Deep learning approach presented the best segmentation performance with a mean Dice similarity coefficient and a mean Hausdorff distance (HD) equal to 0.92+/- 0.02 and 21.02+/- 24.20 mm, respectively. At the global level HD is affected by the performance in the abdominal aorta. Locally, this distance decreases to 9.41+/- 3.45 and 5.82+/- 6.23 for the ascending and descending thoracic aorta, respectively. Moreover, with respect to the pressures from the manual segmentations, the differences in the pressures computed from deep learning were lower than those computed from multi-atlas method. CONCLUSION: To reduce biases in the calculation of aortic wall pressure, accurate segmentation is needed, particularly in regions with high blood flow velocities. Thus, the deep learning segmen-tation method should be preferred.

Comparison of In-Vivo and Ex-Vivo Ascending Aorta Elastic Properties through Automatic Deep Learning Segmentation of Cine-MRI and Biomechanical Testing.

Ascending aortic aneurysm is a pathology that is important to be supervised and treated. During the years the aorta dilates, it becomes stiff, and its elastic properties decrease. In some cases, the aortic wall can rupture leading to aortic dissection with a high mortality rate. The main reference standard to measure when the patient needs to undertake surgery is the aortic diameter. However, the aortic diameter was shown not to be sufficient to predict aortic dissection, implying other characteristics should be considered. Therefore, the main objective of this work is to assess in-vivo the elastic properties of four different quadrants of the ascending aorta and compare the results with equivalent properties obtained ex-vivo. The database consists of 73 cine-MRI sequences of thoracic aorta acquired in axial orientation at the level of the pulmonary trunk. All the patients have dilated aorta and surgery is required. The exams were acquired just prior to surgery, each consisting of 30 slices on average across the cardiac cycle. Multiple deep learning architectures have been explored with different hyperparameters and settings to automatically segment the contour of the aorta on each image and then automatically calculate the aortic compliance. A semantic segmentation U-Net network outperforms the rest explored networks with a Dice score of 98.09% (±0.96%) and a Hausdorff distance of 4.88 mm (±1.70 mm). Local aortic compliance and local aortic wall strain were calculated from the segmented surfaces for each quadrant and then compared with elastic properties obtained ex-vivo. Good agreement was observed between Young's modulus and in-vivo strain. Our results suggest that the lateral and posterior quadrants are the stiffest. In contrast, the medial and anterior quadrants have the lowest aortic stiffness. The in-vivo stiffness tendency agrees with the values obtained ex-vivo. We can conclude that our automatic segmentation method is robust and compatible with clinical practice (thanks to a graphical user interface), while the in-vivo elastic properties are reliable and compatible with the ex-vivo ones.

4D segmentation of the thoracic aorta from 4D flow MRI using deep learning.

BACKGROUND: 4D flow MRI allows the analysis of hemodynamic changes in the aorta caused by pathologies such as thoracic aortic aneurysms (TAA). For personalized management of TAA, new biomarkers are required to analyze the effect of fluid structure iteration which can be obtained from 4D flow MRI. However, the generation of these biomarkers requires prior 4D segmentation of the aorta. OBJECTIVE: To develop an automatic deep learning model to segment the aorta in 4D from 4D flow MRI. METHODS: Segmentation is addressed with a U-Net based segmentation model that treats each 4D flow MRI frame as an independent sample. Performance is measured with respect to Dice score (DS) and Hausdorff distance (HD). In addition, the maximum and minimum surface areas at the level of the ascending aorta are measured and compared with those obtained from cine-MRI. RESULTS: The segmentation performance was 0.90 ± 0.02 for the DS and the mean HD was 9.58 ± 4.36 mm. A correlation coefficient of r = 0.85 was obtained for the maximum surface and r = 0.86 for the minimum surface between the 4D flow MRI and cine-MRI. CONCLUSION: The proposed automatic approach of 4D aortic segmentation from 4D flow MRI seems to be accurate enough to contribute to the wider use of this imaging technique in the analysis of pathologies such as TAA.

LU-Net: A Multistage Attention Network to Improve the Robustness of Segmentation of Left Ventricular Structures in 2-D Echocardiography.

Segmentation of cardiac structures is one of the fundamental steps to estimate volumetric indices of the heart. This step is still performed semiautomatically in clinical routine and is, thus, prone to interobserver and intraobserver variabilities. Recent studies have shown that deep learning has the potential to perform fully automatic segmentation. However, the current best solutions still suffer from a lack of robustness in terms of accuracy and number of outliers. The goal of this work is to introduce a novel network designed to improve the overall segmentation accuracy of left ventricular structures (endocardial and epicardial borders) while enhancing the estimation of the corresponding clinical indices and reducing the number of outliers. This network is based on a multistage framework where both the localization and segmentation steps are optimized jointly through an end-to-end scheme. Results obtained on a large open access data set show that our method outperforms the current best-performing deep learning solution with a lighter architecture and achieved an overall segmentation accuracy lower than the intraobserver variability for the epicardial border (i.e., on average a mean absolute error of 1.5 mm and a Hausdorff distance of 5.1mm) with 11% of outliers. Moreover, we demonstrate that our method can closely reproduce the expert analysis for the end-diastolic and end-systolic left ventricular volumes, with a mean correlation of 0.96 and a mean absolute error of 7.6 ml. Concerning the ejection fraction of the left ventricle, results are more contrasted with a mean correlation coefficient of 0.83 and an absolute mean error of 5.0%, producing scores that are slightly below the intraobserver margin. Based on this observation, areas for improvement are suggested.

Real-Time Automatic Ejection Fraction and Foreshortening Detection Using Deep Learning.

Volume and ejection fraction (EF) measurements of the left ventricle (LV) in 2-D echocardiography are associated with a high uncertainty not only due to interobserver variability of the manual measurement, but also due to ultrasound acquisition errors such as apical foreshortening. In this work, a real-time and fully automated EF measurement and foreshortening detection method is proposed. The method uses several deep learning components, such as view classification, cardiac cycle timing, segmentation and landmark extraction, to measure the amount of foreshortening, LV volume, and EF. A data set of 500 patients from an outpatient clinic was used to train the deep neural networks, while a separate data set of 100 patients from another clinic was used for evaluation, where LV volume and EF were measured by an expert using clinical protocols and software. A quantitative analysis using 3-D ultrasound showed that EF is considerably affected by apical foreshortening, and that the proposed method can detect and quantify the amount of apical foreshortening. The bias and standard deviation of the automatic EF measurements were -3.6 ± 8.1%, while the mean absolute difference was measured at 7.2% which are all within the interobserver variability and comparable with related studies. The proposed real-time pipeline allows for a continuous acquisition and measurement workflow without user interaction, and has the potential to significantly reduce the time spent on the analysis and measurement error due to foreshortening, while providing quantitative volume measurements in the everyday echo lab.

Deep Learning for Segmentation Using an Open Large-Scale Dataset in 2D Echocardiography.

Delineation of the cardiac structures from 2D echocardiographic images is a common clinical task to establish a diagnosis. Over the past decades, the automation of this task has been the subject of intense research. In this paper, we evaluate how far the state-of-the-art encoder-decoder deep convolutional neural network methods can go at assessing 2D echocardiographic images, i.e., segmenting cardiac structures and estimating clinical indices, on a dataset, especially, designed to answer this objective. We, therefore, introduce the cardiac acquisitions for multi-structure ultrasound segmentation dataset, the largest publicly-available and fully-annotated dataset for the purpose of echocardiographic assessment. The dataset contains two and four-chamber acquisitions from 500 patients with reference measurements from one cardiologist on the full dataset and from three cardiologists on a fold of 50 patients. Results show that encoder-decoder-based architectures outperform state-of-the-art non-deep learning methods and faithfully reproduce the expert analysis for the end-diastolic and end-systolic left ventricular volumes, with a mean correlation of 0.95 and an absolute mean error of 9.5 ml. Concerning the ejection fraction of the left ventricle, results are more contrasted with a mean correlation coefficient of 0.80 and an absolute mean error of 5.6%. Although these results are below the inter-observer scores, they remain slightly worse than the intra-observer's ones. Based on this observation, areas for improvement are defined, which open the door for accurate and fully-automatic analysis of 2D echocardiographic images.