Understanding skin color bias in deep learning-based skin lesion segmentation.

BACKGROUND: The field of dermatological image analysis using deep neural networks includes the semantic segmentation of skin lesions, pivotal for lesion analysis, pathology inference, and diagnoses. While biases in neural network-based dermatoscopic image classification against darker skin tones due to dataset imbalance and contrast disparities are acknowledged, a comprehensive exploration of skin color bias in lesion segmentation models is lacking. It is imperative to address and understand the biases in these models. METHODS: Our study comprehensively evaluates skin tone bias within prevalent neural networks for skin lesion segmentation. Since no information about skin color exists in widely used datasets, to quantify the bias we use three distinct skin color estimation methods: Fitzpatrick skin type estimation, Individual Typology Angle estimation as well as manual grouping of images by skin color. We assess bias across common models by training a variety of U-Net-based models on three widely-used datasets with 1758 different dermoscopic and clinical images. We also evaluate commonly suggested methods to mitigate bias. RESULTS: Our findings expose a significant and large correlation between segmentation performance and skin color, revealing consistent challenges in segmenting lesions for darker skin tones across diverse datasets. Using various methods of skin color quantification, we have found significant bias in skin lesion segmentation against darker-skinned individuals when evaluated both in and out-of-sample. We also find that commonly used methods for bias mitigation do not result in any significant reduction in bias. CONCLUSIONS: Our findings suggest a pervasive bias in most published lesion segmentation methods, given our use of commonly employed neural network architectures and publicly available datasets. In light of our findings, we propose recommendations for unbiased dataset collection, labeling, and model development. This presents the first comprehensive evaluation of fairness in skin lesion segmentation.

Segment-then-Segment: Context-Preserving Crop-Based Segmentation for Large Biomedical Images.

Medical images are often of huge size, which presents a challenge in terms of memory requirements when training machine learning models. Commonly, the images are downsampled to overcome this challenge, but this leads to a loss of information. We present a general approach for training semantic segmentation neural networks on much smaller input sizes called Segment-then-Segment. To reduce the input size, we use image crops instead of downscaling. One neural network performs the initial segmentation on a downscaled image. This segmentation is then used to take the most salient crops of the full-resolution image with the surrounding context. Each crop is segmented using a second specially trained neural network. The segmentation masks of each crop are joined to form the final output image. We evaluate our approach on multiple medical image modalities (microscopy, colonoscopy, and CT) and show that this approach greatly improves segmentation performance with small network input sizes when compared to baseline models trained on downscaled images, especially in terms of pixel-wise recall.

Overview of the Whole Heart and Heart Chamber Segmentation Methods.

BACKGROUND: Preservation and improvement of heart and vessel health is the primary motivation behind cardiovascular disease (CVD) research. Development of advanced imaging techniques can improve our understanding of disease physiology and serve as a monitor for disease progression. Various image processing approaches have been proposed to extract parameters of cardiac shape and function from different cardiac imaging modalities with an overall intention of providing full cardiac analysis. Due to differences in image modalities, the selection of an appropriate segmentation algorithm may be a challenging task. PURPOSE: This paper presents a comprehensive and critical overview of research on the whole heart, bi-ventricles and left atrium segmentation methods from computed tomography (CT), magnetic resonance (MRI) and echocardiography (echo) imaging. The paper aims to: (1) summarize the considerable challenges of cardiac image segmentation, (2) provide the comparison of the segmentation methods, (3) classify significant contributions in the field and (4) critically review approaches in terms of their performance and accuracy. CONCLUSION: The methods described are classified based on the used segmentation approach into (1) edge-based segmentation methods, (2) model-fitting segmentation methods and (3) machine and deep learning segmentation methods and are further split based on the targeted cardiac structure. Edge-based methods are mostly developed as semi-automatic and allow end-user interaction, which provides physicians with extra control over the final segmentation. Model-fitting methods are very robust and resistant to the high variability in image contrast and overall image quality. Nevertheless, they are often time-consuming and require appropriate models with prior knowledge. While the emerging deep learning segmentation approaches provide unprecedented performance in some specific scenarios and under the appropriate training, their performance highly depends on the data quality and the amount and the accuracy of provided annotations.

Left atrial appendage segmentation from 3D CCTA images for occluder placement procedure.

BACKGROUND: Percutaneous left atrial appendage (LAA) closure (placement of an occluder to close the appendage) is a novel procedure for stroke prevention in patients suffering from atrial fibrillation. The closure procedure planning requires accurate LAA measurements which can be obtained from computed tomography (CT) images. METHOD: We propose a novel semi-automatic LAA segmentation method from 3D coronary CT angiography (CCTA) images. The method segments the LAA, proposes the location for the occluder placement (a delineation plane between the left atrium and LAA) and calculates measurements needed for closure procedure planning. The method requires only two inputs from the user: a threshold value and a single seed point inside the LAA. Proposed location of the delineation plane can be intuitively corrected if necessary. Measurements are calculated from the segmented LAA according to the final delineation plane. RESULTS: Performance of the proposed method is validated on 17 CCTA images, manually segmented by two medical doctors. We achieve the average dice coefficient overlap of 92.52% and 91.63% against the ground truth segmentations. The average dice coefficient overlap between the two ground truth segmentations is 92.66%. Our proposed LAA orifice localization is evaluated against the desired location of the LAA orifice determined by the expert. The average distance between our proposed location and the desired location is 2.51 mm. CONCLUSION: Segmentation results show high correspondence to the ground truth segmentations. The occluder placement method shows high accuracy which indicates potential in clinical procedure planning.