Impact of color augmentation and tissue type in deep learning for hematoxylin and eosin image super resolution.

Single image super-resolution is an important computer vision task with applications including remote sensing, medical imaging, and surveillance. Modern work on super-resolution utilizes deep learning to synthesize high resolution (HR) images from low resolution images (LR). With the increased utilization of digitized whole slide images (WSI) in pathology workflows, digital pathology has emerged as a promising domain for super-resolution. Despite extensive existing research into super-resolution, there remain challenges specific to digital pathology. Here, we investigated image augmentation techniques for hematoxylin and eosin (H&E) WSI super-resolution and model generalizability across diverse tissue types. In addition, we investigated shortcomings with common quality metrics (peak signal-to-noise ratio (PSNR), structure similarity index (SSIM)) by conducting a perceptual quality survey for super-resolved pathology images. High performing deep super-resolution models were used to generate 20X HR images from LR images (5X or 10X equivalent) for 11 different tissues and 30 human evaluators were asked to score the quality of the generated versus the ground truth 20X HR images. The scores given by a human rater and the PSNR or the SSIM were compared to investigate the correlation between model training parameters. We found that models trained on multiple tissues generalized better than those trained on a single tissue type. We also found that PSNR correlated with perceptual quality (R = 0.26) less accurately than did SSIM (R = 0.64), suggesting that the SSIM quality metric is insufficient. The methods proposed in this study can be used to virtually magnify H&E images with better perceptual quality than interpolation methods (i.e., bicubic interpolation) commonly implemented in digital pathology software. The impact of deep SISR methods is more notable when scaling to 4X is needed, such as in the case of super-resolving a low magnification WSI from 10X to 40X.

Texture based skin lesion abruptness quantification to detect malignancy.

BACKGROUND: Abruptness of pigment patterns at the periphery of a skin lesion is one of the most important dermoscopic features for detection of malignancy. In current clinical setting, abrupt cutoff of a skin lesion determined by an examination of a dermatologist. This process is subjective, nonquantitative, and error-prone. We present an improved computational model to quantitatively measure abruptness of a skin lesion over our previous method. To achieve this, we quantitatively analyze the texture features of a region within the lesion boundary. This region is bounded by an interior border line of the lesion boundary which is determined using level set propagation (LSP) method. This method provides a fast border contraction without a need for extensive boolean operations. Then, we build feature vectors of homogeneity, standard deviation of pixel values, and mean of the pixel values of the region between the contracted border and the original border. These vectors are then classified using neural networks (NN) and SVM classifiers. RESULTS: As lower homogeneity indicates sharp cutoffs, suggesting melanoma, we carried out our experiments on two dermoscopy image datasets, which consist of 800 benign and 200 malignant melanoma cases. LSP method helped produce better results than Kaya et al., 2016 study. By using texture homogeneity at the periphery of a lesion border determined by LSP, as a classification results, we obtained 87% f1-score and 78% specificity; that we obtained better results than in the previous study. We also compared the performances of two different NN classifiers and support vector machine classifier. The best results obtained using combination of RGB color spaces with the fully-connected multi-hidden layer NN. CONCLUSIONS: Computational results also show that skin lesion abrupt cutoff is a reliable indicator of malignancy. Results show that computational model of texture homogeneity along the periphery of skin lesion borders based on LSP is an effective way of quantitatively measuring abrupt cutoff of a lesion.

Abrupt skin lesion border cutoff measurement for malignancy detection in dermoscopy images.

BACKGROUND: Automated skin lesion border examination and analysis techniques have become an important field of research for distinguishing malignant pigmented lesions from benign lesions. An abrupt pigment pattern cutoff at the periphery of a skin lesion is one of the most important dermoscopic features for detection of neoplastic behavior. In current clinical setting, the lesion is divided into a virtual pie with eight sections. Each section is examined by a dermatologist for abrupt cutoff and scored accordingly, which can be tedious and subjective. METHODS: This study introduces a novel approach to objectively quantify abruptness of pigment patterns along the lesion periphery. In the proposed approach, first, the skin lesion border is detected by the density based lesion border detection method. Second, the detected border is gradually scaled through vector operations. Then, along gradually scaled borders, pigment pattern homogeneities are calculated at different scales. Through this process, statistical texture features are extracted. Moreover, different color spaces are examined for the efficacy of texture analysis. RESULTS: The proposed method has been tested and validated on 100 (31 melanoma, 69 benign) dermoscopy images. Analyzed results indicate that proposed method is efficient on malignancy detection. More specifically, we obtained specificity of 0.96 and sensitivity of 0.86 for malignancy detection in a certain color space. The F-measure, harmonic mean of recall and precision, of the framework is reported as 0.87. CONCLUSIONS: The use of texture homogeneity along the periphery of the lesion border is an effective method to detect malignancy of the skin lesion in dermoscopy images. Among different color spaces tested, RGB color space's blue color channel is the most informative color channel to detect malignancy for skin lesions. That is followed by YCbCr color spaces Cr channel, and Cr is closely followed by the green color channel of RGB color space.

Quantification of food intake using food image analysis.

Measuring free-living peoples' food intake represents methodological and technical challenges. The Remote Food Photography Method (RFPM) involves participants capturing pictures of their food selection and plate waste and sending these pictures to the research center via a wireless network, where they are analyzed by Registered Dietitians to estimate food intake. Initial tests indicate that the RFPM is reliable and valid, though the efficiency of the method is limited due to the reliance on human raters to estimate food intake. Herein, we describe the development of a semi-automated computer imaging application to estimate food intake based on pictures captured by participants.