Lesion-aware attention network for diabetic nephropathy diagnosis with optical coherence tomography images.

Purpose: For early screening of diabetic nephropathy patients, we propose a deep learning algorithm to screen high-risk patients with diabetic nephropathy from retinal images of diabetic patients. Methods: We propose the use of attentional mechanisms to improve the model's focus on lesion-prone regions of retinal OCT images. First, the data is trained using the base network and the Grad-CAM algorithm locates image regions that have a large impact on the model output and generates a rough mask localization map. The mask is used as a auxiliary region to realize the auxiliary attention module. We then inserted the region-guided attention module into the baseline model and trained the CNN model to guide the model to better focus on relevant lesion features. The proposed model improves the recognition of the lesion region. Results: To evaluate the lesion-aware attention network, we trained and tested it using OCT volumetric data collected from 66 patients with diabetic retinal microangiopathy (89 eyes, male = 43, female = 23). There were 45 patients (60 eyes, male=27, female = 18) in DR group and 21 patients (29 eyes, male = 16, female = 5) in DN group. Our proposed model performs even better in disease classification, specifically, the accuracy of the proposed model was 91.68%, the sensitivity was 89.99%, and the specificity was 92.18%. Conclusion: The proposed lesion-aware attention model can provide reliable screening of high-risk patients with diabetic nephropathy.

Self-training adversarial learning for cross-domain retinal OCT fluid segmentation.

Accurate measurements of the size, shape and volume of macular edema can provide important biomarkers to jointly assess disease progression and treatment outcome. Although many deep learning-based segmentation algorithms have achieved remarkable success in semantic segmentation, these methods have difficulty obtaining satisfactory segmentation results in retinal optical coherence tomography (OCT) fluid segmentation tasks due to low contrast, blurred boundaries, and varied distribution. Moreover, directly applying a well-trained model on one device to test the images from other devices may cause the performance degradation in the joint analysis of multi-domain OCT images. In this paper, we propose a self-training adversarial learning framework for unsupervised domain adaptation in retinal OCT fluid segmentation tasks. Specifically, we develop an image style transfer module and a fine-grained feature transfer module to reduce discrepancies in the appearance and high-level features of images from different devices. Importantly, we transfer the target images to the appearance of source images to ensure that no image information of the source domain for supervised training is lost. To capture specific features of the target domain, we design a self-training module based on a discrepancy and similarity strategy to select the images with better segmentation results from the target domain and then introduce them into the source domain for the iterative training segmentation model. Extensive experiments on two challenging datasets demonstrate the effectiveness of our proposed method. In Particular, our proposed method achieves comparable results on cross-domain retinal OCT fluid segmentation compared with the state-of-the-art methods.

Exploiting Sparse Self-Representation and Particle Swarm Optimization for CNN Compression.

Structured pruning has received ever-increasing attention as a method for compressing convolutional neural networks. However, most existing methods directly prune the network structure according to the statistical information of the parameters. Besides, these methods differentiate the pruning rates only in each pruning stage or even use the same pruning rate across all layers, rather than using learnable parameters. In this article, we propose a network redundancy elimination approach guided by the pruned model. Our proposed method can easily tackle multiple architectures and is scalable to the deeper neural networks because of the use of joint optimization during the pruning procedure. More specifically, we first construct a sparse self-representation for the filters or neurons of the well-trained model, which is useful for analyzing the relationship among filters. Then, we employ particle swarm optimization to learn pruning rates in a layerwise manner according to the performance of the pruned model, which can determine optimal pruning rates with the best performance of the pruned model. Under this criterion, the proposed pruning approach can remove more parameters without undermining the performance of the model. Experimental results demonstrate the effectiveness of our proposed method on different datasets and different architectures. For example, it can reduce 58.1% FLOPs for ResNet50 on ImageNet with only a 1.6% top-five error increase and 44.1% FLOPs for FCN_ResNet50 on COCO2017 with a 3% error increase, outperforming most state-of-the-art methods.

Spatial-contextual variational autoencoder with attention correction for anomaly detection in retinal OCT images.

Anomaly detection refers to leveraging only normal data to train a model for identifying unseen abnormal cases, which is extensively studied in various fields. Most previous methods are based on reconstruction models, and use anomaly score calculated by the reconstruction error as the metric to tackle anomaly detection. However, these methods just employ single constraint on latent space to construct reconstruction model, resulting in limited performance in anomaly detection. To address this problem, we propose a Spatial-Contextual Variational Autoencoder with Attention Correction for anomaly detection in retinal OCT images. Specifically, we first propose a self-supervised segmentation network to extract retinal regions, which can effectively eliminate interference of background regions. Next, by introducing both multi-dimensional and one-dimensional latent space, our proposed framework can then learn the spatial and contextual manifolds of normal images, which is conducive to enlarging the difference between reconstruction errors of normal images and those of abnormal ones. Furthermore, an ablation-based method is proposed to localize anomalous regions by computing the importance of feature maps, which is used to correct anomaly score calculated by reconstruction error. Finally, a novel anomaly score is constructed to separate the abnormal images from the normal ones. Extensive experiments on two retinal OCT datasets are conducted to evaluate our proposed method, and the experimental results demonstrate the effectiveness of our approach.

Joint Metric Learning-Based Class-Specific Representation for Image Set Classification.

With the rapid advances in digital imaging and communication technologies, recently image set classification has attracted significant attention and has been widely used in many real-world scenarios. As an effective technology, the class-specific representation theory-based methods have demonstrated their superior performances. However, this type of methods either only uses one gallery set to measure the gallery-to-probe set distance or ignores the inner connection between different metrics, leading to the learned distance metric lacking robustness, and is sensitive to the size of image sets. In this article, we propose a novel joint metric learning-based class-specific representation framework (JMLC), which can jointly learn the related and unrelated metrics. By iteratively modeling probe set and related or unrelated gallery sets as affine hull, we reconstruct this hull sparsely or collaboratively over another image set. With the obtained representation coefficients, the combined metric between the query set and the gallery set can then be calculated. In addition, we also derive the kernel extension of JMLC and propose two new unrelated set constituting strategies. Specifically, kernelized JMLC (KJMLC) embeds the gallery sets and probe sets into the high-dimensional Hilbert space, and in the kernel space, the data become approximately linear separable. Extensive experiments on seven benchmark databases show the superiority of the proposed methods to the state-of-the-art image set classifiers.

DarkASDNet: Classification of ASD on Functional MRI Using Deep Neural Network.

Non-invasive whole-brain scans aid the diagnosis of neuropsychiatric disorder diseases such as autism, dementia, and brain cancer. The assessable analysis for autism spectrum disorders (ASD) is rationally challenging due to the limitations of publicly available datasets. For diagnostic or prognostic tools, functional Magnetic Resonance Imaging (fMRI) exposed affirmation to the biomarkers in neuroimaging research because of fMRI pickup inherent connectivity between the brain and regions. There are profound studies in ASD with introducing machine learning or deep learning methods that have manifested advanced steps for ASD predictions based on fMRI data. However, utmost antecedent models have an inadequacy in their capacity to manipulate performance metrics such as accuracy, precision, recall, and F1-score. To overcome these problems, we proposed an avant-garde DarkASDNet, which has the competence to extract features from a lower level to a higher level and bring out promising results. In this work, we considered 3D fMRI data to predict binary classification between ASD and typical control (TC). Firstly, we pre-processed the 3D fMRI data by adopting proper slice time correction and normalization. Then, we introduced a novel DarkASDNet which surpassed the benchmark accuracy for the classification of ASD. Our model's outcomes unveil that our proposed method established state-of-the-art accuracy of 94.70% to classify ASD vs. TC in ABIDE-I, NYU dataset. Finally, we contemplated our model by performing evaluation metrics including precision, recall, F1-score, ROC curve, and AUC score, and legitimize by distinguishing with recent literature descriptions to vindicate our outcomes. The proposed DarkASDNet architecture provides a novel benchmark approach for ASD classification using fMRI processed data.

Improving the Prediction of Benign or Malignant Breast Masses Using a Combination of Image Biomarkers and Clinical Parameters.

Background: Breast cancer is one of the leading causes of death in female cancer patients. The disease can be detected early using Mammography, an effective X-ray imaging technology. The most important step in mammography is the classification of mammogram patches as benign or malignant. Classically, benign or malignant breast tumors are diagnosed by radiologists' interpretation of mammograms based on clinical parameters. However, because masses are heterogeneous, clinical parameters supply limited information on mammography mass. Therefore, this study aimed to predict benign or malignant breast masses using a combination of image biomarkers and clinical parameters. Methods: We trained a deep learning (DL) fusion network of VGG16 and Inception-V3 network in 5,996 mammography images from the training cohort; DL features were extracted from the second fully connected layer of the DL fusion network. We then developed a combined model incorporating DL features, hand-crafted features, and clinical parameters to predict benign or malignant breast masses. The prediction performance was compared between clinical parameters and the combination of the above features. The strengths of the clinical model and the combined model were subsequently validated in a test cohort (n = 244) and an external validation cohort (n = 100), respectively. Results: Extracted features comprised 30 hand-crafted features, 27 DL features, and 5 clinical features (shape, margin type, breast composition, age, mass size). The model combining the three feature types yielded the best performance in predicting benign or malignant masses (AUC = 0.961) in the test cohort. A significant difference in the predictive performance between the combined model and the clinical model was observed in an independent external validation cohort (AUC: 0.973 vs. 0.911, p = 0.019). Conclusion: The prediction of benign or malignant breast masses improves when image biomarkers and clinical parameters are combined; the combined model was more robust than clinical parameters alone.

Adaptive-Guided-Coupling-Probability Level Set for Retinal Layer Segmentation.

Quantitative assessment of retinal layer thickness in spectral domain-optical coherence tomography (SD-OCT) images is vital for clinicians to determine the degree of ophthalmic lesions. However, due to the complex retinal tissues, high-level speckle noises and low intensity constraint, how to accurately recognize the retinal layer structure still remains a challenge. To overcome this problem, this paper proposes an adaptive-guided-coupling-probability level set method for retinal layer segmentation in SD-OCT images. Specifically, based on Bayes's theorem, each voxel probability representation is composed of two probability terms in our method. The first term is constructed as neighborhood Gaussian fitting distribution to characterize intensity information for each intra-retinal layer. The second one is boundary probability map generated by combining anatomical priors and adaptive thickness information to ensure surfaces evolve within a proper range. Then, the voxel probability representation is introduced into the proposed segmentation framework based on coupling probability level set to detect layer boundaries. A total of 1792 retinal B-scan images from 4 SD-OCT cubes in healthy eyes, 5 cubes in abnormal eyes with central serous chorioretinaopathy and 5 SD-OCT cubes in abnormal eyes with age-related macular disease are used to evaluate the proposed method. The experiment demonstrates that the segmentation results obtained by the proposed method have a good consistency with ground truth, and the proposed method outperforms six methods in the layer segmentation of uneven retinal SD-OCT images.

Double-branched and area-constraint fully convolutional networks for automated serous retinal detachment segmentation in SD-OCT images.

BACKGROUND AND OBJECTIVE: Quantitative assessment of subretinal fluid in spectral domain optical coherence tomography (SD-OCT) images is crucial for the diagnosis of central serous chorioretinopathy. For the subretinal fluid segmentation, the traditional methods need to segment retinal layers and then segment subretinal fluid. The layer segmentation has a high influence on subretinal fluid segmentation, so we aim to develop a deep learning model to segment subretinal fluid automatically without layer segmentation. METHODS: In this paper, we propose a novel image-to-image double-branched and area-constraint fully convolutional networks (DA-FCN) for segmenting subretinal fluid in SD-OCT images. Firstly, the dataset is extended by mirroring image, which helps to overcome the over-fitting problem in the training stage. Then, double-branched structures are designed to learn the shallow coarse and deep representations from the SD-OCT images. DA-FCN model is directly trained using the image and corresponding pixel-based ground truth. Finally, we introduce a novel supervision mechanism by jointing the area loss LA with the softmax loss LS to learn more representative features. RESULTS: The testing dataset with 52 SD-OCT volumes from 35 eyes of 35 patients is used for the evaluation of the proposed algorithm based on the cross-validation method. For the three criterions, including the true positive volume fraction, dice similarity coefficient, and positive predicative value, our method can obtain the results of (1) 94.3, 95.3, and 96.4 for dataset 1; (2) 97.3, 95.3, and 93.4 for dataset 2; (3) 93.0, 92.8, and 92.8 for dataset 3; (4) 89.7, 90.1, and 92.6 for dataset 4. CONCLUSION: In this work, we propose a novel fully convolutional network for the automatic segmentation of the subretinal fluid. By constructing the double branched structures and area constraint term, our method shows higher segmentation accuracy without layer segmentation compared with other methods.

A Megavoltage CT Image Enhancement Method for Image-Guided and Adaptive Helical TomoTherapy.

Purpose: To propose a novel method to improve the mega-voltage CT (MVCT) image quality for helical TomoTherapy while maintaining the stability on dose calculation. Materials and Methods: The Block-Matching 3D-transform (BM3D) and Discriminative Feature Representation (DFR) methods were combined into a novel BM3D + DFR method for their respective advantages. A phantom (Catphan504) and three serials of clinical (head & neck, chest, and pelvis) MVCT images from 30 patients were acquired using the helical TomoTherapy system. The contrast-to-noise ratio (CNR) and edge detection algorithm (canny) was employed for image quality comparisons between the original and BM3D + DFR enhanced MVCT. A simulated rectangular field of 6 MV X-ray beams were vertically delivered on the original and post-processed MVCT serials of the same CT density phantom, and the dose curves on both serials were compared to test the effects of image enhancement on dose calculation accuracy. Results: In total, 466 transversal MVCT slices were acquired and processed by both BM3D and the proposed BM3D + DFR methods. Compared to the original MVCT image, the BM3D + DFR method presented a remarkable improvement in terms of the soft tissue contrast and noise reduction. For the phantom image, the CNR of the region of interest (ROI) was improved from 1.70 to 4.03. The average CNR of ROIs for 10 patients from each anatomical group, were increased significantly from 1.45 ± 1.51 to 2.09 ± 1.68 for the head & neck (p < 0.001), from 0.92 ± 0.78 to 1.36 ± 0.85 for the chest (p < 0.001), and from 1.12 ± 1.22 to 1.76 ± 1.31 for the pelvis (p < 0.001), respectively. The canny edge detection operator showed that BM3D + DFR provided clearer organ boundaries with less chaos. The root-mean-square of the dosimetry difference on the iso-center passed horizontal dose profile curves and vertical percentage depth dose curves were only 0.09% and 0.06%, respectively. Conclusions: The proposed BM3D + DFR method is feasible to improve the soft tissue contrast for the original MVCT images with coincidence in dose calculation and without compromising resolution. After integration in clinical workflow, the post-processed MVCT may be better applied on image-guided and adaptive helical TomoTherapy.

Nonlinear system modelling via optimal design of neural trees.

This paper introduces a flexible neural tree model. The model is computed as a flexible multi-layer feed-forward neural network. A hybrid learning/evolutionary approach to automatically optimize the neural tree model is also proposed. The approach includes a modified probabilistic incremental program evolution algorithm (MPIPE) to evolve and determine a optimal structure of the neural tree and a parameter learning algorithm to optimize the free parameters embedded in the neural tree. The performance and effectiveness of the proposed method are evaluated using function approximation, time series prediction and system identification problems and compared with the related methods.