G2NPAN: GAN-guided nuance perceptual attention network for multimodal medical fusion image quality assessment.

Multimodal medical fusion images (MMFI) are formed by fusing medical images of two or more modalities with the aim of displaying as much valuable information as possible in a single image. However, due to the different strategies of various fusion algorithms, the quality of the generated fused images is uneven. Thus, an effective blind image quality assessment (BIQA) method is urgently required. The challenge of MMFI quality assessment is to enable the network to perceive the nuances between fused images of different qualities, and the key point for the success of BIQA is the availability of valid reference information. To this end, this work proposes a generative adversarial network (GAN) -guided nuance perceptual attention network (G2NPAN) to implement BIQA for MMFI. Specifically, we achieve the blind evaluation style via the design of a GAN and develop a Unique Feature Warehouse module to learn the effective features of fused images from the pixel level. The redesigned loss function guides the network to perceive the image quality. In the end, the class activation mapping supervised quality assessment network is employed to obtain the MMFI quality score. Extensive experiments and validation have been conducted in a database of medical fusion images, and the proposed method is superior to the state-of-the-art BIQA method.

TS-DSANN: Texture and shape focused dual-stream attention neural network for benign-malignant diagnosis of thyroid nodules in ultrasound images.

Recently, accurate diagnosis of thyroid nodules has played a critical role in precision medicine and healthcare system management. Due to complicated changes in ultrasound features of texture, and similar visual appearance of benign-malignant nodules, the identification of cancerous thyroid lesions from a given ultrasound image still faces challenges for even experienced radiologists. Learning-based computer-aided diagnosis (CAD) systems have accordingly attracted more and more attention recently. However, little research is committed to developing a deep learning-based CAD system that has greater conformity with radiologists' diagnostic decision-making. In this study, we devise a texture and shape focused dual-stream attention neural network, dubbed TS-DSANN. Specifically, in the texture focused stream, we utilize the ImageNet pre-trained ResNet34 to guide the network to recognize texture-related nodule attributes. Meanwhile, in the shape focused stream, in addition to using ResNet34 backbone, jointly learning from scratch with the contour obtained by contour detection module to enhance the extraction of shape features. Afterward, we employ a concatenation operation to aggregate the abovementioned two stream networks for capturing richer and more representative features. Finally, we further utilize an online class activation mapping mechanism to assist the dual-stream network in generating a localization heatmap to obtain more visualization attention to the nodule from the whole image, and supervise classifier's attention in decision-making. Experimental results conducted on the two-center thyroid nodule ultrasound datasets verify that our proposed method has improved the classification performance, superior to the state-of-the-art methods.

Medical image fusion quality assessment based on conditional generative adversarial network.

Multimodal medical image fusion (MMIF) has been proven to effectively improve the efficiency of disease diagnosis and treatment. However, few works have explored dedicated evaluation methods for MMIF. This paper proposes a novel quality assessment method for MMIF based on the conditional generative adversarial networks. First, with the mean opinion scores (MOS) as the guiding condition, the feature information of the two source images is extracted separately through the dual channel encoder-decoder. The features of different levels in the encoder-decoder are hierarchically input into the self-attention feature block, which is a fusion strategy for self-identifying favorable features. Then, the discriminator is used to improve the fusion objective of the generator. Finally, we calculate the structural similarity index between the fake image and the true image, and the MOS corresponding to the maximum result will be used as the final assessment result of the fused image quality. Based on the established MMIF database, the proposed method achieves the state-of-the-art performance among the comparison methods, with excellent agreement with subjective evaluations, indicating that the method is effective in the quality assessment of medical fusion images.

Longitudinal evaluation for COVID-19 chest CT disease progression based on Tchebichef moments.

Blur is a key property in the perception of COVID-19 computed tomography (CT) image manifestations. Typically, blur causes edge extension, which brings shape changes in infection regions. Tchebichef moments (TM) have been verified efficiently in shape representation. Intuitively, disease progression of same patient over time during the treatment is represented as different blur degrees of infection regions, since different blur degrees cause the magnitudes change of TM on infection regions image, blur of infection regions can be captured by TM. With the above observation, a longitudinal objective quantitative evaluation method for COVID-19 disease progression based on TM is proposed. COVID-19 disease progression CT image database (COVID-19 DPID) is built to employ radiologist subjective ratings and manual contouring, which can test and compare disease progression on the CT images acquired from the same patient over time. Then the images are preprocessed, including lung automatic segmentation, longitudinal registration, slice fusion, and a fused slice image with region of interest (ROI) is obtained. Next, the gradient of a fused ROI image is calculated to represent the shape. The gradient image of fused ROI is separated into same size blocks, a block energy is calculated as quadratic sum of non-direct current moment values. Finally, the objective assessment score is obtained by TM energy-normalized applying block variances. We have conducted experiment on COVID-19 DPID and the experiment results indicate that our proposed metric supplies a satisfactory correlation with subjective evaluation scores, demonstrating effectiveness in the quantitative evaluation for COVID-19 disease progression.