Application of multimodal weakly-supervised learning in image synthesis and segmentation of liver cancer / 中国医学物理学杂志

Although it has high resolution for soft tissues,magnetic resonance imaging(MRI)is not the standard for chest imaging,which results in an insufficient amount of expert-annotated MRI data.Therefore,CT image is usually converted into MRI image.To overcome the difficulty of obtaining the corresponding modal CT and MRI images,a CSCGAN model with CycleGAN as the framework is proposed based on the structural characteristics of generative adversarial networks.Considering the possibility of mode collapse in CycleGAN,StyleGan2 which can control the style and feature details of the synthetic image and realize the synthesis of high-resolution images is integrated into CycleGAN for reconstructing the generator.A noise module is introduced to reduce external interference.In addition,in order to prevent the loss of tumors during conversion,the discriminator structure of the network is modified,and a mixed attention mechanism is added.Experimental results show that compared with the images generated by other methods,those generated by the proposed model are improved in Dice similarity coefficient,Hausdorff distance,volume ratio and mean intersection over union,indicating that the proposed method can effectively realize the mode conversion of liver tumor images,and that the generated data can improve the segmentation accuracy.

Research on the improvement of CBCT image quality based on region-discriminative generative adversarial networks in radiotherapy for cervical cancer / 中国医学装备

Objective:To propose a model that could improve image quality of cone-beam computed tomography(CBCT),which based on region-discriminative generative adversarial networks(GAN),in radiotherapy for cervical cancer,so as to meet the requirements of self-adaptive radiotherapy for image quality.Methods:We employed a region-discriminative strategy and a generative adversarial networks idea to construct a model of improving CBCT image quality that could focus on local details of the images of radiotherapy for cervical cancer,which discriminator could improve the quality of generating local details of images.This model of image quality was applied to CBCT images of radiotherapy for cervical cancer.And then,the effects of processing image were evaluated through quantitative indicators and visualization.Results:Both texture clarity and contrast were significantly enhanced after CBCT image quality was improved.The signal to noise ratio of peak value of images was increased by 47.2%,and the indicator of similarity of structure was enhanced to>0.838.Compared with other model,both visualization and indicators can appear better efficiency of model.Compared with Unet network and CycleGAN network,the similarities of structure were respectively increased by 11.88% and 19.54%,and the signal to noise ratios were respectively increased by 19.75% and 25.99%.Conclusion:The GAN bases on region-discrimination can significantly improve the quality of generating integral and detailed CBCT image of radiotherapy for cervical cancer,which can provide new technical pathway for image quality of CBCT with low dose,and can play an important role for improving safety and effectiveness of radiotherapy.It has importantly clinical value for formulating and executing radiotherapy plan.

Research progress on medical image dataset expansion methods / 生物医学工程学杂志

Computer-aided diagnosis (CAD) systems play a very important role in modern medical diagnosis and treatment systems, but their performance is limited by training samples. However, the training samples are affected by factors such as imaging cost, labeling cost and involving patient privacy, resulting in insufficient diversity of training images and difficulty in data obtaining. Therefore, how to efficiently and cost-effectively augment existing medical image datasets has become a research hotspot. In this paper, the research progress on medical image dataset expansion methods is reviewed based on relevant literatures at home and abroad. First, the expansion methods based on geometric transformation and generative adversarial networks are compared and analyzed, and then improvement of the augmentation methods based on generative adversarial networks are emphasized. Finally, some urgent problems in the field of medical image dataset expansion are discussed and the future development trend is prospected.

Construction of an epileptic seizure prediction model using a semi-supervised method of generative adversarial and long short term memory network combined with Stockwell transform / 南方医科大学学报

OBJECTIVE@#To propose a semi-supervised epileptic seizure prediction model (ST-WGAN-GP-Bi-LSTM) to enhance the prediction performance by improving time-frequency analysis of electroencephalogram (EEG) signals, enhancing the stability of the unsupervised feature learning model and improving the design of back-end classifier.@*METHODS@#Stockwell transform (ST) of the epileptic EEG signals was performed to locate the time-frequency information by adaptive adjustment of the resolution and retaining the absolute phase to obtain the time-frequency inputs. When there was no overlap between the generated data distribution and the real EEG data distribution, to avoid failure of feature learning due to a constant JS divergence, Wasserstein GAN was used as a feature learning model, and the cost function based on EM distance and gradient penalty strategy was adopted to constrain the unsupervised training process to allow the generation of a high-order feature extractor. A temporal prediction model was finally constructed based on a bi-directional long short term memory network (Bi-LSTM), and the classification performance was improved by obtaining the temporal correlation between high-order time-frequency features. The CHB-MIT scalp EEG dataset was used to validate the proposed patient-specific seizure prediction method.@*RESULTS@#The AUC, sensitivity, and specificity of the proposed method reached 90.40%, 83.62%, and 86.69%, respectively. Compared with the existing semi-supervised methods, the propose method improved the original performance by 17.77%, 15.41%, and 53.66%. The performance of this method was comparable to that of a supervised prediction model based on CNN.@*CONCLUSION@#The utilization of ST, WGAN-GP, and Bi-LSTM effectively improves the prediction performance of the semi-supervised deep learning model, which can be used for optimization of unsupervised feature extraction in epileptic seizure prediction.

CT and MRI fusion based on generative adversarial network and convolutional neural networks under image enhancement / 生物医学工程学杂志

Aiming at the problems of missing important features, inconspicuous details and unclear textures in the fusion of multimodal medical images, this paper proposes a method of computed tomography (CT) image and magnetic resonance imaging (MRI) image fusion using generative adversarial network (GAN) and convolutional neural network (CNN) under image enhancement. The generator aimed at high-frequency feature images and used double discriminators to target the fusion images after inverse transform; Then high-frequency feature images were fused by trained GAN model, and low-frequency feature images were fused by CNN pre-training model based on transfer learning. Experimental results showed that, compared with the current advanced fusion algorithm, the proposed method had more abundant texture details and clearer contour edge information in subjective representation. In the evaluation of objective indicators, Q AB/F, information entropy (IE), spatial frequency (SF), structural similarity (SSIM), mutual information (MI) and visual information fidelity for fusion (VIFF) were 2.0%, 6.3%, 7.0%, 5.5%, 9.0% and 3.3% higher than the best test results, respectively. The fused image can be effectively applied to medical diagnosis to further improve the diagnostic efficiency.

An image classification method for arrhythmias based on Gramian angular summation field and improved Inception-ResNet-v2 / 生物医学工程学杂志

Arrhythmia is a significant cardiovascular disease that poses a threat to human health, and its primary diagnosis relies on electrocardiogram (ECG). Implementing computer technology to achieve automatic classification of arrhythmia can effectively avoid human error, improve diagnostic efficiency, and reduce costs. However, most automatic arrhythmia classification algorithms focus on one-dimensional temporal signals, which lack robustness. Therefore, this study proposed an arrhythmia image classification method based on Gramian angular summation field (GASF) and an improved Inception-ResNet-v2 network. Firstly, the data was preprocessed using variational mode decomposition, and data augmentation was performed using a deep convolutional generative adversarial network. Then, GASF was used to transform one-dimensional ECG signals into two-dimensional images, and an improved Inception-ResNet-v2 network was utilized to implement the five arrhythmia classifications recommended by the AAMI (N, V, S, F, and Q). The experimental results on the MIT-BIH Arrhythmia Database showed that the proposed method achieved an overall classification accuracy of 99.52% and 95.48% under the intra-patient and inter-patient paradigms, respectively. The arrhythmia classification performance of the improved Inception-ResNet-v2 network in this study outperforms other methods, providing a new approach for deep learning-based automatic arrhythmia classification.

Application of generative adversarial network in magnetic resonance image reconstruction / 生物医学工程学杂志

Magnetic resonance imaging (MRI) is an important medical imaging method, whose major limitation is its long scan time due to the imaging mechanism, increasing patients' cost and waiting time for the examination. Currently, parallel imaging (PI) and compress sensing (CS) together with other reconstruction technologies have been proposed to accelerate image acquisition. However, the image quality of PI and CS depends on the image reconstruction algorithms, which is far from satisfying in respect to both the image quality and the reconstruction speed. In recent years, image reconstruction based on generative adversarial network (GAN) has become a research hotspot in the field of magnetic resonance imaging because of its excellent performance. In this review, we summarized the recent development of application of GAN in MRI reconstruction in both single- and multi-modality acceleration, hoping to provide a useful reference for interested researchers. In addition, we analyzed the characteristics and limitations of existing technologies and forecasted some development trends in this field.

Research on segmentation method of brain vessel based on deep learning / 医疗卫生装备

Objective To propose a deep learning-based cerebrovascular segmentation method to solve the problems of magnetic resonance angiography(MRA)image auto segmentation due to some tiny or overlapped vessels.Methods Generative adversarial networks(GAN)consisting of multiple generators and discriminators were used to construct a brain vessel segmen-tation model(BVSM).Firstly,the feature fusion and attention mechanism modules were introduced into the generator network to segment and extract the patient's MRA images;secondly,the discriminator network judged the gap between the brain vessel segmentation results respectively by the generator network and the expert's manual operation,so as to optimize the generator network continuously to obtain realistic segmentation images;finally,the MIDAS dataset was used to design ablation experi-ments to compare the cerebrovascular segmentation results of BVSM with the original model(RVGAN retinal vascular gene-rative adversarial network model),the RVGAN+Attention model incorporated with the attention module and the RVGAN+slice-level feature aggregation(SFA)model with the SFA module in terms of Dice coefficient,accuracy,sensitivity and AUC.Results The BVSM behaved better than the RVGAN,RVGAN+Attention and RVGAN+SFA models with Dice coefficient being 87.2％,accuracy being 88.3％,sensitivity being 86.3％and AUC being 0.942.Conclusion The method proposed facilitates the observation of cerebrovascular structure with high accuracy,and provides an auxiliary means for diagnosing cerebrovascular diseases.[Chinese Medical Equipment Journal,2023,44(9):1-7]

Brain age prediction method based on deep convolutional generative adversarial network / 医疗卫生装备

Objective To propose a brain age prediction method based on deep convolutional generative adversarial networks(DCGAN)for objective assessment of brain health status.Methods The DCGAN model was extended from 2D to 3D and improved by integrating the concept of residual block to enhance the ability for feature extraction.The classifiers were pre-trained with unsupervised adversarial learning and fine-tuned with migration learning to eliminate the overfitting of 3D convolutional neural network(CNN)due to small sample size.To verify the effectiveness of the improved model,comparison analyses based on UK Biobank(UKB)database were carried out between the improved model and least absolute shrinkage and selection operator(LASSO)model,machine learning model,3D CNN model and graph convolutional network model by using mean absolute error(MAE)as the evaluation metric.Results The model proposed gained advantages over LASSO model,machine learning model,3D CNN model and graph convolutional network model in predicting brain age with a MAE error of 2.896 years.Conclusion The method proposed behaves well for large-scale datasets,which can predict brain age accurately and assess brain health status objectively.

Postoperative corneal topography generation based on attention mechanism and Pix2Pix network / 国际眼科杂志(Guoji Yanke Zazhi)

AIM:To explore the use of attention mechanism and Pix2Pix generative adversarial network to predict the postoperative corneal topography of age-related cataract patients undergone femtosecond laser arcuate keratotomy.METHODS:In this retrospective case series study, the 210 preoperative and postoperative corneal topographies from 87 age-related cataract patients(105 eyes)undergoing femtosecond laser arcuate keratotomy at Shanxi Eye Hospital between March 2018 and March 2020 were selected and divided into a training set(180)and a test set(30)for model training and testing. The peak signal-to-noise ratio(PSNR), structural similarity(SSIM)and Alpins astigmatism vector analysis were used to compare the accuracy of postoperative corneal topography prediction under different attention mechanisms.RESULTS:The model based on attention mechanism and Pix2Pix network can predict postoperative corneal topography, among which the model based on Self-Attention mechanism has the best prediction effect, with PSNR and SSIM reaching 16.048 and 0.7661, respectively. There were no statistically significant differences in the difference vector, difference vector axis position, surgically induced astigmatism, and correction index between real and generated corneal topography on the 3mm and 5mm rings(all P>0.05).CONCLUSION:Based on the Self-Attention mechanism and Pix2Pix network, the postoperative corneal topography can be well predicted, which can provide reference for the surgical planning and postoperative effects of ophthalmic clinicians.

Electrocardiogram data recognition algorithm based on variable scale fusion network model / 生物医学工程学杂志

The judgment of the type of arrhythmia is the key to the prevention and diagnosis of early cardiovascular disease. Therefore, electrocardiogram (ECG) analysis has been widely used as an important basis for doctors to diagnose. However, due to the large differences in ECG signal morphology among different patients and the unbalanced distribution of categories, the existing automatic detection algorithms for arrhythmias have certain difficulties in the identification process. This paper designs a variable scale fusion network model for automatic recognition of heart rhythm types. In this study, a variable-scale fusion network model was proposed for automatic identification of heart rhythm types. The improved ECG generation network (EGAN) module was used to solve the imbalance of ECG data, and the ECG signal was reproduced in two dimensions in the form of gray recurrence plot (GRP) and spectrogram. Combined with the branching structure of the model, the automatic classification of variable-length heart beats was realized. The results of the study were verified by the Massachusetts institute of technology and Beth Israel hospital (MIT-BIH) arrhythmia database, which distinguished eight heart rhythm types. The average accuracy rate reached 99.36%, and the sensitivity and specificity were 96.11% and 99.84%, respectively. In conclusion, it is expected that this method can be used for clinical auxiliary diagnosis and smart wearable devices in the future.

Preliminary study on motion artifacts removal of coronary CT angiography using generative adversarial network / 上海交通大学学报（医学版）

Objective: To investigate the ability of generative adversarial network (GAN) to remove motion artifacts in coronary CT angiography (CTA) images. Methods: Subjects who underwent single-cardiac-cycle multi-phase CTA were included and divided into training and test group. The middle segment of the right coronary artery (RCA) was investigated because its motion artifact is the most prominent among all coronary branches. The patch image of the vessel with motion artifacts was extracted, and paired images without artifacts were considered as reference. The GAN model was established according to the training group. In the test group, vessel images were segmented out of the surrounding tissues by using ITK-SNAP software, including the vessel with artifacts, GAN-generated images and reference images. The Dice coefficients of the vessel with artifacts vs reference image (dice1) and GAN-generated images vs reference image (dice2) were cal-culated. By comparing the difference between dice1 and dice2, GAN's ability in removing motion artifacts was evaluated. Results: Ninety subjects were included. Seventy-one (11 000 images) were randomly selected as the training group, and the other 19 (3 006 images) were as the test group. Based on subjects, dice1 and dice2 of the middle segment of RCA were 0.38±0.19 and 0.50±0.23, re-spectively (P=0.006). Based on images, the values of the middle segment of RCA were 0.38±0.20 and 0.51±0.26, respectively (P=0.000). Conclusion: GAN can significantly reduce the motion artifacts of CTA in the middle segment of RCA and has the potential to act as a new method to remove motion artifacts of coronary CTA images.

Cross-modal retrieval method for thyroid ultrasound image and text based on generative adversarial network / 生物医学工程学杂志

Ultrasonic examination is a common method in thyroid examination, and the results are mainly composed of thyroid ultrasound images and text reports. Implementation of cross modal retrieval method of images and text reports can provide great convenience for doctors and patients, but currently there is no retrieval method to correlate thyroid ultrasound images with text reports. This paper proposes a cross-modal method based on the deep learning and improved cross-modal generative adversarial network: ①the weight sharing constraints between the fully connection layers used to construct the public representation space in the original network are changed to cosine similarity constraints, so that the network can better learn the common representation of different modal data; ②the fully connection layer is added before the cross-modal discriminator to merge the full connection layer of image and text in the original network with weight sharing. Semantic regularization is realized on the basis of inheriting the advantages of the original network weight sharing. The experimental results show that the mean average precision of cross modal retrieval method for thyroid ultrasound image and text report in this paper can reach 0.508, which is significantly higher than the traditional cross-modal method, providing a new method for cross-modal retrieval of thyroid ultrasound image and text report.

Super-resolution construction of intravascular ultrasound images using generative adversarial networks / 南方医科大学学报

The low-resolution ultrasound images have poor visual effects. Herein we propose a method for generating clearer intravascular ultrasound images based on super-resolution reconstruction combined with generative adversarial networks. We used the generative adversarial networks to generate the images by a generator and to estimate the authenticity of the images by a discriminator. Specifically, the low-resolution image was passed through the sub-pixel convolution layer -feature channels to generate -feature maps in the same size, followed by realignment of the corresponding pixels in each feature map into × sub-blocks, which corresponded to the sub-block in a high-resolution image; after amplification, an image with a -time resolution was generated. The generative adversarial networks can obtain a clearer image through continuous optimization. We compared the method (SRGAN) with other methods including Bicubic, super-resolution convolutional network (SRCNN) and efficient sub-pixel convolutional network (ESPCN), and the proposed method resulted in obvious improvements in the peak signal-to-noise ratio (PSNR) by 2.369 dB and in structural similarity index by 1.79% to enhance the diagnostic visual effects of intravascular ultrasound images.