An Organ-aware Diagnosis Framework for Radiology Report Generation.

Radiology report generation (RRG) is crucial to save the valuable time of radiologists in drafting the report, therefore increasing their work efficiency. Compared to typical methods that directly transfer image captioning technologies to RRG, our approach incorporates organ-wise priors into the report generation. Specifically, in this paper, we propose Organ-aware Diagnosis (OaD) to generate diagnostic reports containing descriptions of each physiological organ. During training, we first develop a task distillation (TD) module to extract organ-level descriptions from reports. We then introduce an organ-aware report generation module that, for one thing, provides a specific description for each organ, and for another, simulates clinical situations to provide short descriptions for normal cases. Furthermore, we design an auto-balance mask loss to ensure balanced training for normal/abnormal descriptions and various organs simultaneously. Being intuitively reasonable and practically simple, our OaD outperforms SOTA alternatives by large margins on commonly used IU-Xray and MIMIC-CXR datasets, as evidenced by a 3.4% BLEU-1 improvement on MIMIC-CXR and 2.0% BLEU-2 improvement on IU-Xray.

Self-architectural knowledge distillation for spiking neural networks.

Spiking neural networks (SNNs) have attracted attention due to their biological plausibility and the potential for low-energy applications on neuromorphic hardware. Two mainstream approaches are commonly used to obtain SNNs, i.e., ANN-to-SNN conversion methods, and Directly-trained-SNN methods. However, the former achieve excellent performance at the cost of a large number of time steps (i.e., latency), while the latter exhibit lower latency but suffers from suboptimal performance. To tackle the performance-latency trade-off, we propose Self-Architectural Knowledge Distillation (SAKD), an intuitive and effective method for SNNs leveraging Knowledge Distillation (KD). We adopt a bilevel teacher-student training strategy in SAKD, i.e., level-1 involves directly transferring same-architectural pre-trained ANN weights to SNNs, and level-2 encourages the SNNs to mimic ANN's behavior, considering both final responses and intermediate features aspects. Learning with informative supervision signals fostered by labels and ANNs, our SAKD achieves new state-of-the-art (SOTA) performance with a few time steps on widely-used classification benchmark datasets. On ImageNet-1K, with only 4 time steps, our Spiking-ResNet34 model attains a Top-1 accuracy of 70.04%, outperforming the previous same-architectural SOTA methods. Notably, our SEW-ResNet152 model reaches a Top-1 accuracy of 77.30% on ImageNet-1K, setting a new SOTA benchmark for SNNs. Furthermore, we apply our SAKD to various dense prediction downstream tasks, such as object detection and semantic segmentation, demonstrating strong generalization ability and superior performance. In conclusion, our proposed SAKD framework presents a promising approach for achieving both high performance and low latency in SNNs, potentially paving the way for future advancements in the field.

MADAv2: Advanced Multi-Anchor Based Active Domain Adaptation Segmentation.

Unsupervised domain adaption has been widely adopted in tasks with scarce annotated data. Unfortunately, mapping the target-domain distribution to the source-domain unconditionally may distort the essential structural information of the target-domain data, leading to inferior performance. To address this issue, we first propose to introduce active sample selection to assist domain adaptation regarding the semantic segmentation task. By innovatively adopting multiple anchors instead of a single centroid, both source and target domains can be better characterized as multimodal distributions, in which way more complementary and informative samples are selected from the target domain. With only a little workload to manually annotate these active samples, the distortion of the target-domain distribution can be effectively alleviated, achieving a large performance gain. In addition, a powerful semi-supervised domain adaptation strategy is proposed to alleviate the long-tail distribution problem and further improve the segmentation performance. Extensive experiments are conducted on public datasets, and the results demonstrate that the proposed approach outperforms state-of-the-art methods by large margins and achieves similar performance to the fully-supervised upperbound, i.e., 71.4% mIoU on GTA5 and 71.8% mIoU on SYNTHIA. The effectiveness of each component is also verified by thorough ablation studies.

Automatic quantification of retinal photoreceptor integrity to predict persistent disease activity in neovascular age-related macular degeneration using deep learning.

Purpose: Using deep learning (DL)-based technique, we identify risk factors and create a prediction model for refractory neovascular age-related macular degeneration (nAMD) characterized by persistent disease activity (PDA) in spectral domain optical coherence tomography (SD-OCT) images. Materials and methods: A total of 671 typical B-scans were collected from 186 eyes of 186 patients with nAMD. Spectral domain optical coherence tomography images were analyzed using a classification convolutional neural network (CNN) and a fully convolutional network (FCN) algorithm to extract six features involved in nAMD, including ellipsoid zone (EZ), external limiting membrane (ELM), intraretinal fluid (IRF), subretinal fluid (SRF), pigment epithelium detachment (PED), and subretinal hyperreflective material (SHRM). Random forest models were probed to predict 1-year disease activity (stable, PDA, and cured) based on the quantitative features computed from automated segmentation and evaluated with cross-validation. Results: The algorithm to segment six SD-OCT features achieved the mean accuracy of 0.930 (95% CI: 0.916-0.943), dice coefficients of 0.873 (95% CI: 0.847-0.899), a sensitivity of 0.873 (95% CI: 0.844-0.910), and a specificity of 0.922 (95% CI: 0.905-0.940). The six-metric model including EZ and ELM achieved the optimal performance to predict 1-year disease activity, with an area under the receiver operating characteristic (ROC) curve (AUC) of 0.980, the accuracy of 0.930, the sensitivity of 0.920, and the specificity of 0.962. The integrity of EZ and ELM significantly improved the performance of the six-metric model than that of the four-metric model. Conclusion: The prediction model reveals the potential to predict PDA in nAMD eyes. The integrity of EZ and ELM constituted the strongest predictive factor for PDA in nAMD eyes in real-world clinical practice. The results of this study are a significant step toward image-guided prediction of long-term disease activity in the management of nAMD and highlight the importance of the automatic identification of photoreceptor layers.

Unsupervised Domain Adaptation for Medical Image Segmentation by Disentanglement Learning and Self-Training.

Unsupervised domain adaption (UDA), which aims to enhance the segmentation performance of deep models on unlabeled data, has recently drawn much attention. In this paper, we propose a novel UDA method (namely DLaST) for medical image segmentation via disentanglement learning and self-training. Disentanglement learning factorizes an image into domain-invariant anatomy and domain-specific modality components. To make the best of disentanglement learning, we propose a novel shape constraint to boost the adaptation performance. The self-training strategy further adaptively improves the segmentation performance of the model for the target domain through adversarial learning and pseudo label, which implicitly facilitates feature alignment in the anatomy space. Experimental results demonstrate that the proposed method outperforms the state-of-the-art UDA methods for medical image segmentation on three public datasets, i.e., a cardiac dataset, an abdominal dataset and a brain dataset. The code will be released soon.