Protecting Prostate Cancer Classification From Rectal Artifacts via Targeted Adversarial Training.

Magnetic resonance imaging (MRI)-based deep neural networks (DNN) have been widely developed to perform prostate cancer (PCa) classification. However, in real-world clinical situations, prostate MRIs can be easily impacted by rectal artifacts, which have been found to lead to incorrect PCa classification. Existing DNN-based methods typically do not consider the interference of rectal artifacts on PCa classification, and do not design specific strategy to address this problem. In this study, we proposed a novel Targeted adversarial training with Proprietary Adversarial Samples (TPAS) strategy to defend the PCa classification model against the influence of rectal artifacts. Specifically, based on clinical prior knowledge, we generated proprietary adversarial samples with rectal artifact-pattern adversarial noise, which can severely mislead PCa classification models optimized by the ordinary training strategy. We then jointly exploited the generated proprietary adversarial samples and original samples to train the models. To demonstrate the effectiveness of our strategy, we conducted analytical experiments on multiple PCa classification models. Compared with ordinary training strategy, TPAS can effectively improve the single- and multi-parametric PCa classification at patient, slice and lesion level, and bring substantial gains to recent advanced models. In conclusion, TPAS strategy can be identified as a valuable way to mitigate the influence of rectal artifacts on deep learning models for PCa classification.

GBSVM: An Efficient and Robust Support Vector Machine Framework via Granular-Ball Computing.

Granular-ball support vector machine (GBSVM) is a significant attempt to construct a classifier using the coarse-to-fine granularity of a granular ball as input, rather than a single data point. It is the first classifier whose input contains no points. However, the existing model has some errors, and its dual model has not been derived. As a result, the current algorithm cannot be implemented or applied. To address these problems, we fix the errors of the original model of the existing GBSVM and derive its dual model. Furthermore, a particle swarm optimization (PSO) algorithm is designed to solve the dual problem. The sequential minimal optimization (SMO) algorithm is also carefully designed to solve the dual problem. The latter is faster and more stable. The experimental results on the UCI benchmark datasets demonstrate that GBSVM is more robust and efficient. All codes have been released in the open source library available at: http://www.cquptshuyinxia.com/GBSVM.html or https://github.com/syxiaa/GBSVM.

Semi-Supervised Learning With Heterogeneous Distribution Consistency for Visible Infrared Person Re-Identification.

Visible infrared person re-identification (VI-ReID) exposes considerable challenges because of the modality gaps between the person images captured by daytime visible cameras and nighttime infrared cameras. Several fully-supervised VI-ReID methods have improved the performance with extensive labeled heterogeneous images. However, the identity of the person is difficult to obtain in real-world situations, especially at night. Limited known identities and large modality discrepancies impede the effectiveness of the model to a great extent. In this paper, we propose a novel Semi-Supervised Learning framework with Heterogeneous Distribution Consistency (HDC-SSL) for VI-ReID. Specifically, through investigating the confidence distribution of heterogeneous images, we introduce a Gaussian Mixture Model-based Pseudo Labeling (GMM-PL) method, which adaptively adjusts different thresholds for each modality to label the identity. Moreover, to facilitate the representation learning of unutilized data whose prediction is lower than the threshold, Modality Consistency Regularization (MCR) is proposed to ensure the prediction consistency of the cross-modality pedestrian images and handle the modality variance. Extensive experiments with different label settings on two VI-ReID datasets demonstrate the effectiveness of our method. Particularly, HDC-SSL achieves competitive performance with state-of-the-art fully-supervised VI-ReID methods on RegDB dataset with only 1 visible label and 1 infrared label per class.

Multi-modality Regional Alignment Network for Covid X-Ray Survival Prediction and Report Generation.

In response to the worldwide COVID-19 pandemic, advanced automated technologies have emerged as valuable tools to aid healthcare professionals in managing an increased workload by improving radiology report generation and prognostic analysis. This study proposes a Multi-modality Regional Alignment Network (MRANet), an explainable model for radiology report generation and survival prediction that focuses on high-risk regions. By learning spatial correlation in the detector, MRANet visually grounds region-specific descriptions, providing robust anatomical regions with a completion strategy. The visual features of each region are embedded using a novel survival attention mechanism, offering spatially and risk-aware features for sentence encoding while maintaining global coherence across tasks. A cross-domain LLMs-Alignment is employed to enhance the image-to-text transfer process, resulting in sentences rich with clinical detail and improved explainability for radiologists. Multi-center experiments validate the overall performance and each module's composition within the model, encouraging further advancements in radiology report generation research emphasizing clinical interpretation and trustworthiness in AI models applied to medical studies.

Effect of intraocular pressure reduction on progressive high myopia (PHM study): study protocol of a randomised controlled trial.

BACKGROUND: In adult patients with high myopia (HM), progressive axial elongation poses a significant risk for the development of subsequent ocular complications that may lead to visual impairment. Effective strategies to reduce or prevent further axial elongation in highly myopic adult patients have not been available so far. Recent studies suggested that medically lowering intraocular pressure (IOP) may reduce axial elongation. OBJECTIVE: This clinical randomised controlled trial (RCT) aims to evaluate the efficacy of medical IOP reduction in adult patients with progressive HM (PHM). TRIAL DESIGN: Single-centre, open-label, prospective RCT. METHODS: This RCT will recruit 152 participants with PHM at the Zhongshan Ophthalmic Center (ZOC). Randomised in a ratio of 1:1, participants will receive IOP-lowering eyedrops (intervention group) or will be followed without treatment (control group) for 12 months. Follow-up visits will be conducted at 1, 6 and 12 months after baseline. Only one eye per eligible participant will be included for analysis. The primary outcome is the change in axial length (AL) within the study period of 12 months. Secondary outcomes include the incidence and progression of visual field (VF) defects, changes in optic disc morphology and incidence and progression of myopic maculopathy. Difference in AL changes between the two groups will be analysed using linear regression analysis. For the secondary outcomes, a multifactor Poisson regression within a generalised linear model will be used to estimate the relative risk of progression in VF defects and myopic maculopathy, and the rate of thinning in retinal nerve fibre layer and ganglion cell-inner plexiform will be assessed through Kaplan-Meier curves and log-rank tests. ETHICS AND DISSEMINATION: Full ethics approval for this trial has been obtained from the Ethics Committee of ZOC, Sun Yat-sen University, China (ID: 2023KYPJ110). Results of this trial will be disseminated through peer-reviewed journals and conference presentations. TRIAL REGISTRATION NUMBER: NCT05850936.

Changes in monsoon precipitation in East Asia under a 2°C interglacial warming.

Past intervals of warming provide the unique opportunity to observe how the East Asia monsoon precipitation response happened in a warming world. However, the available evaluations are primarily limited to the last glacial-to-interglacial warming, which has fundamental differences from the current interglacial warming, particularly in changes in ice volume. Comparative paleoclimate studies of earlier warm interglacial periods can provide more realistic analogs. Here, we present high-resolution quantitative reconstructions of temperature and precipitation from north-central China over the past 800 thousand years. We found that the average precipitation increase, estimated by the interglacial data, was only around one-half of that estimated for the glacial-to-interglacial data, which is attributed to the amplification of climate change by ice volume variations. Analysis of the interglacial data suggests an increase in monsoon precipitation of ~100 mm for a warming level of 2°C on the Chinese Loess Plateau.

Efficacy and Safety of Surgical Peripheral Iridectomy, Goniosynechialysis, and Goniotomy for Advanced Primary Angle-closure Glaucoma without Cataract: One-year Results of a Multicenter Study.

PRCIS: The combination of surgical peripheral iridectomy, goniosynechialysis, and goniotomy is a safe and effective surgical approach for advanced primary angle-closure glaucoma without cataract. PURPOSE: To evaluate the efficacy and safety of surgical peripheral iridectomy (SPI), goniosynechialysis (GSL), and goniotomy (GT) in advanced primary angle-closure glaucoma (PACG) eyes without cataract. PATIENTS AND METHODS: A prospective multicenter observational study was performed for patients who underwent combined SPI, GSL, and GT for advanced PACG without cataract. Patients were assessed before and after the operation. Complete success was defined as achieving intraocular pressure (IOP) between 6-18 mm Hg with at least a 20% reduction compared to baseline, without the use of ocular hypotensive medications or reoperation. Qualified success adopted the same criteria but allowed medication use. Factors associated with surgical success were analyzed using logistic regression. RESULTS: A total of 61 eyes of 50 advanced PACG were included. All participants completed 12 months of follow-up. Thirty-six eyes (59.0%) achieved complete success, and 56 eyes (91.8%) achieved qualified success. Preoperative and postsurgical at 12 months mean IOPs were 29.7±7.7 and 16.1±4.8 mm Hg, respectively. The average number of ocular hypotensive medications decreased from 1.9 to 0.9 over 12 months. The primary complications included IOP spike (n=9), hyphema (n=7), and shallow anterior chamber (n=3). Regression analysis indicated that older age (odds ratio [OR]=1.09; P=0.043) was positively associated with complete success, while a mixed angle closure mechanism (OR=0.17; P=0.036) reduced success rate. CONCLUSIONS: The combination of SPI, GSL, and GT is a safe and effective surgical approach for advanced PACG without cataract. It has great potential as a first-line treatment option for these patients.

MGNR: A Multi-Granularity Neighbor Relationship and Its Application in KNN Classification and Clustering Methods.

In the real world, data distributions often exhibit multiple granularities. However, the majority of existing neighbor-based machine-learning methods rely on manually setting a single-granularity for neighbor relationships. These methods typically handle each data point using a single-granularity approach, which severely affects their accuracy and efficiency. This paper adopts a dual-pronged approach: it constructs a multi-granularity representation of the data using the granular-ball computing model, thereby boosting the algorithm's time efficiency. It leverages the multi-granularity representation of the data to create tailored, multi-granularity neighborhood relationships for different task scenarios, resulting in improved algorithmic accuracy. The experimental results convincingly demonstrate that the proposed multi-granularity neighbor relationship effectively enhances KNN classification and clustering methods. The source code has been publicly released and is now accessible on GitHub at https://github.com/xjnine/MGNR.

De-Biased Disentanglement Learning for Pulmonary Embolism Survival Prediction on Multimodal Data.

Health disparities among marginalized populations with lower socioeconomic status significantly impact the fairness and effectiveness of healthcare delivery. The increasing integration of artificial intelligence (AI) into healthcare presents an opportunity to address these inequalities, provided that AI models are free from bias. This paper aims to address the bias challenges by population disparities within healthcare systems, existing in the presentation of and development of algorithms, leading to inequitable medical implementation for conditions such as pulmonary embolism (PE) prognosis. In this study, we explore the diverse bias in healthcare systems, which highlights the demand for a holistic framework to reducing bias by complementary aggregation. By leveraging de-biasing deep survival prediction models, we propose a framework that disentangles identifiable information from images, text reports, and clinical variables to mitigate potential biases within multimodal datasets. Our study offers several advantages over traditional clinical-based survival prediction methods, including richer survival-related characteristics and bias-complementary predicted results. By improving the robustness of survival analysis through this framework, we aim to benefit patients, clinicians, and researchers by enhancing fairness and accuracy in healthcare AI systems.

Meta Invariance Defense Towards Generalizable Robustness to Unknown Adversarial Attacks.

Despite providing high-performance solutions for computer vision tasks, the deep neural network (DNN) model has been proved to be extremely vulnerable to adversarial attacks. Current defense mainly focuses on the known attacks, but the adversarial robustness to the unknown attacks is seriously overlooked. Besides, commonly used adaptive learning and fine-tuning technique is unsuitable for adversarial defense since it is essentially a zero-shot problem when deployed. Thus, to tackle this challenge, we propose an attack-agnostic defense method named Meta Invariance Defense (MID). Specifically, various combinations of adversarial attacks are randomly sampled from a manually constructed Attacker Pool to constitute different defense tasks against unknown attacks, in which a student encoder is supervised by multi-consistency distillation to learn the attack-invariant features via a meta principle. The proposed MID has two merits: 1) Full distillation from pixel-, feature- and prediction-level between benign and adversarial samples facilitates the discovery of attack-invariance. 2) The model simultaneously achieves robustness to the imperceptible adversarial perturbations in high-level image classification and attack-suppression in low-level robust image regeneration. Theoretical and empirical studies on numerous benchmarks such as ImageNet verify the generalizable robustness and superiority of MID under various attacks.

Adaptive Relation-Aware Network for zero-shot classification.

Supervised learning-based image classification in computer vision relies on visual samples containing a large amount of labeled information. Considering that it is labor-intensive to collect and label images and construct datasets manually, Zero-Shot Learning (ZSL) achieves knowledge transfer from seen categories to unseen categories by mining auxiliary information, which reduces the dependence on labeled image samples and is one of the current research hotspots in computer vision. However, most ZSL methods fail to properly measure the relationships between classes, or do not consider the differences and similarities between classes at all. In this paper, we propose Adaptive Relation-Aware Network (ARAN), a novel ZSL approach that incorporates the improved triplet loss from deep metric learning into a VAE-based generative model, which helps to model inter-class and intra-class relationships for different classes in ZSL datasets and generate an arbitrary amount of high-quality visual features containing more discriminative information. Moreover, we validate the effectiveness and superior performance of our ARAN through experimental evaluations under ZSL and more practical GZSL settings on three popular datasets AWA2, CUB, and SUN.

Few-Shot Face Stylization via GAN Prior Distillation.

Face stylization has made notable progress in recent years. However, when training on limited data, the performance of existing approaches significantly declines. Although some studies have attempted to tackle this problem, they either failed to achieve the few-shot setting (less than 10) or can only get suboptimal results. In this article, we propose GAN Prior Distillation (GPD) to enable effective few-shot face stylization. GPD contains two models: a teacher network with GAN Prior and a student network that fulfills end-to-end translation. Specifically, we adapt the teacher network trained on large-scale data in the source domain to the target domain using a handful of samples, where it can learn the target domain's knowledge. Then, we can achieve few-shot augmentation by generating source domain and target domain images simultaneously with the same latent codes. We propose an anchor-based knowledge distillation module that can fully use the difference between the training and the augmented data to distill the knowledge of the teacher network into the student network. The trained student network achieves excellent generalization performance with the absorption of additional knowledge. Qualitative and quantitative experiments demonstrate that our method achieves superior results than state-of-the-art approaches in a few-shot setting.

Neighbor-Guided Pseudo-Label Generation and Refinement for Single-Frame Supervised Temporal Action Localization.

Due to the sparse single-frame annotations, current Single-Frame Temporal Action Localization (SF-TAL) methods generally employ threshold-based pseudo-label generation strategies. However, these approaches suffer from inefficient data utilization, as only parts of unlabeled frames with confidence scores surpassing a predefined threshold are selected for training. Moreover, the variability of single-frame annotations and unreliable model predictions introduce pseudo-label noise. To address these challenges, we propose two strategies by using the relationship of the video segments with their neighbors': 1) temporal neighbor-guided soft pseudo-label generation (TNPG); and 2) semantic neighbor-guided pseudo-label refinement (SNPR). TNPG utilizes a local-global self-attention mechanism in a transformer encoder to capture temporal neighbor information while focusing on the whole video. Then the generated self-attention map is multiplied by the network predictions to propagate information between labeled and unlabeled frames, and produce soft pseudo-label for all segments. Despite this, label noise persists due to unreliable model predictions. To mitigate this, SNPR refines pseudo-labels based on the assumption that predictions should resemble their semantic nearest neighbors'. Specifically, we search for semantic nearest neighbors of each video segment by cosine similarity in the feature space. Then the refined soft pseudo-labels can be obtained by a weight combination of the original pseudo-label and the semantic nearest neighbors'. Finally, the model can be trained with the refined pseudo-labels, and the performance has been greatly improved. Comprehensive experimental results on different benchmarks show that we achieve state-of-the-art performances on THUMOS14, ActivityNet1.2, and ActivityNet1.3 datasets.

Sparse Dynamic Volume TransUNet with multi-level edge fusion for brain tumor segmentation.

3D MRI Brain Tumor Segmentation is of great significance in clinical diagnosis and treatment. Accurate segmentation results are critical for localization and spatial distribution of brain tumors using 3D MRI. However, most existing methods mainly focus on extracting global semantic features from the spatial and depth dimensions of a 3D volume, while ignoring voxel information, inter-layer connections, and detailed features. A 3D brain tumor segmentation network SDV-TUNet (Sparse Dynamic Volume TransUNet) based on an encoder-decoder architecture is proposed to achieve accurate segmentation by effectively combining voxel information, inter-layer feature connections, and intra-axis information. Volumetric data is fed into a 3D network consisting of extended depth modeling for dense prediction by using two modules: sparse dynamic (SD) encoder-decoder module and multi-level edge feature fusion (MEFF) module. The SD encoder-decoder module is utilized to extract global spatial semantic features for brain tumor segmentation, which employs multi-head self-attention and sparse dynamic adaptive fusion in a 3D extended shifted window strategy. In the encoding stage, dynamic perception of regional connections and multi-axis information interactions are realized through local tight correlations and long-range sparse correlations. The MEFF module achieves the fusion of multi-level local edge information in a layer-by-layer incremental manner and connects the fusion to the decoder module through skip connections to enhance the propagation ability of spatial edge information. The proposed method is applied to the BraTS2020 and BraTS2021 benchmarks, and the experimental results show its superior performance compared with state-of-the-art brain tumor segmentation methods. The source codes of the proposed method are available at https://github.com/SunMengw/SDV-TUNet.

Stimulant medications in children with ADHD normalize the structure of brain regions associated with attention and reward.

Children with ADHD show abnormal brain function and structure. Neuroimaging studies found that stimulant medications may improve brain structural abnormalities in children with ADHD. However, prior studies on this topic were conducted with relatively small sample sizes and wide age ranges and showed inconsistent results. In this cross-sectional study, we employed latent class analysis and linear mixed-effects models to estimate the impact of stimulant medications using demographic, clinical measures, and brain structure in a large and diverse sample of children aged 9-11 from the Adolescent Brain and Cognitive Development Study. We studied 273 children with low ADHD symptoms and received stimulant medication (Stim Low-ADHD), 1002 children with high ADHD symptoms and received no medications (No-Med ADHD), and 5378 typically developing controls (TDC). After controlling for the covariates, compared to Stim Low-ADHD and TDC, No-Med ADHD showed lower cortical thickness in the right insula (INS, d = 0.340, PFDR = 0.003) and subcortical volume in the left nucleus accumbens (NAc, d = 0.371, PFDR = 0.003), indicating that high ADHD symptoms were associated with structural abnormalities in these brain regions. In addition, there was no difference in brain structural measures between Stim Low-ADHD and TDC children, suggesting that the stimulant effects improved both ADHD symptoms and ADHD-associated brain structural abnormalities. These findings together suggested that children with ADHD appear to have structural abnormalities in brain regions associated with saliency and reward processing, and treatment with stimulant medications not only improve the ADHD symptoms but also normalized these brain structural abnormalities.

BPMTrack: Multi-Object Tracking With Detection Box Application Pattern Mining.

The key to multi-object tracking is its stability and the retention of identity information. A common problem with most detection-based approaches is trusting and using all the detector outputs for the association. However, some settings of detectors can affect stable long-range tracking. Based on the principle of reducing the association noise in the detection processing step, we propose a new framework, the Box application Pattern Mining Tracker (BPMTrack), to address this issue. Specifically, we worked on three main aspects: output threshold, association strategy, and motion model. Due to the problem of inconsistency between classification scores and localization accuracy, we propose the Box Quality Estimation Network (BQENet) to predict the localization quality scores of all detections in the current frame, reserving high-quality boxes for the tracker. In addition, based on observations of intensive scenarios, we propose a simple and effective data association method, the Non-Maximum Suppression Integration (NMSI) matching strategy. It recovers the Non-Maximum Suppression (NMS) detection, inputs them into BQENet, and then performs hierarchical matching with reasonable control of box priority to alleviate the problem of absent objects caused by occlusion. Finally, we propose an improved Measurement Correct and Noise Scale (MCNS) Kalman algorithm to improve the prediction accuracy of object positions and, thus, the association quality. We performed an extensive ablation evaluation of the proposed framework to prove its effectiveness. Moreover, the three tracking benchmarks show our method's accuracy and long-distance performance.

Phacogoniotomy versus phacotrabeculectomy for advanced primary angle-closure glaucoma with cataract: A randomized non-inferiority trial.

PURPOSE: To investigate the effectiveness and safety of phacogoniotomy versus phacotrabeculectomy (PVP) among patients with advanced primary angle-closure glaucoma (PACG) and cataracts. DESIGN: Multicenter, randomized controlled, non-inferiority trial. METHODS: A total of 124 patients (124 eyes) with advanced PACG and cataracts were enrolled, with 65 in the phacogoniotomy group and 59 in the phacotrabeculectomy group. Patients were followed up for 12 months with standardized evaluations. The primary outcome was the reduction in intraocular pressure (IOP) from baseline to 12 months postoperatively, of which a non-inferiority margin of 4 mmHg was evaluated. Secondary outcomes included the cumulative surgical success rate, postoperative complications, and changes in the number of glaucoma medications. RESULTS: After 12 months, phacogoniotomy demonstrated non-inferiority to phacotrabeculectomy in terms of IOP reduction, with mean IOP reductions of - 26.1 mmHg and - 25.7 mmHg (P = 0.383), respectively, from baseline values of around 40 mmHg. Both groups experienced a significant reduction in the mean number of medications used postoperatively (P < 0.001). The cumulative success rate was comparable between the groups (P = 0.890). However, phacogoniotomy had a lower rate of postoperative complications and interventions (12.3% and 4.6%) compared to phacotrabeculectomy (23.7% and 20.3% respectively). The phacogoniotomy group reported shorter surgery time (22.1 ± 6.5 vs. 38.8 ± 11.1 min; P = 0.030) and higher quality of life (EQ-5D-5 L) improvement at 12 months (7.0 ± 11.5 vs. 3.0 ± 12.9, P = 0.010) than the phacotrabeculectomy group. CONCLUSIONS: Phacogoniotomy was non-inferior to phacotrabeculectomy in terms of IOP reduction for advanced PACG and cataracts. Additionally, phacogoniotomy provided a shorter surgical time, lower postoperative complication rate, fewer postoperative interventions, and better postoperative quality of life.

Macrophage-induced integrin signaling promotes Schlemm's canal formation to prevent intraocular hypertension and glaucomatous optic neuropathy.

Schlemm's canal (SC) functions to maintain proper intraocular pressure (IOP) by draining aqueous humor and has emerged as a promising therapeutic target for glaucoma, the second-leading cause of irreversible blindness worldwide. However, our current understanding of the mechanisms governing SC development and functionality remains limited. Here, we show that vitronectin (VTN) produced by limbal macrophages promotes SC formation and prevents intraocular hypertension by activating integrin αvß3 signaling. Genetic inactivation of this signaling system inhibited the phosphorylation of AKT and FOXO1 and reduced ß-catenin activity and FOXC2 expression, thereby causing impaired Prox1 expression and deteriorated SC morphogenesis. This ultimately led to increased IOP and glaucomatous optic neuropathy. Intriguingly, we found that aged SC displayed downregulated integrin ß3 in association with dampened Prox1 expression. Conversely, FOXO1 inhibition rejuvenated the aged SC by inducing Prox1 expression and SC regrowth, highlighting a possible strategy by targeting VTN/integrin αvß3 signaling to improve SC functionality.

Associations Among Birth Weight, Adrenarche, Brain Morphometry, and Cognitive Function in Preterm Children Ages 9 to 11 Years.

BACKGROUND: Preterm infants with low birth weight are at heightened risk of developmental sequelae, including neurological and cognitive dysfunction that can persist into adolescence or adulthood. In addition, preterm birth and low birth weight can provoke changes in endocrine and metabolic processes that likely impact brain health throughout development. However, few studies have examined associations among birth weight, pubertal endocrine processes, and long-term neurological and cognitive development. METHODS: We investigated the associations between birth weight and brain morphometry, cognitive function, and onset of adrenarche assessed 9 to 11 years later in 3571 preterm and full-term children using the ABCD (Adolescent Brain Cognitive Development) Study dataset. RESULTS: The preterm children showed lower birth weight and early adrenarche, as expected. Birth weight was positively associated with cognitive function (all Cohen's d > 0.154, p < .005), global brain volumes (all Cohen's d > 0.170, p < .008), and regional volumes in frontal, temporal, and parietal cortices in preterm and full-term children (all Cohen's d > 0.170, p < .0007); cortical volume in the lateral orbitofrontal cortex partially mediated the effect of low birth weight on cognitive function in preterm children. In addition, adrenal score and cortical volume in the lateral orbitofrontal cortex mediated the associations between birth weight and cognitive function only in preterm children. CONCLUSIONS: These findings highlight the impact of low birth weight on long-term brain structural and cognitive function development and show important associations with early onset of adrenarche during the puberty. This understanding may help with prevention and treatment.

MDEformer: Mixed Difference Equation Inspired Transformer for Compressed Video Quality Enhancement.

Deep learning methods have achieved impressive performance in compressed video quality enhancement tasks. However, these methods rely excessively on practical experience by manually designing the network structure and do not fully exploit the potential of the feature information contained in the video sequences, i.e., not taking full advantage of the multiscale similarity of the compressed artifact information and not seriously considering the impact of the partition boundaries in the compressed video on the overall video quality. In this article, we propose a novel Mixed Difference Equation inspired Transformer (MDEformer) for compressed video quality enhancement, which provides a relatively reliable principle to guide the network design and yields a new insight into the interpretable transformer. Specifically, drawing on the graphical concept of the mixed difference equation (MDE), we utilize multiple cross-layer cross-attention aggregation (CCA) modules to establish long-range dependencies between encoders and decoders of the transformer, where partition boundary smoothing (PBS) modules are inserted as feedforward networks. The CCA module can make full use of the multiscale similarity of compression artifacts to effectively remove compression artifacts, and recover the texture and detail information of the frame. The PBS module leverages the sensitivity of smoothing convolution to partition boundaries to eliminate the impact of partition boundaries on the quality of compressed video and improve its overall quality, while not having too much impacts on non-boundary pixels. Extensive experiments on the MFQE 2.0 dataset demonstrate that the proposed MDEformer can eliminate compression artifacts for improving the quality of the compressed video, and surpasses the state-of-the-arts (SOTAs) in terms of both objective metrics and visual quality.