Weakening the Dominant Role of Text: CMOSI Dataset and Multimodal Semantic Enhancement Network.

Multimodal sentiment analysis (MSA) is important for quickly and accurately understanding people's attitudes and opinions about an event. However, existing sentiment analysis methods suffer from the dominant contribution of text modality in the dataset; this is called text dominance. In this context, we emphasize that weakening the dominant role of text modality is important for MSA tasks. To solve the above two problems, from the perspective of datasets, we first propose the Chinese multimodal opinion-level sentiment intensity (CMOSI) dataset. Three different versions of the dataset were constructed: manually proofreading subtitles, generating subtitles using machine speech transcription, and generating subtitles using human cross-language translation. The latter two versions radically weaken the dominant role of the textual model. We randomly collected 144 real videos from the Bilibili video site and manually edited 2557 clips containing emotions from them. From the perspective of network modeling, we propose a multimodal semantic enhancement network (MSEN) based on a multiheaded attention mechanism by taking advantage of the multiple versions of the CMOSI dataset. Experiments with our proposed CMOSI show that the network performs best with the text-unweakened version of the dataset. The loss of performance is minimal on both versions of the text-weakened dataset, indicating that our network can fully exploit the latent semantics in nontext patterns. In addition, we conducted model generalization experiments with MSEN on MOSI, MOSEI, and CH-SIMS datasets, and the results show that our approach is also very competitive and has good cross-language robustness.

Facial Expression Recognition Using Local Sliding Window Attention.

There are problems associated with facial expression recognition (FER), such as facial occlusion and head pose variations. These two problems lead to incomplete facial information in images, making feature extraction extremely difficult. Most current methods use prior knowledge or fixed-size patches to perform local cropping, thereby enhancing the ability to acquire fine-grained features. However, the former requires extra data processing work and is prone to errors; the latter destroys the integrity of local features. In this paper, we propose a local Sliding Window Attention Network (SWA-Net) for FER. Specifically, we propose a sliding window strategy for feature-level cropping, which preserves the integrity of local features and does not require complex preprocessing. Moreover, the local feature enhancement module mines fine-grained features with intraclass semantics through a multiscale depth network. The adaptive local feature selection module is introduced to prompt the model to find more essential local features. Extensive experiments demonstrate that our SWA-Net model achieves a comparable performance to that of state-of-the-art methods with scores of 90.03% on RAF-DB, 89.22% on FERPlus, 63.97% on AffectNet.

Assessment of the expression of the immune checkpoint molecules PD-1, CTLA4, TIM-3 and LAG-3 across different cancers in relation to treatment response, tumor-infiltrating immune cells and survival.

Immune checkpoint molecules have been identified as crucial regulators of the immune response, which motivated the emergence of immune checkpoint-targeting therapeutic strategies. However, the prognostic significance of the immune checkpoint molecules PD-1, CTLA4, TIM-3 and LAG-3 remains controversial. The aim of our study was to conduct a systematic assessment of the expression of these immune checkpoint molecules across different cancers in relation to treatment response, tumor-infiltrating immune cells and survival. Oncomine and PrognoScan database analyses were used to investigate the expression levels and prognostic values of these immune checkpoint molecule genes across various cancers. Then, we used Kaplan-Meier plotter to validate the associations between the checkpoint molecules and cancer survival identified in the PrognoScan analysis. TIMER analysis was used to evaluate immune cell infiltration data from The Cancer Genome Atlas. Finally, we used Gene Expression Profiling Interactive Analysis to investigate the prognostic value of these four checkpoint molecules and assess the correlations between these four checkpoint molecules and genetic markers. These immune checkpoint molecules may potentially serve as prognostic factors and therapeutic targets in breast cancer, ovarian cancer and lung cancer. The prognostic roles of these checkpoint molecules varied greatly across cancers, which implied a noteworthy amount of heterogeneity among tumors, even within the same molecular subtype. In addition, the expression patterns of these checkpoint molecules were closely associated with treatment response and provided some useful direction when choosing chemotherapeutic drugs. These findings enhance our understanding of these checkpoints in cancer treatment and identify strategies to promote synergistic activities in the context of other immunotherapies.