Dual Causes Generation Assisted Model for Multimodal Aspect-Based Sentiment Classification.

Multimodal aspect-based sentiment classification (MABSC) aims to identify the sentiment polarity toward specific aspects in multimodal data. It has gained significant attention with the increasing use of social media platforms. Existing approaches primarily focus on analyzing the content of posts to predict sentiment. However, they often struggle with limited contextual information inherent in social media posts, hindering accurate sentiment detection. To overcome this issue, we propose a novel multimodal dual cause analysis (MDCA) method to track the underlying causes behind expressed sentiments. MDCA can provide additional reasoning cause (RC) and direct cause (DC) to explain why users express certain emotions, thus helping improve the accuracy of sentiment prediction. To develop a model with MDCA, we construct MABSC datasets with RC and DC by utilizing large language models (LLMs) and visual-language models. Subsequently, we devise a multitask learning framework that leverages the datasets with cause data to train a small generative model, which can generate RC and DC, and predict the sentiment assisted by these causes. Experimental results on MABSC benchmark datasets demonstrate that our MDCA model achieves the state-of-the-art performance, and the small fine-tuned model exhibits superior adaptability to MABSC compared to large models like ChatGPT and BLIP-2.

A Stack-Propagation Framework With Slot Filling for Multi-Domain Dialogue State Tracking.

Dialogue state tracking (DST) is a core component of task-oriented dialogue systems. Recent works focus mainly on end-to-end DST models that omit the spoken language understanding (SLU) module to directly obtain the dialogue state based on a user's dialogue. However, the slot information detected by slot filling in SLU is closely tied to the slot-value pair that needs to be updated in DST. Efficient use of the key slot semantic knowledge obtained by slot filling contributes to improving the performance of DST. Based on this idea, we introduce slot filling as a subtask and build an end-to-end joint model to explicitly integrate the slot information detected by slot filling, which further guides DST. In this article, a novel stack-propagation framework with slot filling for multidomain DST is proposed. The stack-propagation framework is introduced to jointly model slot filling and DST. The framework directly feeds the key slot semantic knowledge detected by slot filling into the DST module. In addition, a slot-masked attention mechanism is designed to enable DST to focus on the key slot information obtained by slot filling. When the slot value is updated, a slot-value softcopy mechanism is designed to enhance the influence of the words marked by key slots. Experiments show that our approach outperforms previous methods and performs outstandingly on two benchmark datasets.