Language-Aware Spatial-Temporal Collaboration for Referring Video Segmentation.

Given a natural language referring expression, the goal of referring video segmentation task is to predict the segmentation mask of the referred object in the video. Previous methods only adopt 3D CNNs upon the video clip as a single encoder to extract a mixed spatio-temporal feature for the target frame. Though 3D convolutions are able to recognize which object is performing the described actions, they still introduce misaligned spatial information from adjacent frames, which inevitably confuses features of the target frame and leads to inaccurate segmentation. To tackle this issue, we propose a language-aware spatial-temporal collaboration framework that contains a 3D temporal encoder upon the video clip to recognize the described actions, and a 2D spatial encoder upon the target frame to provide undisturbed spatial features of the referred object. For multimodal features extraction, we propose a Cross-Modal Adaptive Modulation (CMAM) module and its improved version CMAM+ to conduct adaptive cross-modal interaction in the encoders with spatial- or temporal-relevant language features which are also updated progressively to enrich linguistic global context. In addition, we also propose a Language-Aware Semantic Propagation (LASP) module in the decoder to propagate semantic information from deep stages to the shallow stages with language-aware sampling and assignment, which is able to highlight language-compatible foreground visual features and suppress language-incompatible background visual features for better facilitating the spatial-temporal collaboration. Extensive experiments on four popular referring video segmentation benchmarks demonstrate the superiority of our method over the previous state-of-the-art methods.

Cross-Modal Progressive Comprehension for Referring Segmentation.

Given a natural language expression and an image/video, the goal of referring segmentation is to produce the pixel-level masks of the entities described by the subject of the expression. Previous approaches tackle this problem by implicit feature interaction and fusion between visual and linguistic modalities in a one-stage manner. However, human tends to solve the referring problem in a progressive manner based on informative words in the expression, i.e., first roughly locating candidate entities and then distinguishing the target one. In this paper, we propose a cross-modal progressive comprehension (CMPC) scheme to effectively mimic human behaviors and implement it as a CMPC-I (Image) module and a CMPC-V (Video) module to improve referring image and video segmentation models. For image data, our CMPC-I module first employs entity and attribute words to perceive all the related entities that might be considered by the expression. Then, the relational words are adopted to highlight the target entity as well as suppress other irrelevant ones by spatial graph reasoning. For video data, our CMPC-V module further exploits action words based on CMPC-I to highlight the correct entity matched with the action cues by temporal graph reasoning. In addition to the CMPC, we also introduce a simple yet effective Text-Guided Feature Exchange (TGFE) module to integrate the reasoned multimodal features corresponding to different levels in the visual backbone under the guidance of textual information. In this way, multi-level features can communicate with each other and be mutually refined based on the textual context. Combining CMPC-I or CMPC-V with TGFE can form our image or video version referring segmentation frameworks and our frameworks achieve new state-of-the-art performances on four referring image segmentation benchmarks and three referring video segmentation benchmarks respectively. Our code is available at https://github.com/spyflying/CMPC-Refseg.

ORDNet: Capturing Omni-Range Dependencies for Scene Parsing.

Learning to capture dependencies between spatial positions is essential to many visual tasks, especially the dense labeling problems like scene parsing. Existing methods can effectively capture long-range dependencies with self-attention mechanism while short ones by local convolution. However, there is still much gap between long-range and short-range dependencies, which largely reduces the models' flexibility in application to diverse spatial scales and relationships in complicated natural scene images. To fill such a gap, we develop a Middle-Range (MR) branch to capture middle-range dependencies by restricting self-attention into local patches. Also, we observe that the spatial regions which have large correlations with others can be emphasized to exploit long-range dependencies more accurately, and thus propose a Reweighed Long-Range (RLR) branch. Based on the proposed MR and RLR branches, we build an Omni-Range Dependencies Network (ORDNet) which can effectively capture short-, middle- and long-range dependencies. Our ORDNet is able to extract more comprehensive context information and well adapt to complex spatial variance in scene images. Extensive experiments show that our proposed ORDNet outperforms previous state-of-the-art methods on three scene parsing benchmarks including PASCAL Context, COCO Stuff and ADE20K, demonstrating the superiority of capturing omni-range dependencies in deep models for scene parsing task.

TLR4 signaling pathway mediates the LPS/ischemia-induced expression of monocytechemotactic protein-induced protein 1 in microglia.

Monocytechemotactic protein-induced protein 1 (MCPIP1), a newly recognized mRNA endonuclease, can be induced by lipopolysaccharide (LPS) and ischemic attack, then exerts a negative feedback loop against neuroinflammatory injury, but the specific underlying signaling pathway of the induction is unclear. Toll-like receptors (TLRs) and receptor for advanced glycation end products (RAGE) signaling pathways are involved in LPS/ischemia-evoked inflammation. This study aims to explore which receptor signaling is mainly involved in the induction of MCPIP1 by LPS and ischemic attack. BV2 cells and mice were subjected to LPS stimulation or transient middle cerebral artery occlusion (MCAO) to examine the modulation of MCPIP1. Specific inhibitors for TLR4, TLR2 or RAGE were preadministered to explore the mechanisms of MCPIP1 expression. Results showed that MCPIP1 was significantly increased by LPS and ischemic stress both in vitro and in vivo in time and dose dependent manners. Inhibition of TLR4, rather than TLR2 or RAGE, downregulated the LPS/ischemia-induced expression of MCPIP1 and reduced the levels of TLR4, MyD88, phosphorylated-MAPK (p-P38), phosphorylated-IκBα (p-IκBα), as well as the translocation of NF-κB (p65). In conclusion, we firstly demonstrate that TLR4 signaling pathway, not TLR2 or RAGE, predominantly mediates the LPS/ischemia-induced expression of MCPIP1 in microglia.

Regnase-1 in microglia negatively regulates high mobility group box 1-mediated inflammation and neuronal injury.

Extracellular high mobility group box 1 (HMGB1) has been demonstrated to function as a proinflammatory cytokine and induces neuronal injury in response to various pathological stimuli in central nervous system (CNS). However, the regulatory factor involved in HMGB1-mediated inflammatory signaling is largely unclear. Regulatory RNase 1 (Regnase-1) is a potent anti-inflammation enzyme that can degrade a set of mRNAs encoding proinflammatory cytokines. The present study aims to determine the role of Regnase-1 in the regulation of HMGB1-mediated inflammatory injury in CNS. Cultured microglia and rat brain were treated with recombinant HMGB1 to examine the induction of Regnase-1 expression. Moreover, the role of Regnase-1 in modulating the expression of inflammatory cytokines and neuronal injury was then investigated in microglia by specific siRNA knockdown upon HMGB1 treatment. Results showed that HMGB1 could significantly induce the de novo synthesis of Regnase-1 in cultured microglia. Consistently, Regnase-1 was elevated and found to be co-localized with microglia marker in the brain of rat treated with HMGB1. Silencing Regnase-1 in microglia enhanced HMGB1-induced expression of proinflammatory cytokines and exacerbated neuronal toxicity. Collectively, these results suggest that Regnase-1 can be induced by HMGB1 in microglia and negatively regulates HMGB1-mediated neuroinflammation and neuronal toxicity.

Emerging Roles of CCCH-Type Zinc Finger Proteins in Destabilizing mRNA Encoding Inflammatory Factors and Regulating Immune Responses.

Posttranscriptional gene regulation is a rapid and effective way to mediate the expression of inflammatory genes. CCCH-type zinc finger proteins are nucleotide-binding molecules involved in RNA metabolism pathways such as RNA splicing, polyadenylation, and messenger RNA (mRNA) decay. Among these proteins, tristetraproline, Roquins, and Regnase-1/monocyte chemotactic protein-1-induced protein-1 have been recently reported to be responsible for mRNA instability. They bind to mRNAs harboring unique motifs and induce mRNA decay. In this review we summarize current progress regarding the specific characteristics of sequences and structures in the 3' untranslated regions of mRNAs that are recognized by tristetraproline, Roquins, and Regnase-1. The target mRNAs to be destabilized by those CCCH-type zinc finger proteins also are included. Notably, most target mRNAs encode cytokines and other inflammatory mediators, suggesting the immune regulation role of CCCH zinc finger proteins. Mice carrying a genetic null allele or modification of these genes display severe symptoms of autoimmune diseases. Taken together, data show that CCCH-type zinc finger proteins play a crucial role in regulating immune response by targeting multiple mRNAs, and including decay. Further understanding the functions of these proteins may provide new therapeutic targets for immune-related disorders in the future.