Multi-Stage Network With Geometric Semantic Attention for Two-View Correspondence Learning.

The removal of outliers is crucial for establishing correspondence between two images. However, when the proportion of outliers reaches nearly 90%, the task becomes highly challenging. Existing methods face limitations in effectively utilizing geometric transformation consistency (GTC) information and incorporating geometric semantic neighboring information. To address these challenges, we propose a Multi-Stage Geometric Semantic Attention (MSGSA) network. The MSGSA network consists of three key modules: the multi-branch (MB) module, the GTC module, and the geometric semantic attention (GSA) module. The MB module, structured with a multi-branch design, facilitates diverse and robust spatial transformations. The GTC module captures transformation consistency information from the preceding stage. The GSA module categorizes input based on the prior stage's output, enabling efficient extraction of geometric semantic information through a graph-based representation and inter-category information interaction using Transformer. Extensive experiments on the YFCC100M and SUN3D datasets demonstrate that MSGSA outperforms current state-of-the-art methods in outlier removal and camera pose estimation, particularly in scenarios with a high prevalence of outliers. Source code is available at https://github.com/shuyuanlin.

Fractional Amplitude of Low-Frequency Fluctuation and Voxel-Mirrored Homotopic Connectivity in Patients with Persistent Postural-Perceptual Dizziness: Resting-State Functional Magnetic Resonance Imaging Study.

Purpose: Persistent postural-perception dizziness (PPPD) is a chronic subjective form of dizziness characterized by the exacerbation of dizziness with active or passive movement, complex visual stimuli, and upright posture. Therefore, we aimed to analyze the resting-state functional magnetic resonance imaging (fMRI) in patients with PPPD using fractional amplitude of low-frequency fluctuation (fALFF) and voxel-mirrored homotopic connectivity (VMHC) and evaluate the correlation between abnormal regions in the brain and clinical features to investigate the pathogenesis of PPPD. Methods: Thirty patients with PPPD (19 females and 11 males) and 30 healthy controls (HCs; 18 females and 12 males) were closely matched for age and sex. The fALFF and VMHC methods were used to investigate differences in fMRI (BOLD sequences) between the PPPD and HC groups and to explore the associations between areas of functional abnormality and clinical characteristics (dizziness, anxiety, depression, and duration). Result: Compared to the HC group, patients with PPPD displayed different functional change patterns, with increased fALFF in the right precuneus and decreased VMHC in the bilateral precuneus. In addition, patients with PPPD had a positive correlation between precuneus fALFF values and dizziness handicap inventory (DHI) scores, and a negative correlation between VMHC values and the disease duration. Conclusions: Precuneus dysfunction was observed in patients with PPPD. The fALFF values correlated with the degree of dizziness in PPPD, and changes in VMHC values were associated with the duration of dizziness, suggesting that fMRI changes in the precuneus of patients could be used as a potential imaging marker for PPPD.

Sinkhorn Distance Minimization for Adaptive Semi-Supervised Social Network Alignment.

Social network alignment, aiming at linking identical identities across different social platforms, is a fundamental task in social graph mining. Most existing approaches are supervised models and require a large number of manually labeled data, which are infeasible in practice considering the yawning gap between social platforms. Recently, isomorphism across social networks is incorporated as complementary to link identities from the distribution level, which contributes to alleviating the dependency on sample-level annotations. Adversarial learning is adopted to learn a shared projection function by minimizing the distance between two social distributions. However, the hypothesis of isomorphism might not always hold true as social user behaviors are generally unpredictable, and thus a shared projection function is insufficient to handle the sophisticated cross-platform correlations. In addition, adversarial learning suffers from training instability and uncertainty, which may hinder model performance. In this article, we propose a novel meta-learning-based social network alignment model Meta-SNA to effectively capture the isomorphism and the unique characteristics of each identity. Our motivation lies in learning a shared meta-model to preserve the global cross-platform knowledge and an adaptor to learn a specific projection function for each identity. Sinkhorn distance is further introduced as the distribution closeness measurement to tackle the limitations of adversarial learning, which owns an explicitly optimal solution and can be efficiently computed by the matrix scaling algorithm. Empirically, we evaluate the proposed model over multiple datasets, and the experimental results demonstrate the superiority of Meta-SNA.

Cross-Attentional Spatio-Temporal Semantic Graph Networks for Video Question Answering.

Due to the rich spatio-temporal visual content and complex multimodal relations, Video Question Answering (VideoQA) has become a challenging task and attracted increasing attention. Current methods usually leverage visual attention, linguistic attention, or self-attention to uncover latent correlations between video content and question semantics. Although these methods exploit interactive information between different modalities to improve comprehension ability, inter- and intra-modality correlations cannot be effectively integrated in a uniform model. To address this problem, we propose a novel VideoQA model called Cross-Attentional Spatio-Temporal Semantic Graph Networks (CASSG). Specifically, a multi-head multi-hop attention module with diversity and progressivity is first proposed to explore fine-grained interactions between different modalities in a crossing manner. Then, heterogeneous graphs are constructed from the cross-attended video frames, clips, and question words, in which the multi-stream spatio-temporal semantic graphs are designed to synchronously reasoning inter- and intra-modality correlations. Last, the global and local information fusion method is proposed to coalesce the local reasoning vector learned from multi-stream spatio-temporal semantic graphs and the global vector learned from another branch to infer the answer. Experimental results on three public VideoQA datasets confirm the effectiveness and superiority of our model compared with state-of-the-art methods.

Visual Sentiment Analysis With Social Relations-Guided Multiattention Networks.

These days, social media users tend to express their feelings through sharing images online. Capturing the emotions embedded in these social images involves great research challenges and practical values. Most existing works concentrate on extracting the visual feature from a global view, while ignoring the fact that visual objects are also rich in emotion. How to leverage the multilevel visual features to improve the sentiment analysis performance is important yet challenging. Besides, existing works view each social image as an independent sample while ignoring the rich correlations among social images, which may be helpful in detecting visual emotion. In this article, we propose a novel model called social relations-guided multiattention networks (SRGMANs) to incorporate both the multilevel (region-level and object-level) visual features of a single image and the correlations among multiple social images to conduct visual sentiment analysis. Specifically, we first construct a heterogeneous network consisting of various types of social relations and introduce a heterogeneous network embedding method to learn the network representation for each image. Then, two visual attention branches (region attention network and object attention network) are devised to extract emotional and discriminative visual features. For each branch, we design a self-attention module to capture the emotional dependencies among visual parts. Besides, a network-guided attention module is also designed in each branch to focus on more network-related emotional visual parts with the guidance of the topology information. Finally, the attended visual features from the two attention models, together with network representation features, are combined within a holistic framework to predict the sentiment of social images. Extensive experiments demonstrate the superiority of our model on three benchmark datasets.

Multilevel Attention Networks and Policy Reinforcement Learning for Image Caption Generation.

The analysis of large-scale multimodal data has become very popular recently. Image captioning, whose goal is to describe the content of image with natural language automatically, is an essential and challenging task in artificial intelligence. Commonly, most existing image caption methods utilize the mixture of Convolutional Neural Network and Recurrent Neural Network framework. These methods either pay attention to global representation at the image level or only focus on the specific concepts, such as regions and objects. To make the most of characteristics about a given image, in this study, we present a novel model named Multilevel Attention Networks and Policy Reinforcement Learning for image caption generation. Specifically, our model is composed of a multilevel attention network module and a policy reinforcement learning module. In the multilevel attention network, the object-attention network aims to capture global and local details about objects, whereas the region-attention network obtains global and local features about regions. After that, a policy reinforcement learning algorithm is adopted to overcome the exposure bias problem in the training phase and solve the loss-evaluation mismatching problem at the caption generation stage. With the attention network and policy algorithm, our model can automatically generate accurate and natural sentences for any particular image. We carry out extensive experiments on the MSCOCO and Flickr30k data sets, demonstrating that our model is superior to other competitive methods.

Multimodal Learning of Social Image Representation by Exploiting Social Relations.

Learning the representation for social images has recently made remarkable achievements for many tasks, such as cross-modal retrieval and multilabel classification. However, since social images contain both multimodal contents (e.g., visual images and textual descriptions) and social relations among images, simply modeling the content information may lead to suboptimal embedding. In this paper, we propose a novel multimodal representation learning model for social images, that is, correlational multimodal variational autoencoder (CMVAE) via triplet network. Specifically, in order to mine the highly nonlinear correlation between the visual content and the textual content, a CMVAE is proposed to learn a unified representation for the multiple modalities of social images. Both common information in all modalities and private information in each modality are encoded for the representation learning. To incorporate the social relations among images, we employ the triplet network to embed multiple types of social links in the representation learning. Then, a joint embedding model is proposed to combine the social relations for representation learning of the multimodal contents. Comprehensive experiment results on four datasets confirm the effectiveness of our method in two tasks, namely, multilabel classification and cross-modal retrieval. Our method outperforms the state-of-the-art multimodal representation learning methods with significant improvement of performance.

Adversarial Learning With Multi-Modal Attention for Visual Question Answering.

Visual question answering (VQA) has been proposed as a challenging task and attracted extensive research attention. It aims to learn a joint representation of the question-image pair for answer inference. Most of the existing methods focus on exploring the multi-modal correlation between the question and image to learn the joint representation. However, the answer-related information is not fully captured by these methods, which results that the learned representation is ineffective to reflect the answer of the question. To tackle this problem, we propose a novel model, i.e., adversarial learning with multi-modal attention (ALMA), for VQA. An adversarial learning-based framework is proposed to learn the joint representation to effectively reflect the answer-related information. Specifically, multi-modal attention with the Siamese similarity learning method is designed to build two embedding generators, i.e., question-image embedding and question-answer embedding. Then, adversarial learning is conducted as an interplay between the two embedding generators and an embedding discriminator. The generators have the purpose of generating two modality-invariant representations for the question-image and question-answer pairs, whereas the embedding discriminator aims to discriminate the two representations. Both the multi-modal attention module and the adversarial networks are integrated into an end-to-end unified framework to infer the answer. Experiments performed on three benchmark data sets confirm the favorable performance of ALMA compared with state-of-the-art approaches.

Brilliant blue G attenuates neuro-inflammation via regulating MAPKs and NF-κB signaling pathways in lipopolysaccharide-induced BV2 microglia cells.

Previous studies have demonstrated that the P2X purinoceptor 7 (P2X7) receptor (P2X7R) serves a critical role in regulating the inflammatory response of various diseases in the central nervous system. The anti-inflammatory effect of brilliant blue G (BBG), a specific antagonist of the P2X7R, remains unclear in lipopolysaccharide (LPS)-induced BV-2 cells. The present study suggested that BBG attenuated the neuroinflammatory response; the protein levels of inducible oxide synthase and cyclooxygenase-2, and the mRNA and secretion levels of pro-inflammatory cytokines including interleukin (IL)-16, IL-1ß and tumor necrosis factor-α (TNF-α), were all decreased in LPS-induced BV2 cells. BBG inhibited the activation of MAPKs by inhibiting the phosphorylation of p38 mitogen-activated protein kinase, c-Jun N-terminal kinase and extracellular signal-regulated kinase. Notably, transcription factor p65 nuclear translocation was also inhibited, thereby leading to the inactivation of NF-κB. The inhibitory effects of BBG on MAPKs and NF-κB were additionally enhanced through the application of MAPK and NF-κB inhibitors. Taken together, the results demonstrated that BBG contributed to the suppression of the inflammatory effects in LPS-induced BV2 cells via the inhibition of NF-κB and MAPKs signaling pathways.

Clinicopathological Analysis of Acquired Melanocytic Nevi and a Preliminary Study on the Possible Origin of Nevus Cells.

BACKGROUND: The pathogenesis of acquired melanocytic nevi (AMN) is still unclear, and the origin of nevus cells has not been clarified. OBJECTIVE: To analyze the clinical features and pathological types of AMN and identify the possible origin of nevus cells. METHODS: A retrospective study of 2929 cases of AMN was conducted, and 96 specimens of intradermal and junctional nevi were selected. Immunohistochemical assays were performed to detect the expression of basement membrane component receptor DDR-1 and the molecular markers on epidermal melanocytes, dermal stem cells (DSCs), and hair follicle stem cells. RESULTS: Junctional nevi and compound nevi were prone to occur on glabrous skin, such as the palms, soles, and vulva, and on the extremities in children, whereas intradermal nevi tended to develop on the trunk, head, and face of adults. The immunohistochemical data revealed that both junctional nevi and intradermal nevi expressed the epidermal melanocyte surface markers E-cadherin, DDR-1, and integrin α6 and the DSC molecular markers NGFRp-75 and nestin. CD34 was expressed only in junctional nevi, whereas K19 was not expressed in any type of melanocytic nevi. There was no significant difference in molecular expression at different sites or in different ages of onset. Nestin expression was markedly stronger in the intradermal nevi than in the junctional nevi, but there was no difference between the superficial and deep nevus cell nests of intradermal nevi. CONCLUSION: AMN may have a multicellular origin that commonly follows the mode of Abtropfung. Furthermore, DSCs may partly or independently participate in the formation of nevus cells.

Association between the DRD2 TaqIA gene polymorphism and Parkinson disease risk: an updated meta-analysis.

BACKGROUND: DRD2 TaqIA polymorphism may be associated with an increased risk of developing Parkinson disease (PD). However, the individual study's results are still inconsistent. METHODS: A meta-analysis of 4232 cases and 4774 controls from 14 separate studies were performed to explore the possible relationship between the DRD2 TaqIA gene polymorphism and PD. Pooled odds ratios (ORs) for the association and the corresponding 95% confidence intervals (CIs) were evaluated by a fixed-effect model. RESULTS: The pooled results revealed a significant association between DRD2 gene TaqIA polymorphism under recessive genetic model (OR: 0.91, 95% CI:0.83,0.99, P = .031) and additive genetic models (OR:0.93,95%CI:0.87,0.99, P = .032), but not associated with PD susceptibility under other genetic models in the whole population. Moreover, subgroups based on ethnicity and genotyping methods showed this association in the Caucasian subgroup under recessive genetic model (OR: 0.85, 95% CI:0.76,0.95, P = .003) and additive genetic models (OR:0.87,95%CI:0.79,0.96, P = .004) were existed. Besides, no significant association was detected under 6 genetic models in the Asian populations and PCR-RFLP subgroup. CONCLUSIONS: The current meta-analysis suggested that a significant association between DRD2 TaqIA polymorphism and PD under the recessive genetic mode, and additive genetic models, especially in Caucasians.

Modified shave surgery combined with nail window technique for the treatment of longitudinal melanonychia: Evaluation of the method on a series of 67 cases.

BACKGROUND: Nail matrix histopathologic examination is still the criterion standard to diagnose longitudinal melanonychia (LM). OBJECTIVE: To introduce modified shave surgery combined with the nail window technique for managing LM and evaluate the postoperative outcome of the procedure. METHODS: We retrospectively reviewed the medical records of 67 patients with LM who underwent shave surgery combined with the longitudinal-strip nail window technique at our institution from March 2015 to June 2018. RESULTS: Pathologic diagnosis was accessible in all cases, and 60 cases were assessable for the postoperative outcomes. A total of 45 cases (75.0%) had no postoperative nail dystrophy, and recurrence of nail pigmentation was found in only 8 cases (13.3%). LIMITATIONS: This was a retrospective study. CONCLUSION: Modified shave surgery combined with the nail window technique is the preferable management for LM cases, with limited postoperative nail dystrophy and recurrence of pigmentation.

Bi-directional Spatial-Semantic Attention Networks for Image-Text Matching.

Image-text matching by deep models has recently made remarkable achievements in many tasks, such as image caption and image search. A major challenge of matching the image and text lies in that they usually have complicated underlying relations between them and simply modeling the relations may lead to suboptimal performance. In this paper, we develop a novel approach Bi-directional Spatial-Semantic Attention Networks (BSSAN), which leverages both the word to regions (W2R) relation and image object to words (O2W) relation in a holistic deep framework for more effectively matching. Specifically, to effectively encode the W2R relation, we adopt LSTM with bilinear attention function to infer the image regions which are more related to the particular words, which is referred as the W2R attention network. On the other side, the O2W attention network is proposed to discover the semanticallyclose words for each visual object in the image, i.e., the visual object to words (O2W) relation. Then a deep model unifying both of the two directional attention networks into a holistic learning framework is proposed to learn the matching scores of image and text pairs. Compared to existing image-text matching methods, our approach achieves state-of-the-art performance on the datasets of Flickr30K and MSCOCO.

Preliminary study on the treatment of vitiligo with carbon dioxide fractional laser together with tacrolimus.

BACKGROUND: Tacrolimus is a conventional medication for the treatment of vitiligo, but the effect of a single medication is limited. OBJECTIVE: This paper aims at observing the effects, adverse responses, and repigmentation results of the joint treatment of vitiligo by Carbon dioxide (CO2 ) fractional laser together with tacrolimus. METHODS: Forty-five patients with vitiligo were randomly divided into two groups: treatment (T) group and control (C) group, and each group was further divided into three subgroups (face, torso and limbs, and hand and foot) according to the location of the skin defect. Both groups used topical 0.1% tacrolimus cream, but the T group was given one CO2 fractional laser treatment each month. We observed the clinical efficacy, adverse responses, and repigmentation results after 6 months. RESULTS: Compared to the C group, the T group showed better improvement in both objective and subjective assessments. When the treatment time was increased, the efficacy was also improved, and the repigmentation in the T group occured in three ways: perifollicular repigmentation, marginal repigmentation and diffuse repigmentation. There were three cases of isomorphic responses (2 cases in the rapid progression stage, one case in the progression stage), and 1 case formed scarring on the neck in the T group. CONCLUSIONS: The treatment of vitiligo by CO2 fractional laser together with tacrolimus is significantly effective and is most suitable for patients in the progression stage. Patients in the rapid progression stage should use this approach with caution, and its efficacy was limited for patients in the stable stage. An extended course of treatment is helpful for the repigmentation of white patches. All three forms of repigmentation can occur in the joint treatment of vitiligo by CO2 fractional laser together with tacrolimus. Lasers Surg. Med. 50:829-836, 2018. © 2018 Wiley Periodicals, Inc.

Real-time particle tracking for studying intracellular trafficking of pharmaceutical nanocarriers.

Real-time particle tracking is a technique that combines fluorescence microscopy with object tracking and computing and can be used to extract quantitative transport parameters for small particles inside cells. Since the success of a nanocarrier can often be determined by how effectively it delivers cargo to the target organelle, understanding the complex intracellular transport of pharmaceutical nanocarriers is critical. Real-time particle tracking provides insight into the dynamics of the intracellular behavior of nanoparticles, which may lead to significant improvements in the design and development of novel delivery systems. Unfortunately, this technique is not often fully understood, limiting its implementation by researchers in the field of nanomedicine. In this chapter, one of the most complicated aspects of particle tracking, the mean square displacement (MSD) calculation, is explained in a simple manner designed for the novice particle tracker. Pseudo code for performing the MSD calculation in MATLAB is also provided. This chapter contains clear and comprehensive instructions for a series of basic procedures in the technique of particle tracking. Instructions for performing confocal microscopy of nanoparticle samples are provided, and two methods of determining particle trajectories that do not require commercial particle-tracking software are provided. Trajectory analysis and determination of the tracking resolution are also explained. By providing comprehensive instructions needed to perform particle-tracking experiments, this chapter will enable researchers to gain new insight into the intracellular dynamics of nanocarriers, potentially leading to the development of more effective and intelligent therapeutic delivery vectors.

Comparison of membrane electroporation and protein denature in response to pulsed electric field with different durations.

In this paper, we compared the minimum potential differences in the electroporation of membrane lipid bilayers and the denaturation of membrane proteins in response to an intensive pulsed electric field with various pulse durations. Single skeletal muscle fibers were exposed to a pulsed external electric field. The field-induced changes in the membrane integrity (leakage current) and the Na channel currents were monitored to identify the minimum electric field needed to damage the membrane lipid bilayer and the membrane proteins, respectively. We found that in response to a relatively long pulsed electric shock (longer than the membrane intrinsic time constant), a lower membrane potential was needed to electroporate the cell membrane than for denaturing the membrane proteins, while for a short pulse a higher membrane potential was needed. In other words, phospholipid bilayers are more sensitive to the electric field than the membrane proteins for a long pulsed shock, while for a short pulse the proteins become more vulnerable. We can predict that for a short or ultrashort pulsed electric shock, the minimum membrane potential required to start to denature the protein functions in the cell plasma membrane is lower than that which starts to reduce the membrane integrity.

Real-time gene delivery vector tracking in the endo-lysosomal pathway of live cells.

Using live-cell confocal microscopy and particle tracking technology, the simultaneous transport of intracellular vesicles of the endo-lysosomal pathway and nonviral polyethylenimine (PEI)/DNA nanocomplexes was investigated. Due to potential problems associated with the use of acid-sensitive probes in combination with a gene vector that is hypothesized to buffer the pH of intracellular vesicles, the biological location of PEI/DNA gene vectors was revealed by probing their trafficking in cells expressing fluorescent versions of either early endosome antigen 1, a protein that localizes to early endosomes, or Niemann Pick C1, a protein that localizes to late endosomes and lysosomes. Studies directly show that PEI/DNA nanoparticles are actively transported within both early and late endosomes, and display similar overall transport rates in each. Additionally, gene vector transfer between endosomes is observed. Over time post-transfection, gene vectors accumulate in late endosomes/lysosomes; however, real-time escape of vectors from membrane-bound vesicles is not observed.

Quantitative nanoparticle tracking: applications to nanomedicine.

Particle tracking is an invaluable technique to extract quantitative and qualitative information regarding the transport of nanomaterials through complex biological environments. This technique can be used to probe the dynamic behavior of nanoparticles as they interact with and navigate through intra- and extra-cellular barriers. In this article, we focus on the recent developments in the application of particle-tracking technology to nanomedicine, including the study of synthetic and virus-based materials designed for gene and drug delivery. Specifically, we cover research where mean square displacements of nanomaterial transport were explicitly determined in order to quantitatively assess the transport of nanoparticles through biological environments. Particle-tracking experiments can provide important insights that may help guide the design of more intelligent and effective diagnostic and therapeutic nanoparticles.

Distribution of the NA/K pumps' turnover rates as a function of membrane potential, temperature, and ion concentration gradients and effect of fluctuations.

Because of structural independence of the Na/K pump molecules, the pumping rates of individual pumps may not be the same, instead showing some sort of distribution. Detailed information about the distribution has not previously been reported. The pumping rate of Na/K pumps depends on many parameters, such as membrane potential, temperature, and ion concentration gradients across the cell membrane. Fluctuation of any of the variables will change the pumping rate, resulting in a distribution. On the basis of a simplified six-state model, a steady-state pumping flux and therefore the pumping rate were obtained. Parameters were determined based on previous experimental results on amphibian skeletal muscle and theoretical work. Gaussian fluctuations of all the variables were considered to determine the changes in the pumping rate. These variable fluctuations may be totally independent or correlated to each other. The results showed that the pumping rates of the Na/K pumps are distributed in an asymmetric profile, which has a higher probability at the lower pumping rate. We present a model distribution of pumping rates as a function of temperature, membrane potential, and ion concentration.