Dynamic Frame Interpolation in Wavelet Domain.

Video frame interpolation is an important low-level vision task, which can increase frame rate for more fluent visual experience. Existing methods have achieved great success by employing advanced motion models and synthesis networks. However, the spatial redundancy when synthesizing the target frame has not been fully explored, that can result in lots of inefficient computation. On the other hand, the computation compression degree in frame interpolation is highly dependent on both texture distribution and scene motion, which demands to understand the spatial-temporal information of each input frame pair for a better compression degree selection. In this work, we propose a novel two-stage frame interpolation framework termed WaveletVFI to address above problems. It first estimates intermediate optical flow with a lightweight motion perception network, and then a wavelet synthesis network uses flow aligned context features to predict multi-scale wavelet coefficients with sparse convolution for efficient target frame reconstruction, where the sparse valid masks that control computation in each scale are determined by a crucial threshold ratio. Instead of setting a fixed value like previous methods, we find that embedding a classifier in the motion perception network to learn a dynamic threshold for each sample can achieve more computation reduction with almost no loss of accuracy. On the common high resolution and animation frame interpolation benchmarks, proposed WaveletVFI can reduce computation up to 40% while maintaining similar accuracy, making it perform more efficiently against other state-of-the-arts.

Long-term oxytetracycline exposure potentially alters brain thyroid hormone and serotonin homeostasis in zebrafish.

The impact of oxytetracycline (OTC) exposure in water on the fish still remains unclear. We hypothesized OTC exposure could alter fish gut microbiome and affect thyroid hormone and serotonin homeostasis in the brain via "chemical-gut-brain" axis. Here, ∼2-month-old juvenile zebrafish (Danio rerio) was exposed to two concentrations of OTC (1 and 100 µg/L) for one month until adulthood. Thyroxine-associated gene analysis in the brain revealed that deiodinase 2 (DIO2), deiodinase 3 (DIO3), and thyroid hormone receptor beta (THRß) expression was significantly decreased. Quantification of thyroid hormones showed a decrease in triiodothyronine (T3) under OTC treatment, which agrees with reduced activity of DIO2. For the serotonin (5-HT) synthesis, the expression of tryptophan hydroxylase (TPH2) was 41 % and 9.3 % of the control group for 1 and 100 µg/L OTC exposed groups; respectively. The intestinal 16S rRNA analysis revealed an increased abundance of Fusobacteria and Proteobacteria, while Actinobacteria was decreased significantly. The altered microbial balance between Proteobacteria and Firmicutes have been previously reported to affect nutrient uptakes such as zinc, which can potentially reduce the activity of DIO2. In summary, this study suggests that long-term OTC exposure not only alters gut microbiome but also changes thyroid hormone and serotonin homeostasis.

A Better Way to Attend: Attention with Trees for Video Question Answering.

We propose a new attention model for video question answering. The main idea of the attention models is to locate on the most informative parts of the visual data. The attention mechanisms are quite popular these days. However, most existing visual attention mechanisms regard the question as a whole. They ignore the word-level semantics where each word can have different attentions and some words need no attention. Neither do they consider the semantic structure of the sentences. Although the Extended Soft Attention (E-SA) model for video question answering leverages the word-level attention, it performs poorly on long question sentences. In this paper, we propose the heterogeneous tree-structured memory network (HTreeMN) for video question answering. Our proposed approach is based upon the syntax parse trees of the question sentences. The HTreeMN treats the words differently where the visual words are processed with an attention module and the verbal ones not. It also utilizes the semantic structure of the sentences by combining the neighbors based on the recursive structure of the parse trees. The understandings of the words and the videos are propagated and merged from leaves to the root. Furthermore, we build a hierarchical attention mechanism to distill the attended features. We evaluate our approach on two datasets. The experimental results show the superiority of our HTreeMN model over the other attention models especially on complex questions.