An Energy-Based Prior for Generative Saliency.

We propose a novel generative saliency prediction framework that adopts an informative energy-based model as a prior distribution. The energy-based prior model is defined on the latent space of a saliency generator network that generates the saliency map based on a continuous latent variables and an observed image. Both the parameters of saliency generator and the energy-based prior are jointly trained via Markov chain Monte Carlo-based maximum likelihood estimation, in which the sampling from the intractable posterior and prior distributions of the latent variables are performed by Langevin dynamics. With the generative saliency model, we can obtain a pixel-wise uncertainty map from an image, indicating model confidence in the saliency prediction. Different from existing generative models, which define the prior distribution of the latent variables as a simple isotropic Gaussian distribution, our model uses an energy-based informative prior which can be more expressive in capturing the latent space of the data. With the informative energy-based prior, we extend the Gaussian distribution assumption of generative models to achieve a more representative distribution of the latent space, leading to more reliable uncertainty estimation. We apply the proposed frameworks to both RGB and RGB-D salient object detection tasks with both transformer and convolutional neural network backbones. We further propose an adversarial learning algorithm and a variational inference algorithm as alternatives to train the proposed generative framework. Experimental results show that our generative saliency model with an energy-based prior can achieve not only accurate saliency predictions but also reliable uncertainty maps that are consistent with human perception.

Energy-Based Continuous Inverse Optimal Control.

The problem of continuous inverse optimal control (over finite time horizon) is to learn the unknown cost function over the sequence of continuous control variables from expert demonstrations. In this article, we study this fundamental problem in the framework of energy-based model (EBM), where the observed expert trajectories are assumed to be random samples from a probability density function defined as the exponential of the negative cost function up to a normalizing constant. The parameters of the cost function are learned by maximum likelihood via an "analysis by synthesis" scheme, which iterates: 1) synthesis step: sample the synthesized trajectories from the current probability density using the Langevin dynamics via backpropagation through time and 2) analysis step: update the model parameters based on the statistical difference between the synthesized trajectories and the observed trajectories. Given the fact that an efficient optimization algorithm is usually available for an optimal control problem, we also consider a convenient approximation of the above learning method, where we replace the sampling in the synthesis step by optimization. Moreover, to make the sampling or optimization more efficient, we propose to train the EBM simultaneously with a top-down trajectory generator via cooperative learning, where the trajectory generator is used to fast initialize the synthesis step of the EBM. We demonstrate the proposed methods on autonomous driving tasks and show that they can learn suitable cost functions for optimal control.

Insight into the fraction variations of selenium and their effects on humification during composting.

This study investigated the variation of selenium fractions and their effects on humification during composting. Selenite and selenate were added to a mixture of goat manure and wheat straw for composting. The results demonstrated that the bioavailable Se in the selenite added treatment (9.3-13.8%) was lower than in the selenate added treatment (18.1-47.3%). Meanwhile, the HA/FA of selenite and selenate added treatments were higher than in control, indicating that the selenium addition (especially selenite) promoted the humification of composting. Importantly, selenite enriched the abundance of Tepidimicrobium and Virgibacillus which were responsible to improve humification performance. Selenate increased the abundance of Thermobifida and Cellvibrio which facilitated the composting humification. The genes encoding CAZymes involved in the degradation of organic materials were also analyzed, and selenium could contribute to the synthesis of humus. KEGG pathway analysis revealed that the selenite addition promoted amino acids and carbohydrate metabolism compared to the control.

Theoretical Model for the Impact-Initiated Chemical Reaction of Al/PTFE Reactive Material.

Reactive material (RM) is a special kind of energetic material that can react and release chemical energy under highly dynamic loads. However, its energy release behavior is limited by its own strength, showing unique unsustainable characteristics, which lack a theoretical description. In this paper, an impact-initiated chemical reaction model is proposed to describe the ignition and energy release behavior of Al/PTFE RM. The hotspot formation mechanism of pore collapse was first introduced to describe the decomposition process of PTFE. Material fragmentation and PTFE decomposition were used as ignition criteria. Then the reaction rate of the decomposition product with aluminum was calculated according to the gas-solid chemical reaction model. Finally, the reaction states of RM calculated by the model are compared and qualitatively consistent with the experimental results. The model provides insight into the thermal-mechanical-chemical responses and references for the numerical simulation of impact ignition and energy release behavior of RM.

Role of selenite on the nitrogen conservation and greenhouse gases mitigation during the goat manure composting process.

This study aimed to explore the roles of selenite (Se) on nitrogen conservation and greenhouse gases (GHGs) mitigation during the composting process. Six levels of Se(IV) dosages (i.e. 0, 2, 4, 6, 8 and 10 mg/kg) were examined for 80-day composting of goat manure and wheat straw mixtures, where the different blending proportions were marked as T1 (Control), T2, T3, T4, T5 and T6, respectively. The results showed that adding Se(IV) was beneficial for reducing NH3 by 3.50-42.41% by buffering pH and promoting nitrification. For N2O, it showed different responses to different Se(IV) dosages, and it was increased by 29.62-71.29% in T2-T4 but reduced by 30.45-69.54% in T5-T6. Methane (CH4), another main component of GHGs, was increased by 1.35-107.42% by adding 2-10 mg/kg Se(IV). To further evaluate the effect of Se(IV) on GHGs, global warming potential value was calculated, which was 103.32-499.80 and minimum value was in T5. Furthermore, the physicochemical indexes, especially temperature and OM, had vital effects on microbial community. Overall, the results obtained from this study demonstrated that the application of Se (IV) in composting was reasonable to generate Se-rich organic fertilizer, and the 8 mg/kg was suggested from perspectives of nitrogen conservation and GHGs reduction.

Effect of micronutrient selenium on greenhouse gas emissions and related functional genes during goat manure composting.

To explore the effect of microelement selenium on greenhouse gas emission, nitrogen loss and related functional genes during the composting. Selenite and selenate were respectively mixed with goat manure and wheat straw and then composted the mixture without selenium regarded as control. The results indicated adding selenite prolonged the thermophilic phase and improved the organic matter degradation, while the selenate presented the opposite results. Selenite and selenate influenced ammonium transformation while prompting the formation of nitrate. Compared to the control, adding selenite and selenate both decreased NH3 emissions (by 26.7%-53.1%) and increased the total nitrogen content of compost. The addition of selenium increased mcrA in the early phase of composting, thereby promoting CH4 emission (by 3.5-18.4%). Meanwhile, adding selenate significantly reduced nirK abundance and consequently reduced N2O emission. Moreover, selenate added treatment presented the highest compost maturity (88.77%) and the lowest global warm potential (117.46 g/kg CO2-eq.) among all treatments.

Formation Behavior and Reaction Characteristic of a PTFE/Al Reactive Jet.

To reveal the expansion phenomenon and reaction characteristics of an aluminum particle filled polytetrafluoroethylene (PTFE/Al) reactive jet during the forming process, and to control the penetration and explosion coupling damage ability of the reactive jet, the temperature and density distribution of the reactive jet were investigated by combining numerical simulation and experimental study. Based on the platform of AUTODYN-3D code, the Smoothed Particle Hydrodynamics (SPH) algorithm was used to study the evolution behaviors and distribution regularity of the morphology, density, temperature, and velocity field during the formation process of the reactive composite jet. The reaction characteristic in the forming process was revealed by combining the distribution of the high-temperature zone in numerical simulation and the Differential Scanning Calorimeter/Thermo-Gravimetry (DSC/TG) experiment results. The results show that the distribution of the high-temperature zone of the reactive composite jet is mainly concentrated in the jet tip and the axial direction, and the reactive composite jet tip reacts first. Combining the density distribution in the numerical simulation and the pulsed X-ray experimental results, the forming behavior of the reactive composite jet was analyzed. The results show that the reactive composite jet has an obvious expansion effect, accompanied by a significant decrease in the overall density.

Generative VoxelNet: Learning Energy-Based Models for 3D Shape Synthesis and Analysis.

3D data that contains rich geometry information of objects and scenes is valuable for understanding 3D physical world. With the recent emergence of large-scale 3D datasets, it becomes increasingly crucial to have a powerful 3D generative model for 3D shape synthesis and analysis. This paper proposes a deep 3D energy-based model to represent volumetric shapes. The maximum likelihood training of the model follows an "analysis by synthesis" scheme. The benefits of the proposed model are six-fold: first, unlike GANs and VAEs, the model training does not rely on any auxiliary models; second, the model can synthesize realistic 3D shapes by Markov chain Monte Carlo (MCMC); third, the conditional model can be applied to 3D object recovery and super resolution; fourth, the model can serve as a building block in a multi-grid modeling and sampling framework for high resolution 3D shape synthesis; fifth, the model can be used to train a 3D generator via MCMC teaching; sixth, the unsupervisedly trained model provides a powerful feature extractor for 3D data, which is useful for 3D object classification. Experiments demonstrate that the proposed model can generate high-quality 3D shape patterns and can be useful for a wide variety of 3D shape analysis.

Cooperative Training of Fast Thinking Initializer and Slow Thinking Solver for Conditional Learning.

This paper studies the problem of learning the conditional distribution of a high-dimensional output given an input, where the output and input may belong to two different domains, e.g., the output is a photo image and the input is a sketch image. We solve this problem by cooperative training of a fast thinking initializer and slow thinking solver. The initializer generates the output directly by a non-linear transformation of the input as well as a noise vector that accounts for latent variability in the output. The slow thinking solver learns an objective function in the form of a conditional energy function, so that the output can be generated by optimizing the objective function, or more rigorously by sampling from the conditional energy-based model. We propose to learn the two models jointly, where the fast thinking initializer serves to initialize the sampling of the slow thinking solver, and the solver refines the initial output by an iterative algorithm. The solver learns from the difference between the refined output and the observed output, while the initializer learns from how the solver refines its initial output. We demonstrate the effectiveness of the proposed method on various conditional learning tasks, e.g., class-to-image generation, image-to-image translation, and image recovery. The advantage of our method over GAN-based methods is that our method is equipped with a slow thinking process that refines the solution guided by a learned objective function.

Monocular 3D Pose Estimation via Pose Grammar and Data Augmentation.

In this paper, we propose a pose grammar to tackle the problem of 3D human pose estimation from a monocular RGB image. Our model takes estimated 2D pose as the input and learns a generalized 2D-3D mapping function to leverage into 3D pose. The proposed model consists of a base network which efficiently captures pose-aligned features and a hierarchy of Bi-directional RNNs (BRNNs) on the top to explicitly incorporate a set of knowledge regarding human body configuration (i.e., kinematics, symmetry, motor coordination). The proposed model thus enforces high-level constraints over human poses. In learning, we develop a data augmentation algorithm to further improve model robustness against appearance variations and cross-view generalization ability. We validate our method on public 3D human pose benchmarks and propose a new evaluation protocol working on cross-view setting to verify the generalization capability of different methods. We empirically observe that most state-of-the-art methods encounter difficulty under such setting while our method can well handle such challenges.

Learning Energy-Based Spatial-Temporal Generative ConvNets for Dynamic Patterns.

Video sequences contain rich dynamic patterns, such as dynamic texture patterns that exhibit stationarity in the temporal domain, and action patterns that are non-stationary in either spatial or temporal domain. We show that an energy-based spatial-temporal generative ConvNet can be used to model and synthesize dynamic patterns. The model defines a probability distribution on the video sequence, and the log probability is defined by a spatial-temporal ConvNet that consists of multiple layers of spatial-temporal filters to capture spatial-temporal patterns of different scales. The model can be learned from the training video sequences by an "analysis by synthesis" learning algorithm that iterates the following two steps. Step 1 synthesizes video sequences from the currently learned model. Step 2 then updates the model parameters based on the difference between the synthesized video sequences and the observed training sequences. We show that the learning algorithm can synthesize realistic dynamic patterns. We also show that it is possible to learn the model from incomplete training sequences with either occluded pixels or missing frames, so that model learning and pattern completion can be accomplished simultaneously.

Revisiting Video Saliency Prediction in the Deep Learning Era.

Predicting where people look in static scenes, a.k.a visual saliency, has received significant research interest recently. However, relatively less effort has been spent in understanding and modeling visual attention over dynamic scenes. This work makes three contributions to video saliency research. First, we introduce a new benchmark, called DHF1K (Dynamic Human Fixation 1K), for predicting fixations during dynamic scene free-viewing, which is a long-time need in this field. DHF1K consists of 1K high-quality elaborately-selected video sequences annotated by 17 observers using an eye tracker device. The videos span a wide range of scenes, motions, object types and backgrounds. Second, we propose a novel video saliency model, called ACLNet (Attentive CNN-LSTM Network), that augments the CNN-LSTM architecture with a supervised attention mechanism to enable fast end-to-end saliency learning. The attention mechanism explicitly encodes static saliency information, thus allowing LSTM to focus on learning a more flexible temporal saliency representation across successive frames. Such a design fully leverages existing large-scale static fixation datasets, avoids overfitting, and significantly improves training efficiency and testing performance. Third, we perform an extensive evaluation of the state-of-the-art saliency models on three datasets : DHF1K, Hollywood-2, and UCF sports. An attribute-based analysis of previous saliency models and cross-dataset generalization are also presented. Experimental results over more than 1.2K testing videos containing 400K frames demonstrate that ACLNet outperforms other contenders and has a fast processing speed (40 fps using a single GPU). Our code and all the results are available at https://github.com/wenguanwang/DHF1K.

Cooperative Training of Descriptor and Generator Networks.

This paper studies the cooperative training of two generative models for image modeling and synthesis. Both models are parametrized by convolutional neural networks (ConvNets). The first model is a deep energy-based model, whose energy function is defined by a bottom-up ConvNet, which maps the observed image to the energy. We call it the descriptor network. The second model is a generator network, which is a non-linear version of factor analysis. It is defined by a top-down ConvNet, which maps the latent factors to the observed image. The maximum likelihood learning algorithms of both models involve MCMC sampling such as Langevin dynamics. We observe that the two learning algorithms can be seamlessly interwoven into a cooperative learning algorithm that can train both models simultaneously. Specifically, within each iteration of the cooperative learning algorithm, the generator model generates initial synthesized examples to initialize a finite-step MCMC that samples and trains the energy-based descriptor model. After that, the generator model learns from how the MCMC changes its synthesized examples. That is, the descriptor model teaches the generator model by MCMC, so that the generator model accumulates the MCMC transitions and reproduces them by direct ancestral sampling. We call this scheme MCMC teaching. We show that the cooperative algorithm can learn highly realistic generative models.

Penetration Behavior of High-Density Reactive Material Liner Shaped Charge.

The traditional polytetrafluoroethylene (PTFE)/Al reactive material liner shaped charge generally produces insufficient penetration depth, although it enlarges the penetration hole diameter by chemical energy release inside the penetration crater. As such, a novel high-density reactive material liner based on the PTFE matrix was fabricated, and the corresponding penetration performance was investigated. Firstly, the PTFE/W/Cu/Pb high-density reactive material liner was fabricated via a cold pressing/sintering process. Then, jet formation and penetration behaviors at different standoffs were studied by pulse X-ray and static experiments, respectively. The X-ray results showed that the PTFE/W/Cu/Pb high-density reactive material liner forms an excellent reactive jet penetrator, and the static experimental results demonstrated that the penetration depth of this high-density reactive jet increased firstly and then decreased by increasing the standoff. When the standoff was 1.5 CD (charge diameter), the penetration depth of this reactive jet reached 2.82 CD, which was significantly higher than that of the traditional PTFE/Al reactive jet. Moreover, compared with the conventional metal copper jet penetrating steel plates, the entrance hole diameter caused by this high-density reactive jet improved 29.2% at the same standoff. Lastly, the chemical reaction characteristics of PTFE/W/Cu/Pb reactive materials were analyzed, and a semi-empirical penetration model of the high-density reactive jet was established based on the quasi-steady ideal incompressible fluid dynamics theory.

Water metamaterial for ultra-broadband and wide-angle absorption.

A subwavelength water metamaterial is proposed and analyzed for ultra-broadband perfect absorption at microwave frequencies. We experimentally demonstrate that this metamaterial shows over 90% absorption within almost the entire frequency band of 12-29.6 GHz. It is also shown that the proposed metamaterial exhibits a good thermal stability with its absorption performance almost unchanged for the temperature range from 0 to 100°C. The study of the angular tolerance of the metamaterial absorber shows its ability of working at wide angles of incidence. Given that the proposed water metamaterial absorber is low-cost and easy for manufacture, we envision it may find numerous applications in electromagnetics such as broadband scattering reduction and electromagnetic energy harvesting.

Decoding the encoding of functional brain networks: An fMRI classification comparison of non-negative matrix factorization (NMF), independent component analysis (ICA), and sparse coding algorithms.

BACKGROUND: Brain networks in fMRI are typically identified using spatial independent component analysis (ICA), yet other mathematical constraints provide alternate biologically-plausible frameworks for generating brain networks. Non-negative matrix factorization (NMF) would suppress negative BOLD signal by enforcing positivity. Spatial sparse coding algorithms (L1 Regularized Learning and K-SVD) would impose local specialization and a discouragement of multitasking, where the total observed activity in a single voxel originates from a restricted number of possible brain networks. NEW METHOD: The assumptions of independence, positivity, and sparsity to encode task-related brain networks are compared; the resulting brain networks within scan for different constraints are used as basis functions to encode observed functional activity. These encodings are then decoded using machine learning, by using the time series weights to predict within scan whether a subject is viewing a video, listening to an audio cue, or at rest, in 304 fMRI scans from 51 subjects. RESULTS AND COMPARISON WITH EXISTING METHOD: The sparse coding algorithm of L1 Regularized Learning outperformed 4 variations of ICA (p<0.001) for predicting the task being performed within each scan using artifact-cleaned components. The NMF algorithms, which suppressed negative BOLD signal, had the poorest accuracy compared to the ICA and sparse coding algorithms. Holding constant the effect of the extraction algorithm, encodings using sparser spatial networks (containing more zero-valued voxels) had higher classification accuracy (p<0.001). Lower classification accuracy occurred when the extracted spatial maps contained more CSF regions (p<0.001). CONCLUSION: The success of sparse coding algorithms suggests that algorithms which enforce sparsity, discourage multitasking, and promote local specialization may capture better the underlying source processes than those which allow inexhaustible local processes such as ICA. Negative BOLD signal may capture task-related activations.

Non-negative matrix factorization of multimodal MRI, fMRI and phenotypic data reveals differential changes in default mode subnetworks in ADHD.

In the multimodal neuroimaging framework, data on a single subject are collected from inherently different sources such as functional MRI, structural MRI, behavioral and/or phenotypic information. The information each source provides is not independent; a subset of features from each modality maps to one or more common latent dimensions, which can be interpreted using generative models. These latent dimensions, or "topics," provide a sparse summary of the generative process behind the features for each individual. Topic modeling, an unsupervised generative model, has been used to map seemingly disparate features to a common domain. We use Non-Negative Matrix Factorization (NMF) to infer the latent structure of multimodal ADHD data containing fMRI, MRI, phenotypic and behavioral measurements. We compare four different NMF algorithms and find that the sparsest decomposition is also the most differentiating between ADHD and healthy patients. We identify dimensions that map to interpretable, recognizable dimensions such as motion, default mode network activity, and other such features of the input data. For example, structural and functional graph theory features related to default mode subnetworks clustered with the ADHD-Inattentive diagnosis. Structural measurements of the default mode network (DMN) regions such as the posterior cingulate, precuneus, and parahippocampal regions were all related to the ADHD-Inattentive diagnosis. Ventral DMN subnetworks may have more functional connections in ADHD-I, while dorsal DMN may have less. ADHD topics are dependent upon diagnostic site, suggesting diagnostic differences across geographic locations. We assess our findings in light of the ADHD-200 classification competition, and contrast our unsupervised, nominated topics with previously published supervised learning methods. Finally, we demonstrate the validity of these latent variables as biomarkers by using them for classification of ADHD in 730 patients. Cumulatively, this manuscript addresses how multimodal data in ADHD can be interpreted by latent dimensions.

[Study on the disabilities in aged 0-7 years children in Shenzhen, China].

OBJECTIVE: To explore the prevalence of vision, mental, audibility, language, psychiatry, extremity, and influence factors in the 0 - 7 year olds. METHODS: A total number of 77,727 0 - 7 year old children living in Shenzhen city were tested with tree phase screening under the Chinese standard of evaluation in disabilities. RESULTS: The prevalence of all disabilities was 5.59 per thousand (adjusted rate was 8.49 per thousand with a false negative of 3.1 per thousand ). The prevalence of mental disease was the highest (1.88 per thousand, with adjusted rate 3.43 per thousand ), the prevalence of language disability was 1.88 per thousand (including retarded language development, with adjusted rate 3.43 per thousand ). The prevalence rates of psychiatry, extremity and audibility disability were 1.59 per thousand, 1.56 per thousand, 1.11 per thousand respectively with of vision the lowest (0.37 per thousand ). The prevalence of all disabilities, audibility, language and mental was on the increase with age. The difference was statistically significant. Among all different age groups regarding psychiatric disease, the highest fell in the 2 - 4 year olds. The prevalence of extremity was not statistically different among age groups. The suspected agents of disease which occurred before or during pregnancy took up 45.7%. CONCLUSION: The prevalence of six kinds disabilities in Shenzhen was about 10 per thousand lower than that of the samples of the nation in 1989, but two times higher than that of similar studies in Japan. The prevalence rates of language and psychiatric disease were higher than that of the nation in 1989. The causation should be further studied.