Energy-Based Continuous Inverse Optimal Control.

The problem of continuous inverse optimal control (over finite time horizon) is to learn the unknown cost function over the sequence of continuous control variables from expert demonstrations. In this article, we study this fundamental problem in the framework of energy-based model (EBM), where the observed expert trajectories are assumed to be random samples from a probability density function defined as the exponential of the negative cost function up to a normalizing constant. The parameters of the cost function are learned by maximum likelihood via an "analysis by synthesis" scheme, which iterates: 1) synthesis step: sample the synthesized trajectories from the current probability density using the Langevin dynamics via backpropagation through time and 2) analysis step: update the model parameters based on the statistical difference between the synthesized trajectories and the observed trajectories. Given the fact that an efficient optimization algorithm is usually available for an optimal control problem, we also consider a convenient approximation of the above learning method, where we replace the sampling in the synthesis step by optimization. Moreover, to make the sampling or optimization more efficient, we propose to train the EBM simultaneously with a top-down trajectory generator via cooperative learning, where the trajectory generator is used to fast initialize the synthesis step of the EBM. We demonstrate the proposed methods on autonomous driving tasks and show that they can learn suitable cost functions for optimal control.

Mapping Chinese annual gross primary productivity with eddy covariance measurements and machine learning.

Annual gross primary productivity (AGPP) is the basis for grain production and terrestrial carbon sequestration. Mapping regional AGPP from site measurements provides methodological support for analysing AGPP spatiotemporal variations thereby ensures regional food security and mitigates climate change. Based on 641 site-year eddy covariance measuring AGPP from China, we built an AGPP mapping scheme based on its formation and selected the optimal mapping way, which was conducted through analysing the predicting performances of divergent mapping tools, variable combinations, and mapping approaches in predicting observed AGPP variations. The reasonability of the selected optimal scheme was confirmed by assessing the consistency between its generating AGPP and previous products in spatiotemporal variations and total amount. Random forest regression tree explained 85 % of observed AGPP variations, outperforming other machine learning algorithms and classical statistical methods. Variable combinations containing climate, soil, and biological factors showed superior performance to other variable combinations. Mapping AGPP through predicting AGPP per leaf area (PAGPP) explained 86 % of AGPP variations, which was superior to other approaches. The optimal scheme was thus using a random forest regression tree, combining climate, soil, and biological variables, and predicting PAGPP. The optimal scheme generating AGPP of Chinese terrestrial ecosystems decreased from southeast to northwest, which was highly consistent with previous products. The interannual trend and interannual variation of our generating AGPP showed a decreasing trend from east to west and from southeast to northwest, respectively, which was consistent with data-oriented products. The mean total amount of generated AGPP was 7.03 ± 0.45 PgC yr-1 falling into the range of previous works. Considering the consistency between the generated AGPP and previous products, our optimal mapping way was suitable for mapping AGPP from site measurements. Our results provided a methodological support for mapping regional AGPP and other fluxes.

Pre-infection antiviral innate immunity contributes to sex differences in SARS-CoV-2 infection.

Male sex is a major risk factor for SARS-CoV-2 infection severity. To understand the basis for this sex difference, we studied SARS-CoV-2 infection in a young adult cohort of United States Marine recruits. Among 2,641 male and 244 female unvaccinated and seronegative recruits studied longitudinally, SARS-CoV-2 infections occurred in 1,033 males and 137 females. We identified sex differences in symptoms, viral load, blood transcriptome, RNA splicing, and proteomic signatures. Females had higher pre-infection expression of antiviral interferon-stimulated gene (ISG) programs. Causal mediation analysis implicated ISG differences in number of symptoms, levels of ISGs, and differential splicing of CD45 lymphocyte phosphatase during infection. Our results indicate that the antiviral innate immunity set point causally contributes to sex differences in response to SARS-CoV-2 infection. A record of this paper's transparent peer review process is included in the supplemental information.

[Collaborative monitoring network of ecologically fragile areas in China and its application in carrying capacity research]. / ä¸­å›½ç”Ÿæ€è„†å¼±åŒºè”ç½‘ååŒè§‚æµ‹åŠå ¶åœ¨æ‰¿è½½åŠ›ç ”ç©¶ä¸­çš„åº”ç”¨.

Ecologically fragile areas account for more than 60% of land area in China. Global change and human activities are aggravating ecosystem degradation and reducing the carrying capacity of resources and environment. It is important to accurately quantify the carrying capacity of resources and environment in ecologically fragile areas to deal with the risk and challenge of global change and to speed up the construction of ecological civilization. How-ever, existing methods evaluating carrying capacity of resources and environment are difficult to reflect the transmission effect of ecosystem structures, processes and functions changes among resource, environment and carrying capacity. Therefore, it is essential to establish a field observation network and obtain the comprehensive data set of resource and environment elements-ecosystem structure, function and process-ecosystem carrying capacity for develo-ping the theory and evaluation method. We introduced the collaborative monitoring networks of flux and UAV photographing, including the thoughts, practice, and preliminary results in the study of ecosystem structure, process and function in the fragile ecosystems of China. Based on the achievements and progress, we proposed the application of collaborative monitoring networks in capacity evaluation.

In situ bidirectional human-robot value alignment.

A prerequisite for social coordination is bidirectional communication between teammates, each playing two roles simultaneously: as receptive listeners and expressive speakers. For robots working with humans in complex situations with multiple goals that differ in importance, failure to fulfill the expectation of either role could undermine group performance due to misalignment of values between humans and robots. Specifically, a robot needs to serve as an effective listener to infer human users' intents from instructions and feedback and as an expressive speaker to explain its decision processes to users. Here, we investigate how to foster effective bidirectional human-robot communications in the context of value alignment-collaborative robots and users form an aligned understanding of the importance of possible task goals. We propose an explainable artificial intelligence (XAI) system in which a group of robots predicts users' values by taking in situ feedback into consideration while communicating their decision processes to users through explanations. To learn from human feedback, our XAI system integrates a cooperative communication model for inferring human values associated with multiple desirable goals. To be interpretable to humans, the system simulates human mental dynamics and predicts optimal explanations using graphical models. We conducted psychological experiments to examine the core components of the proposed computational framework. Our results show that real-time human-robot mutual understanding in complex cooperative tasks is achievable with a learning model based on bidirectional communication. We believe that this interaction framework can shed light on bidirectional value alignment in communicative XAI systems and, more broadly, in future human-machine teaming systems.

Deformable Generator Networks: Unsupervised Disentanglement of Appearance and Geometry.

We present a deformable generator model to disentangle the appearance and geometric information for both image and video data in a purely unsupervised manner. The appearance generator network models the information related to appearance, including color, illumination, identity or category, while the geometric generator performs geometric warping, such as rotation and stretching, through generating deformation field which is used to warp the generated appearance to obtain the final image or video sequences. Two generators take independent latent vectors as input to disentangle the appearance and geometric information from image or video sequences. For video data, a nonlinear transition model is introduced to both the appearance and geometric generators to capture the dynamics over time. The proposed scheme is general and can be easily integrated into different generative models. An extensive set of qualitative and quantitative experiments shows that the appearance and geometric information can be well disentangled, and the learned geometric generator can be conveniently transferred to other image datasets that share similar structure regularity to facilitate knowledge transfer tasks.

Generative VoxelNet: Learning Energy-Based Models for 3D Shape Synthesis and Analysis.

3D data that contains rich geometry information of objects and scenes is valuable for understanding 3D physical world. With the recent emergence of large-scale 3D datasets, it becomes increasingly crucial to have a powerful 3D generative model for 3D shape synthesis and analysis. This paper proposes a deep 3D energy-based model to represent volumetric shapes. The maximum likelihood training of the model follows an "analysis by synthesis" scheme. The benefits of the proposed model are six-fold: first, unlike GANs and VAEs, the model training does not rely on any auxiliary models; second, the model can synthesize realistic 3D shapes by Markov chain Monte Carlo (MCMC); third, the conditional model can be applied to 3D object recovery and super resolution; fourth, the model can serve as a building block in a multi-grid modeling and sampling framework for high resolution 3D shape synthesis; fifth, the model can be used to train a 3D generator via MCMC teaching; sixth, the unsupervisedly trained model provides a powerful feature extractor for 3D data, which is useful for 3D object classification. Experiments demonstrate that the proposed model can generate high-quality 3D shape patterns and can be useful for a wide variety of 3D shape analysis.

Cooperative Training of Fast Thinking Initializer and Slow Thinking Solver for Conditional Learning.

This paper studies the problem of learning the conditional distribution of a high-dimensional output given an input, where the output and input may belong to two different domains, e.g., the output is a photo image and the input is a sketch image. We solve this problem by cooperative training of a fast thinking initializer and slow thinking solver. The initializer generates the output directly by a non-linear transformation of the input as well as a noise vector that accounts for latent variability in the output. The slow thinking solver learns an objective function in the form of a conditional energy function, so that the output can be generated by optimizing the objective function, or more rigorously by sampling from the conditional energy-based model. We propose to learn the two models jointly, where the fast thinking initializer serves to initialize the sampling of the slow thinking solver, and the solver refines the initial output by an iterative algorithm. The solver learns from the difference between the refined output and the observed output, while the initializer learns from how the solver refines its initial output. We demonstrate the effectiveness of the proposed method on various conditional learning tasks, e.g., class-to-image generation, image-to-image translation, and image recovery. The advantage of our method over GAN-based methods is that our method is equipped with a slow thinking process that refines the solution guided by a learned objective function.

Mining Interpretable AOG Representations From Convolutional Networks via Active Question Answering.

In this paper, we present a method to mine object-part patterns from conv-layers of a pre-trained convolutional neural network (CNN). The mined object-part patterns are organized by an And-Or graph (AOG). This interpretable AOG representation consists of a four-layer semantic hierarchy, i.e., semantic parts, part templates, latent patterns, and neural units. The AOG associates each object part with certain neural units in feature maps of conv-layers. The AOG is constructed with very few annotations (e.g., 3-20) of object parts. We develop a question-answering (QA) method that uses active human-computer communications to mine patterns from a pre-trained CNN, in order to explain features in conv-layers incrementally. During the learning process, our QA method uses the current AOG for part localization. The QA method actively identifies objects, whose feature maps cannot be explained by the AOG. Then, our method asks people to annotate parts on the unexplained objects, and uses answers to discover CNN patterns corresponding to newly labeled parts. In this way, our method gradually grows new branches and refines existing branches on the AOG to semanticize CNN representations. In experiments, our method exhibited a high learning efficiency. Our method used about 1/6- 1/3 of the part annotations for training, but achieved similar or better part-localization performance than fast-RCNN methods.

Learning Energy-Based Spatial-Temporal Generative ConvNets for Dynamic Patterns.

Video sequences contain rich dynamic patterns, such as dynamic texture patterns that exhibit stationarity in the temporal domain, and action patterns that are non-stationary in either spatial or temporal domain. We show that an energy-based spatial-temporal generative ConvNet can be used to model and synthesize dynamic patterns. The model defines a probability distribution on the video sequence, and the log probability is defined by a spatial-temporal ConvNet that consists of multiple layers of spatial-temporal filters to capture spatial-temporal patterns of different scales. The model can be learned from the training video sequences by an "analysis by synthesis" learning algorithm that iterates the following two steps. Step 1 synthesizes video sequences from the currently learned model. Step 2 then updates the model parameters based on the difference between the synthesized video sequences and the observed training sequences. We show that the learning algorithm can synthesize realistic dynamic patterns. We also show that it is possible to learn the model from incomplete training sequences with either occluded pixels or missing frames, so that model learning and pattern completion can be accomplished simultaneously.

Interpretable CNNs for Object Classification.

This paper proposes a generic method to learn interpretable convolutional filters in a deep convolutional neural network (CNN) for object classification, where each interpretable filter encodes features of a specific object part. Our method does not require additional annotations of object parts or textures for supervision. Instead, we use the same training data as traditional CNNs. Our method automatically assigns each interpretable filter in a high conv-layer with an object part of a certain category during the learning process. Such explicit knowledge representations in conv-layers of the CNN help people clarify the logic encoded in the CNN, i.e., answering what patterns the CNN extracts from an input image and uses for prediction. We have tested our method using different benchmark CNNs with various architectures to demonstrate the broad applicability of our method. Experiments have shown that our interpretable filters are much more semantically meaningful than traditional filters.

Extraction of an Explanatory Graph to Interpret a CNN.

This paper introduces an explanatory graph representation to reveal object parts encoded inside convolutional layers of a CNN. Given a pre-trained CNN, each filter1 in a conv-layer usually represents a mixture of object parts. We develop a simple yet effective method to learn an explanatory graph, which automatically disentangles object parts from each filter without any part annotations. Specifically, given the feature map of a filter, we mine neural activations from the feature map, which correspond to different object parts. The explanatory graph is constructed to organize each mined part as a graph node. Each edge connects two nodes, whose corresponding object parts usually co-activate and keep a stable spatial relationship. Experiments show that each graph node consistently represented the same object part through different images, which boosted the transferability of CNN features. The explanatory graph transferred features of object parts to the task of part localization, and our method significantly outperformed other approaches.

Detecting Allele-Specific Alternative Splicing from Population-Scale RNA-Seq Data.

RNA sequencing (RNA-seq) is a powerful technology for studying human transcriptome variation. We introduce PAIRADISE (Paired Replicate Analysis of Allelic Differential Splicing Events), a method for detecting allele-specific alternative splicing (ASAS) from RNA-seq data. Unlike conventional approaches that detect ASAS events one sample at a time, PAIRADISE aggregates ASAS signals across multiple individuals in a population. By treating the two alleles of an individual as paired, and multiple individuals sharing a heterozygous SNP as replicates, we formulate ASAS detection using PAIRADISE as a statistical problem for identifying differential alternative splicing from RNA-seq data with paired replicates. PAIRADISE outperforms alternative statistical models in simulation studies. Applying PAIRADISE to replicate RNA-seq data of a single individual and to population-scale RNA-seq data across many individuals, we detect ASAS events associated with genome-wide association study (GWAS) signals of complex traits or diseases. Additionally, PAIRADISE ASAS analysis detects the effects of rare variants on alternative splicing. PAIRADISE provides a useful computational tool for elucidating the genetic variation and phenotypic association of alternative splicing in populations.

Cooperative Training of Descriptor and Generator Networks.

This paper studies the cooperative training of two generative models for image modeling and synthesis. Both models are parametrized by convolutional neural networks (ConvNets). The first model is a deep energy-based model, whose energy function is defined by a bottom-up ConvNet, which maps the observed image to the energy. We call it the descriptor network. The second model is a generator network, which is a non-linear version of factor analysis. It is defined by a top-down ConvNet, which maps the latent factors to the observed image. The maximum likelihood learning algorithms of both models involve MCMC sampling such as Langevin dynamics. We observe that the two learning algorithms can be seamlessly interwoven into a cooperative learning algorithm that can train both models simultaneously. Specifically, within each iteration of the cooperative learning algorithm, the generator model generates initial synthesized examples to initialize a finite-step MCMC that samples and trains the energy-based descriptor model. After that, the generator model learns from how the MCMC changes its synthesized examples. That is, the descriptor model teaches the generator model by MCMC, so that the generator model accumulates the MCMC transitions and reproduces them by direct ancestral sampling. We call this scheme MCMC teaching. We show that the cooperative algorithm can learn highly realistic generative models.

Randomized, Placebo-Controlled Trial of Targeting Neuroinflammation with Ibudilast to Treat Methamphetamine Use Disorder.

Methamphetamine (MA) triggers neuroinflammation and medications that counteract MA-induced neuroinflammation may reduce MA-induced neurodegeneration and improve neurocognition and treatment outcomes in MA use disorder. We performed a randomized, placebo-controlled trial to determine the safety and efficacy of ibudilast (IBUD), a phosphodiesterase inhibitor that reduces neuroinflammation, for the treatment of MA use disorder. Treatment-seeking volunteers with MA use disorder were randomly assigned to receive 12 weeks of IBUD 50 mg twice daily (N = 64) or placebo (N = 61) with medication management counseling. Participants visited the outpatient research clinic twice weekly to provide urine specimens for drug screens and undergo study assessments. The primary outcome was end of treatment MA-abstinence (EOTA) during weeks 11 and 12 of treatment. Serum IBUID levels were measured for IBUD participants during week 3 of treatment. There was no difference in EOTA for IBUD (14%) versus placebo (16%, p > 0.05). There was no correlation between serum IBUD levels and MA use during treatment and mean IBUD levels for participants with (mean = 51.3, SD = 20.3) and without (mean = 54.7, SD = 33.0, p = 0.70) EOTA. IBUD was well tolerated. IBUD did not facilitate MA abstinence in this outpatient trial. Whether targeting neuroinflammation, either with IBUD in other subgroups of MA users or clinical trial designs, or with other anti-inflammatory medications, is an effective strategy for treating MA use disorder is not clear. Graphical Abstract The proportion of urine drug screens negative for methamphetamine (MA) during the two week lead-in period (weeks -2 and - 1) and the 12 week medication treatment period (weeks 1-12) for ibudilast versus placebo.

Replicating Neuroscience Observations on ML/MF and AM Face Patches by Deep Generative Model.

A recent Cell paper (Chang & Tsao, 2017) reports an interesting discovery. For the face stimuli generated by a pretrained active appearance model (AAM), the responses of neurons in the areas of the primate brain that are responsible for face recognition exhibit a strong linear relationship with the shape variables and appearance variables of the AAM that generates the face stimuli. In this letter, we show that this behavior can be replicated by a deep generative model, the generator network, that assumes that the observed signals are generated by latent random variables via a top-down convolutional neural network. Specifically, we learn the generator network from the face images generated by a pretrained AAM model using a variational autoencoder, and we show that the inferred latent variables of the learned generator network have a strong linear relationship with the shape and appearance variables of the AAM model that generates the face images. Unlike the AAM model, which has an explicit shape model where the shape variables generate the control points or landmarks, the generator network has no such shape model and shape variables. Yet it can learn the shape knowledge in the sense that some of the latent variables of the learned generator network capture the shape variations in the face images generated by AAM.

Emergence of analogy from relation learning.

By middle childhood, humans are able to learn abstract semantic relations (e.g., antonym, synonym, category membership) and use them to reason by analogy. A deep theoretical challenge is to show how such abstract relations can arise from nonrelational inputs, thereby providing key elements of a protosymbolic representation system. We have developed a computational model that exploits the potential synergy between deep learning from "big data" (to create semantic features for individual words) and supervised learning from "small data" (to create representations of semantic relations between words). Given as inputs labeled pairs of lexical representations extracted by deep learning, the model creates augmented representations by remapping features according to the rank of differences between values for the two words in each pair. These augmented representations aid in coping with the feature alignment problem (e.g., matching those features that make "love-hate" an antonym with the different features that make "rich-poor" an antonym). The model extracts weight distributions that are used to estimate the probabilities that new word pairs instantiate each relation, capturing the pattern of human typicality judgments for a broad range of abstract semantic relations. A measure of relational similarity can be derived and used to solve simple verbal analogies with human-level accuracy. Because each acquired relation has a modular representation, basic symbolic operations are enabled (notably, the converse of any learned relation can be formed without additional training). Abstract semantic relations can be induced by bootstrapping from nonrelational inputs, thereby enabling relational generalization and analogical reasoning.

A tale of two explanations: Enhancing human trust by explaining robot behavior.

The ability to provide comprehensive explanations of chosen actions is a hallmark of intelligence. Lack of this ability impedes the general acceptance of AI and robot systems in critical tasks. This paper examines what forms of explanations best foster human trust in machines and proposes a framework in which explanations are generated from both functional and mechanistic perspectives. The robot system learns from human demonstrations to open medicine bottles using (i) an embodied haptic prediction model to extract knowledge from sensory feedback, (ii) a stochastic grammar model induced to capture the compositional structure of a multistep task, and (iii) an improved Earley parsing algorithm to jointly leverage both the haptic and grammar models. The robot system not only shows the ability to learn from human demonstrators but also succeeds in opening new, unseen bottles. Using different forms of explanations generated by the robot system, we conducted a psychological experiment to examine what forms of explanations best foster human trust in the robot. We found that comprehensive and real-time visualizations of the robot's internal decisions were more effective in promoting human trust than explanations based on summary text descriptions. In addition, forms of explanation that are best suited to foster trust do not necessarily correspond to the model components contributing to the best task performance. This divergence shows a need for the robotics community to integrate model components to enhance both task execution and human trust in machines.

JMJD5 cleaves monomethylated histone H3 N-tail under DNA damaging stress.

The histone H3 N-terminal protein domain (N-tail) is regulated by multiple posttranslational modifications, including methylation, acetylation, phosphorylation, and by proteolytic cleavage. However, the mechanism underlying H3 N-tail proteolytic cleavage is largely elusive. Here, we report that JMJD5, a Jumonji C (JmjC) domain-containing protein, is a Cathepsin L-type protease that mediates histone H3 N-tail proteolytic cleavage under stress conditions that cause a DNA damage response. JMJD5 clips the H3 N-tail at the carboxyl side of monomethyl-lysine (Kme1) residues. In vitro H3 peptide digestion reveals that JMJD5 exclusively cleaves Kme1 H3 peptides, while little or no cleavage effect of JMJD5 on dimethyl-lysine (Kme2), trimethyl-lysine (Kme3), or unmethyl-lysine (Kme0) H3 peptides is observed. Although H3 Kme1 peptides of K4, K9, K27, and K36 can all be cleaved by JMJD5 in vitro, K9 of H3 is the major cleavage site in vivo, and H3.3 is the major H3 target of JMJD5 cleavage. Cleavage is enhanced at gene promoters bound and repressed by JMJD5 suggesting a role for H3 N-tail cleavage in gene expression regulation.

Population and allelic variation of A-to-I RNA editing in human transcriptomes.

BACKGROUND: A-to-I RNA editing is an important step in RNA processing in which specific adenosines in some RNA molecules are post-transcriptionally modified to inosines. RNA editing has emerged as a widespread mechanism for generating transcriptome diversity. However, there remain significant knowledge gaps about the variation and function of RNA editing. RESULTS: In order to determine the influence of genetic variation on A-to-I RNA editing, we integrate genomic and transcriptomic data from 445 human lymphoblastoid cell lines by combining an RNA editing QTL (edQTL) analysis with an allele-specific RNA editing (ASED) analysis. We identify 1054 RNA editing events associated with cis genetic polymorphisms. Additionally, we find that a subset of these polymorphisms is linked to genome-wide association study signals of complex traits or diseases. Finally, compared to random cis polymorphisms, polymorphisms associated with RNA editing variation are located closer spatially to their respective editing sites and have a more pronounced impact on RNA secondary structure. CONCLUSIONS: Our study reveals widespread cis variation in RNA editing among genetically distinct individuals and sheds light on possible phenotypic consequences of such variation on complex traits and diseases.