PECAN Predicts Patterns of Cancer Cell Cytostatic Activity of Natural Products Using Deep Learning.

Many machine learning techniques are used as drug discovery tools with the intent to speed characterization by determining relationships between compound structure and biological function. However, particularly in anticancer drug discovery, these models often make only binary decisions about the biological activity for a narrow scope of drug targets. We present a feed-forward neural network, PECAN (Prediction Engine for the Cytostatic Activity of Natural product-like compounds), that simultaneously classifies the potential antiproliferative activity of compounds against 59 cancer cell lines. It predicts the activity to be one of six categories, indicating not only if activity is present but the degree of activity. Using an independent subset of NCI data as a test set, we show that PECAN can reach 60.1% accuracy in a six-way classification and present further evidence that it classifies based on useful structural features of compounds using a "within-one" measure that reaches 93.0% accuracy.

DeepSAT: Learning Molecular Structures from Nuclear Magnetic Resonance Data.

The identification of molecular structure is essential for understanding chemical diversity and for developing drug leads from small molecules. Nevertheless, the structure elucidation of small molecules by Nuclear Magnetic Resonance (NMR) experiments is often a long and non-trivial process that relies on years of training. To achieve this process efficiently, several spectral databases have been established to retrieve reference NMR spectra. However, the number of reference NMR spectra available is limited and has mostly facilitated annotation of commercially available derivatives. Here, we introduce DeepSAT, a neural network-based structure annotation and scaffold prediction system that directly extracts the chemical features associated with molecular structures from their NMR spectra. Using only the 1H-13C HSQC spectrum, DeepSAT identifies related known compounds and thus efficiently assists in the identification of molecular structures. DeepSAT is expected to accelerate chemical and biomedical research by accelerating the identification of molecular structures.

Adversarial Joint-Learning Recurrent Neural Network for Incomplete Time Series Classification.

Incomplete time series classification (ITSC) is an important issue in time series analysis since temporal data often has missing values in practical applications. However, integrating imputation (replacing missing data) and classification within a model often rapidly amplifies the error from imputed values. Reducing this error propagation from imputation to classification remains a challenge. To this end, we propose an adversarial joint-learning recurrent neural network (AJ-RNN) for ITSC, an end-to-end model trained in an adversarial and joint learning manner. We train the system to categorize the time series as well as impute missing values. To alleviate the error introduced by each imputation value, we use an adversarial network to encourage the network to impute realistic missing values by distinguishing real and imputed values. Hence, AJ-RNN can directly perform classification with missing values and greatly reduce the error propagation from imputation to classification, boosting the accuracy. Extensive experiments on 68 synthetic datasets and 4 real-world datasets from the expanded UCR time series archive demonstrate that AJ-RNN achieves state-of-the-art performance. Furthermore, we show that our model can effectively alleviate the accumulating error problem through qualitative and quantitative analysis based on the trajectory of the dynamical system learned by the RNN. We also provide an analysis of the model behavior to verify the effectiveness of our approach.

SMART-Miner: A convolutional neural network-based metabolite identification from 1 H-13 C HSQC spectra.

The identification of metabolites from complex biofluids and extracts of tissues is an essential process for understanding metabolic profiles. Nuclear magnetic resonance (NMR) spectroscopy is widely used in metabolomics studies for identification and quantification of metabolites. However, the accurate identification of individual metabolites is still a challenging process with higher peak intensity or similar chemical shifts from different metabolites. In this study, we applied a convolutional neural network (CNN) to 1 H-13 C HSQC NMR spectra to achieve accurate peak identification in complex mixtures. The results reveal that the neural network was successfully trained on metabolite identification from these 2D NMR spectra and achieved very good performance compared with other NMR-based metabolomic tools.

NPClassifier: A Deep Neural Network-Based Structural Classification Tool for Natural Products.

Computational approaches such as genome and metabolome mining are becoming essential to natural products (NPs) research. Consequently, a need exists for an automated structure-type classification system to handle the massive amounts of data appearing for NP structures. An ideal semantic ontology for the classification of NPs should go beyond the simple presence/absence of chemical substructures, but also include the taxonomy of the producing organism, the nature of the biosynthetic pathway, and/or their biological properties. Thus, a holistic and automatic NP classification framework could have considerable value to comprehensively navigate the relatedness of NPs, and especially so when analyzing large numbers of NPs. Here, we introduce NPClassifier, a deep-learning tool for the automated structural classification of NPs from their counted Morgan fingerprints. NPClassifier is expected to accelerate and enhance NP discovery by linking NP structures to their underlying properties.

The ventral striatum dissociates information expectation, reward anticipation, and reward receipt.

Do dopaminergic reward structures represent the expected utility of information similarly to a reward? Optimal experimental design models from Bayesian decision theory and statistics have proposed a theoretical framework for quantifying the expected value of information that might result from a query. In particular, this formulation quantifies the value of information before the answer to that query is known, in situations where payoffs are unknown and the goal is purely epistemic: That is, to increase knowledge about the state of the world. Whether and how such a theoretical quantity is represented in the brain is unknown. Here we use an event-related functional MRI (fMRI) task design to disentangle information expectation, information revelation and categorization outcome anticipation, and response-contingent reward processing in a visual probabilistic categorization task. We identify a neural signature corresponding to the expectation of information, involving the left lateral ventral striatum. Moreover, we show a temporal dissociation in the activation of different reward-related regions, including the nucleus accumbens, medial prefrontal cortex, and orbitofrontal cortex, during information expectation versus reward-related processing.

Author Correction: Small Molecule Accurate Recognition Technology (SMART) to Enhance Natural Products Research.

An amendment to this paper has been published and can be accessed via a link at the top of the paper.

A Convolutional Neural Network-Based Approach for the Rapid Annotation of Molecularly Diverse Natural Products.

This report describes the first application of the novel NMR-based machine learning tool "Small Molecule Accurate Recognition Technology" (SMART 2.0) for mixture analysis and subsequent accelerated discovery and characterization of new natural products. The concept was applied to the extract of a filamentous marine cyanobacterium known to be a prolific producer of cytotoxic natural products. This environmental Symploca extract was roughly fractionated, and then prioritized and guided by cancer cell cytotoxicity, NMR-based SMART 2.0, and MS2-based molecular networking. This led to the isolation and rapid identification of a new chimeric swinholide-like macrolide, symplocolide A, as well as the annotation of swinholide A, samholides A-I, and several new derivatives. The planar structure of symplocolide A was confirmed to be a structural hybrid between swinholide A and luminaolide B by 1D/2D NMR and LC-MS2 analysis. A second example applies SMART 2.0 to the characterization of structurally novel cyclic peptides, and compares this approach to the recently appearing "atomic sort" method. This study exemplifies the revolutionary potential of combined traditional and deep learning-assisted analytical approaches to overcome longstanding challenges in natural products drug discovery.

Pagoamide A, a Cyclic Depsipeptide Isolated from a Cultured Marine Chlorophyte, Derbesia sp., Using MS/MS-Based Molecular Networking.

A thiazole-containing cyclic depsipeptide with 11 amino acid residues, named pagoamide A (1), was isolated from laboratory cultures of a marine Chlorophyte, Derbesia sp. This green algal sample was collected from America Samoa, and pagoamide A was isolated using guidance by MS/MS-based molecular networking. Cultures were grown in a light- and temperature-controlled environment and harvested after several months of growth. The planar structure of pagoamide A (1) was characterized by detailed 1D and 2D NMR experiments along with MS and UV analysis. The absolute configurations of its amino acid residues were determined by advanced Marfey's analysis following chemical hydrolysis and hydrazinolysis reactions. Two of the residues in pagoamide A (1), phenylalanine and serine, each occurred twice in the molecule, once in the d- and once in the l-configuration. The biosynthetic origin of pagoamide A (1) was considered in light of other natural products investigations with coenocytic green algae.

End-to-End Incomplete Time-Series Modeling From Linear Memory of Latent Variables.

Time series with missing values (incomplete time series) are ubiquitous in real life on account of noise or malfunctioning sensors. Time-series imputation (replacing missing data) remains a challenge due to the potential for nonlinear dependence on concurrent and previous values of the time series. In this paper, we propose a novel framework for modeling incomplete time series, called a linear memory vector recurrent neural network (LIME-RNN), a recurrent neural network (RNN) with a learned linear combination of previous history states. The technique bears some similarity to residual networks and graph-based temporal dependency imputation. In particular, we introduce a linear memory vector [called the residual sum vector (RSV)] that integrates over previous hidden states of the RNN, and is used to fill in missing values. A new loss function is developed to train our model with time series in the presence of missing values in an end-to-end way. Our framework can handle imputation of both missing-at-random and consecutive missing inputs. Moreover, when conducting time-series prediction with missing values, LIME-RNN allows imputation and prediction simultaneously. We demonstrate the efficacy of the model via extensive experimental evaluation on univariate and multivariate time series, achieving state-of-the-art performance on synthetic and real-world data. The statistical results show that our model is significantly better than most existing time-series univariate or multivariate imputation methods.

Time series classification with Echo Memory Networks.

Echo state networks (ESNs) are randomly connected recurrent neural networks (RNNs) that can be used as a temporal kernel for modeling time series data, and have been successfully applied on time series prediction tasks. Recently, ESNs have been applied to time series classification (TSC) tasks. However, previous ESN-based classifiers involve either training the model by predicting the next item of a sequence, or predicting the class label at each time step. The former is essentially a predictive model adapted from time series prediction work, rather than a model designed specifically for the classification task. The latter approach only considers local patterns at each time step and then averages over the classifications. Hence, rather than selecting the most discriminating sections of the time series, this approach will incorporate non-discriminative information into the classification, reducing accuracy. In this paper, we propose a novel end-to-end framework called the Echo Memory Network (EMN) in which the time series dynamics and multi-scale discriminative features are efficiently learned from an unrolled echo memory using multi-scale convolution and max-over-time pooling. First, the time series data are projected into the high dimensional nonlinear space of the reservoir and the echo states are collected into the echo memory matrix, followed by a single multi-scale convolutional layer to extract multi-scale features from the echo memory matrix. Max-over-time pooling is used to maintain temporal invariance and select the most important local patterns. Finally, a fully-connected hidden layer feeds into a softmax layer for classification. This architecture is applied to both time series classification and human action recognition datasets. For the human action recognition datasets, we divide the action data into five different components of the human body, and propose two spatial information fusion strategies to integrate the spatial information over them. With one training-free recurrent layer and only one layer of convolution, the EMN is a very efficient end-to-end model, and ranks first in overall classification ability on 55 TSC benchmark datasets and four 3D skeleton-based human action recognition tasks.

Small Molecule Accurate Recognition Technology (SMART) to Enhance Natural Products Research.

Various algorithms comparing 2D NMR spectra have been explored for their ability to dereplicate natural products as well as determine molecular structures. However, spectroscopic artefacts, solvent effects, and the interactive effect of functional group(s) on chemical shifts combine to hinder their effectiveness. Here, we leveraged Non-Uniform Sampling (NUS) 2D NMR techniques and deep Convolutional Neural Networks (CNNs) to create a tool, SMART, that can assist in natural products discovery efforts. First, an NUS heteronuclear single quantum coherence (HSQC) NMR pulse sequence was adapted to a state-of-the-art nuclear magnetic resonance (NMR) instrument, and data reconstruction methods were optimized, and second, a deep CNN with contrastive loss was trained on a database containing over 2,054 HSQC spectra as the training set. To demonstrate the utility of SMART, several newly isolated compounds were automatically located with their known analogues in the embedded clustering space, thereby streamlining the discovery pipeline for new natural products.

Central and peripheral vision for scene recognition: A neurocomputational modeling exploration.

What are the roles of central and peripheral vision in human scene recognition? Larson and Loschky (2009) showed that peripheral vision contributes more than central vision in obtaining maximum scene recognition accuracy. However, central vision is more efficient for scene recognition than peripheral, based on the amount of visual area needed for accurate recognition. In this study, we model and explain the results of Larson and Loschky (2009) using a neurocomputational modeling approach. We show that the advantage of peripheral vision in scene recognition, as well as the efficiency advantage for central vision, can be replicated using state-of-the-art deep neural network models. In addition, we propose and provide support for the hypothesis that the peripheral advantage comes from the inherent usefulness of peripheral features. This result is consistent with data presented by Thibaut, Tran, Szaffarczyk, and Boucart (2014), who showed that patients with central vision loss can still categorize natural scenes efficiently. Furthermore, by using a deep mixture-of-experts model ("The Deep Model," or TDM) that receives central and peripheral visual information on separate channels simultaneously, we show that the peripheral advantage emerges naturally in the learning process: When trained to categorize scenes, the model weights the peripheral pathway more than the central pathway. As we have seen in our previous modeling work, learning creates a transform that spreads different scene categories into different regions in representational space. Finally, we visualize the features for the two pathways, and find that different preferences for scene categories emerge for the two pathways during the training process.

Are Face and Object Recognition Independent? A Neurocomputational Modeling Exploration.

Are face and object recognition abilities independent? Although it is commonly believed that they are, Gauthier et al. [Gauthier, I., McGugin, R. W., Richler, J. J., Herzmann, G., Speegle, M., & VanGulick, A. E. Experience moderates overlap between object and face recognition, suggesting a common ability. Journal of Vision, 14, 7, 2014] recently showed that these abilities become more correlated as experience with nonface categories increases. They argued that there is a single underlying visual ability, v, that is expressed in performance with both face and nonface categories as experience grows. Using the Cambridge Face Memory Test and the Vanderbilt Expertise Test, they showed that the shared variance between Cambridge Face Memory Test and Vanderbilt Expertise Test performance increases monotonically as experience increases. Here, we address why a shared resource across different visual domains does not lead to competition and to an inverse correlation in abilities? We explain this conundrum using our neurocomputational model of face and object processing ["The Model", TM, Cottrell, G. W., & Hsiao, J. H. Neurocomputational models of face processing. In A. J. Calder, G. Rhodes, M. Johnson, & J. Haxby (Eds.), The Oxford handbook of face perception. Oxford, UK: Oxford University Press, 2011]. We model the domain general ability v as the available computational resources (number of hidden units) in the mapping from input to label and experience as the frequency of individual exemplars in an object category appearing during network training. Our results show that, as in the behavioral data, the correlation between subordinate level face and object recognition accuracy increases as experience grows. We suggest that different domains do not compete for resources because the relevant features are shared between faces and objects. The essential power of experience is to generate a "spreading transform" for faces (separating them in representational space) that generalizes to objects that must be individuated. Interestingly, when the task of the network is basic level categorization, no increase in the correlation between domains is observed. Hence, our model predicts that it is the type of experience that matters and that the source of the correlation is in the fusiform face area, rather than in cortical areas that subserve basic level categorization. This result is consistent with our previous modeling elucidating why the FFA is recruited for novel domains of expertise [Tong, M. H., Joyce, C. A., & Cottrell, G. W. Why is the fusiform face area recruited for novel categories of expertise? A neurocomputational investigation. Brain Research, 1202, 14-24, 2008].

Humans have idiosyncratic and task-specific scanpaths for judging faces.

Since Yarbus's seminal work, vision scientists have argued that our eye movement patterns differ depending upon our task. This has recently motivated the creation of multi-fixation pattern analysis algorithms that try to infer a person's task (or mental state) from their eye movements alone. Here, we introduce new algorithms for multi-fixation pattern analysis, and we use them to argue that people have scanpath routines for judging faces. We tested our methods on the eye movements of subjects as they made six distinct judgments about faces. We found that our algorithms could detect whether a participant is trying to distinguish angriness, happiness, trustworthiness, tiredness, attractiveness, or age. However, our algorithms were more accurate at inferring a subject's task when only trained on data from that subject than when trained on data gathered from other subjects, and we were able to infer the identity of our subjects using the same algorithms. These results suggest that (1) individuals have scanpath routines for judging faces, and that (2) these are diagnostic of that subject, but that (3) at least for the tasks we used, subjects do not converge on the same "ideal" scanpath pattern. Whether universal scanpath patterns exist for a task, we suggest, depends on the task's constraints and the level of expertise of the subject.

Observed, Executed, and Imagined Action Representations can be Decoded From Ventral and Dorsal Areas.

Previous functional magnetic resonance imaging (fMRI) research on action observation has emphasized the role of putative mirror neuron areas such as Broca's area, ventral premotor cortex, and the inferior parietal lobule. However, recent evidence suggests action observation involves many distributed cortical regions, including dorsal premotor and superior parietal cortex. How these different regions relate to traditional mirror neuron areas, and whether traditional mirror neuron areas play a special role in action representation, is unclear. Here we use multi-voxel pattern analysis (MVPA) to show that action representations, including observation, imagery, and execution of reaching movements: (1) are distributed across both dorsal (superior) and ventral (inferior) premotor and parietal areas; (2) can be decoded from areas that are jointly activated by observation, execution, and imagery of reaching movements, even in cases of equal-amplitude blood oxygen level-dependent (BOLD) responses; and (3) can be equally accurately classified from either posterior parietal or frontal (premotor and inferior frontal) regions. These results challenge the presumed dominance of traditional mirror neuron areas such as Broca's area in action observation and action representation more generally. Unlike traditional univariate fMRI analyses, MVPA was able to discriminate between imagined and observed movements from previously indistinguishable BOLD activations in commonly activated regions, suggesting finer-grained distributed patterns of activation.

What evidence supports special processing for faces? A cautionary tale for fMRI interpretation.

We trained a neurocomputational model on six categories of photographic images that were used in a previous fMRI study of object and face processing. Multivariate pattern analyses of the activations elicited in the object-encoding layer of the model yielded results consistent with two previous, contradictory fMRI studies. Findings from one of the studies [Haxby, J. V., Gobbini, M. I., Furey, M. L., Ishai, A., Schouten, J. L., & Pietrini, P. Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science, 293, 2425-2430, 2001] were interpreted as evidence for the object-form topography model. Findings from the other study [Spiridon, M., & Kanwisher, N. How distributed is visual category information in human occipito-temporal cortex? An fMRI study. Neuron, 35, 1157-1165, 2002] were interpreted as evidence for neural processing mechanisms in the fusiform face area that are specialized for faces. Because the model contains no special processing mechanism or specialized architecture for faces and yet it can reproduce the fMRI findings used to support the claim that there are specialized face-processing neurons, we argue that these fMRI results do not actually support that claim. Results from our neurocomputational model therefore constitute a cautionary tale for the interpretation of fMRI data.

Hemispheric asymmetry in perception: a differential encoding account.

Hemispheric asymmetry in the processing of local and global features has been argued to originate from differences in frequency filtering in the two hemispheres, with little neurophysiological support. Here we test the hypothesis that this asymmetry takes place at an encoding stage beyond the sensory level, due to asymmetries in anatomical connections within each hemisphere. We use two simple encoding networks with differential connection structures as models of differential encoding in the two hemispheres based on a hypothesized generalization of neuroanatomical evidence from the auditory modality to the visual modality: The connection structure between columns is more distal in the language areas of the left hemisphere and more local in the homotopic regions in the right hemisphere. We show that both processing differences and differential frequency filtering can arise naturally in this neurocomputational model with neuroanatomically inspired differences in connection structures within the two model hemispheres, suggesting that hemispheric asymmetry in the processing of local and global features may be due to hemispheric asymmetry in connection structure rather than in frequency tuning.

Color-to-grayscale: does the method matter in image recognition?

In image recognition it is often assumed the method used to convert color images to grayscale has little impact on recognition performance. We compare thirteen different grayscale algorithms with four types of image descriptors and demonstrate that this assumption is wrong: not all color-to-grayscale algorithms work equally well, even when using descriptors that are robust to changes in illumination. These methods are tested using a modern descriptor-based image recognition framework, on face, object, and texture datasets, with relatively few training instances. We identify a simple method that generally works best for face and object recognition, and two that work well for recognizing textures.

Evidence and a computational explanation of cultural differences in facial expression recognition.

Facial expressions are crucial to human social communication, but the extent to which they are innate and universal versus learned and culture dependent is a subject of debate. Two studies explored the effect of culture and learning on facial expression understanding. In Experiment 1, Japanese and U.S. participants interpreted facial expressions of emotion. Each group was better than the other at classifying facial expressions posed by members of the same culture. In Experiment 2, this reciprocal in-group advantage was reproduced by a neurocomputational model trained in either a Japanese cultural context or an American cultural context. The model demonstrates how each of us, interacting with others in a particular cultural context, learns to recognize a culture-specific facial expression dialect.