Relative molecule self-attention transformer.

The prediction of molecular properties is a crucial aspect in drug discovery that can save a lot of money and time during the drug design process. The use of machine learning methods to predict molecular properties has become increasingly popular in recent years. Despite advancements in the field, several challenges remain that need to be addressed, like finding an optimal pre-training procedure to improve performance on small datasets, which are common in drug discovery. In our paper, we tackle these problems by introducing Relative Molecule Self-Attention Transformer for molecular representation learning. It is a novel architecture that uses relative self-attention and 3D molecular representation to capture the interactions between atoms and bonds that enrich the backbone model with domain-specific inductive biases. Furthermore, our two-step pretraining procedure allows us to tune only a few hyperparameter values to achieve good performance comparable with state-of-the-art models on a wide selection of downstream tasks.

Zero time waste in pre-trained early exit neural networks.

The problem of reducing processing time of large deep learning models is a fundamental challenge in many real-world applications. Early exit methods strive towards this goal by attaching additional Internal Classifiers (ICs) to intermediate layers of a neural network. ICs can quickly return predictions for easy examples and, as a result, reduce the average inference time of the whole model. However, if a particular IC does not decide to return an answer early, its predictions are discarded, with its computations effectively being wasted. To solve this issue, we introduce Zero Time Waste (ZTW), a novel approach in which each IC reuses predictions returned by its predecessors by (1) adding direct connections between ICs and (2) combining previous outputs in an ensemble-like manner. We conduct extensive experiments across various multiple modes, datasets, and architectures to demonstrate that ZTW achieves a significantly better accuracy vs. inference time trade-off than other early exit methods. On the ImageNet dataset, it obtains superior results over the best baseline method in 11 out of 16 cases, reaching up to 5 percentage points of improvement on low computational budgets.

Feature-Based Interpolation and Geodesics in the Latent Spaces of Generative Models.

Interpolating between points is a problem connected simultaneously with finding geodesics and study of generative models. In the case of geodesics, we search for the curves with the shortest length, while in the case of generative models, we typically apply linear interpolation in the latent space. However, this interpolation uses implicitly the fact that Gaussian is unimodal. Thus, the problem of interpolating in the case when the latent density is non-Gaussian is an open problem. In this article, we present a general and unified approach to interpolation, which simultaneously allows us to search for geodesics and interpolating curves in latent space in the case of arbitrary density. Our results have a strong theoretical background based on the introduced quality measure of an interpolating curve. In particular, we show that maximizing the quality measure of the curve can be equivalently understood as a search of geodesic for a certain redefinition of the Riemannian metric on the space. We provide examples in three important cases. First, we show that our approach can be easily applied to finding geodesics on manifolds. Next, we focus our attention in finding interpolations in pretrained generative models. We show that our model effectively works in the case of arbitrary density. Moreover, we can interpolate in the subset of the space consisting of data possessing a given feature. The last case is focused on finding interpolation in the space of chemical compounds.

Docking-based generative approaches in the search for new drug candidates.

Despite the popularity of virtual screening (VS) of existing compound libraries, the search for new potential drug candidates also takes advantage of generative protocols, where new compound suggestions are enumerated using various algorithms. To increase the activity potency of generative approaches, they have recently been coupled with molecular docking, a leading methodology of structure-based drug design (SBDD). In this review, we summarize progress since docking-based generative models emerged. We propose a new taxonomy for these methods and discuss their importance for the field of computer-aided drug design (CADD). In addition, we discuss the most promising directions for the further development of generative protocols coupled with docking.

Decoding working memory-related information from repeated psychophysiological EEG experiments using convolutional and contrastive neural networks.

Objective.Extracting reliable information from electroencephalogram (EEG) is difficult because the low signal-to-noise ratio and significant intersubject variability seriously hinder statistical analyses. However, recent advances in explainable machine learning open a new strategy to address this problem.Approach.The current study evaluates this approach using results from the classification and decoding of electrical brain activity associated with information retention. We designed four neural network models differing in architecture, training strategies, and input representation to classify single experimental trials of a working memory task.Main results.Our best models achieved an accuracy (ACC) of 65.29 ± 0.76 and Matthews correlation coefficient of 0.288 ± 0.018, outperforming the reference model trained on the same data. The highest correlation between classification score and behavioral performance was 0.36 (p= 0.0007). Using analysis of input perturbation, we estimated the importance of EEG channels and frequency bands in the task at hand. The set of essential features identified for each network varies. We identified a subset of features common to all models that identified brain regions and frequency bands consistent with current neurophysiological knowledge of the processes critical to attention and working memory. Finally, we proposed sanity checks to examine further the robustness of each model's set of features.Significance.Our results indicate that explainable deep learning is a powerful tool for decoding information from EEG signals. It is crucial to train and analyze a range of models to identify stable and reliable features. Our results highlight the need for explainable modeling as the model with the highest ACC appeared to use residual artifactual activity.

Neural spatio-temporal patterns of information processing related to cognitive conflict and correct or false recognitions.

Using a visual short-term memory task and employing a new methodological approach, we analyzed neural responses from the perspective of the conflict level and correctness/erroneous over a longer time window. Sixty-five participants performed the short-term memory task in the fMRI scanner. We explore neural spatio-temporal patterns of information processing in the context of correct or erroneous response and high or low level of cognitive conflict using classical fMRI analysis, surface-based cortical data, temporal analysis of interpolated mean activations, and machine learning classifiers. Our results provide evidence that information processing dynamics during the retrieval process vary depending on the correct or false recognition-for stimuli inducing a high level of cognitive conflict and erroneous response, information processing is prolonged. The observed phenomenon may be interpreted as the manifestation of the brain's preparation for future goal-directed action.

2D SIFt: a matrix of ligand-receptor interactions.

Depicting a ligand-receptor complex via Interaction Fingerprints has been shown to be both a viable data visualization and an analysis tool. The spectrum of its applications ranges from simple visualization of the binding site through analysis of molecular dynamics runs, to the evaluation of the homology models and virtual screening. Here we present a novel tool derived from the Structural Interaction Fingerprints providing a detailed and unique insight into the interactions between receptor and specific regions of the ligand (grouped into pharmacophore features) in the form of a matrix, a 2D-SIFt descriptor. The provided implementation is easy to use and extends the python library, allowing the generation of interaction matrices and their manipulation (reading and writing as well as producing the average 2D-SIFt). The library for handling the interaction matrices is available via repository http://bitbucket.org/zchl/sift2d .

Machine Learning-Based Identification of Suicidal Risk in Patients With Schizophrenia Using Multi-Level Resting-State fMRI Features.

BACKGROUND: Some studies suggest that as much as 40% of all causes of death in a group of patients with schizophrenia can be attributed to suicides and compared with the general population, patients with schizophrenia have an 8.5-fold greater suicide risk (SR). There is a vital need for accurate and reliable methods to predict the SR among patients with schizophrenia based on biological measures. However, it is unknown whether the suicidal risk in schizophrenia can be related to alterations in spontaneous brain activity, or if the resting-state functional magnetic resonance imaging (rsfMRI) measures can be used alongside machine learning (ML) algorithms in order to identify patients with SR. METHODS: Fifty-nine participants including patients with schizophrenia with and without SR as well as age and gender-matched healthy underwent 13 min resting-state functional magnetic resonance imaging. Both static and dynamic indexes of the amplitude of low-frequency fluctuation (ALFF), the fractional amplitude of low-frequency fluctuations (fALFF), regional homogeneity as well as functional connectivity (FC) were calculated and used as an input for five machine learning algorithms: Gradient boosting (GB), LASSO, Logistic Regression (LR), Random Forest and Support Vector Machine. RESULTS: All groups revealed different intra-network functional connectivity in ventral DMN and anterior SN. The best performance was reached for the LASSO applied to FC with an accuracy of 70% and AUROC of 0.76 (p < 0.05). Significant classification ability was also reached for GB and LR using fALFF and ALFF measures. CONCLUSION: Our findings suggest that SR in schizophrenia can be seen on the level of DMN and SN functional connectivity alterations. ML algorithms were able to significantly differentiate SR patients. Our results could be useful in developing neuromarkers of SR in schizophrenia based on non-invasive rsfMRI.