Examining multi-objective deep reinforcement learning frameworks for molecular design.

Drug design and optimization are challenging tasks that call for strategic and efficient exploration of the extremely vast search space. Multiple fragmentation strategies have been proposed in the literature to mitigate the complexity of the molecular search space. From an optimization standpoint, drug design can be considered as a multi-objective optimization problem. Deep reinforcement learning (DRL) frameworks have demonstrated encouraging results in the field of drug design. However, the scalability of these frameworks is impeded by substantial training intervals and inefficient use of sample data. In this paper, we (1) examine the core principles of deep or multi-objective RL methods and their applications in molecular design, (2) analyze the performance of a recent multi-objective DRL-based and fragment-based drug design framework, named DeepFMPO, in a real-world application by incorporating optimization of protein-ligand docking affinity with varying numbers of other objectives, and (3) compare this method with a single-objective variant. Through trials, our results indicate that the DeepFMPO framework (with docking score) can achieve success, however, it suffers from training instability. Our findings encourage additional exploration and improvement of the framework. Potential sources of the framework's instability and suggestions of further modifications to stabilize the framework are discussed.

Adversarial deep evolutionary learning for drug design.

The design of a new therapeutic agent is a time-consuming and expensive process. The rise of machine intelligence provides a grand opportunity of expeditiously discovering novel drug candidates through smart search in the vast molecular structural space. In this paper, we propose a new approach called adversarial deep evolutionary learning (ADEL) to search for novel molecules in the latent space of an adversarial generative model and keep improving the latent representation space. In ADEL, a custom-made adversarial autoencoder (AAE) model is developed and trained under a deep evolutionary learning (DEL) process. This involves an initial training of the AAE model, followed by an integration of multi-objective evolutionary optimization in the continuous latent representation space of the AAE rather than the discrete structural space of molecules. By using the AAE, an arbitrary distribution can be provided to the training of AAE such that the latent representation space is set to that distribution. This allows for a starting latent space from which new samples can be produced. Throughout the process of learning, new samples of high quality are generated after each iteration of training and then added back into the full dataset, therefore, allowing for a more comprehensive procedure of understanding the data structure. This combination of evolving data and continuous learning not only enables improvement in the generative model, but the data as well. By comparing ADEL to the previous work in DEL, we see that ADEL can obtain better property distributions. We show that ADEL is able to design high-quality molecular structures which can be further used for virtual and experimental screenings.

Multi-Objective Drug Design Based on Graph-Fragment Molecular Representation and Deep Evolutionary Learning.

Drug discovery is a challenging process with a huge molecular space to be explored and numerous pharmacological properties to be appropriately considered. Among various drug design protocols, fragment-based drug design is an effective way of constraining the search space and better utilizing biologically active compounds. Motivated by fragment-based drug search for a given protein target and the emergence of artificial intelligence (AI) approaches in this field, this work advances the field of in silico drug design by (1) integrating a graph fragmentation-based deep generative model with a deep evolutionary learning process for large-scale multi-objective molecular optimization, and (2) applying protein-ligand binding affinity scores together with other desired physicochemical properties as objectives. Our experiments show that the proposed method can generate novel molecules with improved property values and binding affinities.