Progressive transfer learning for advancing machine learning-based reduced-order modeling.

To maximize knowledge transfer and improve the data requirement for data-driven machine learning (ML) modeling, a progressive transfer learning for reduced-order modeling (p-ROM) framework is proposed. A key concept of p-ROM is to selectively transfer knowledge from previously trained ML models and effectively develop a new ML model(s) for unseen tasks by optimizing information gates in hidden layers. The p-ROM framework is designed to work with any type of data-driven ROMs. For demonstration purposes, we evaluate the p-ROM with specific Barlow Twins ROMs (p-BT-ROMs) to highlight how progress learning can apply to multiple topological and physical problems with an emphasis on a small training set regime. The proposed p-BT-ROM framework has been tested using multiple examples, including transport, flow, and solid mechanics, to illustrate the importance of progressive knowledge transfer and its impact on model accuracy with reduced training samples. In both similar and different topologies, p-BT-ROM achieves improved model accuracy with much less training data. For instance, p-BT-ROM with four-parent (i.e., pre-trained models) outperforms the no-parent counterpart trained on data nine times larger. The p-ROM framework is poised to significantly enhance the capabilities of ML-based ROM approaches for scientific and engineering applications by mitigating data scarcity through progressively transferring knowledge.

G Protein-Coupled Receptor-Ligand Pose and Functional Class Prediction.

G protein-coupled receptor (GPCR) transmembrane protein family members play essential roles in physiology. Numerous pharmaceuticals target GPCRs, and many drug discovery programs utilize virtual screening (VS) against GPCR targets. Improvements in the accuracy of predicting new molecules that bind to and either activate or inhibit GPCR function would accelerate such drug discovery programs. This work addresses two significant research questions. First, do ligand interaction fingerprints provide a substantial advantage over automated methods of binding site selection for classical docking? Second, can the functional status of prospective screening candidates be predicted from ligand interaction fingerprints using a random forest classifier? Ligand interaction fingerprints were found to offer modest advantages in sampling accurate poses, but no substantial advantage in the final set of top-ranked poses after scoring, and, thus, were not used in the generation of the ligand-receptor complexes used to train and test the random forest classifier. A binary classifier which treated agonists, antagonists, and inverse agonists as active and all other ligands as inactive proved highly effective in ligand function prediction in an external test set of GPR31 and TAAR2 candidate ligands with a hit rate of 82.6% actual actives within the set of predicted actives.

Discovery of Novel TLR7 Agonists as Systemic Agent for Combination With aPD1 for Use in Immuno-oncology.

We have designed and developed novel and selective TLR7 agonists that exhibited potent receptor activity in a cell-based reporter assay. In vitro, these agonists significantly induced secretion of cytokines IL-6, IL-1ß, IL-10, TNFa, IFNa, and IP-10 in human and mouse whole blood. Pharmacokinetic and pharmacodynamic studies in mice showed a significant secretion of IFNα and TNFα cytokines. When combined with aPD1 in a CT-26 tumor model, the lead compound showed strong synergistic antitumor activity with complete tumor regression in 8/10 mice dosed using the intravenous route. Structure-activity relationship studies enabled by structure-based designs of TLR7 agonists are disclosed.

Addressing quantum's "fine print" with efficient state preparation and information extraction for quantum algorithms and geologic fracture networks.

Quantum algorithms provide an exponential speedup for solving certain classes of linear systems, including those that model geologic fracture flow. However, this revolutionary gain in efficiency does not come without difficulty. Quantum algorithms require that problems satisfy not only algorithm-specific constraints, but also application-specific ones. Otherwise, the quantum advantage carefully attained through algorithmic ingenuity can be entirely negated. Previous work addressing quantum algorithms for geologic fracture flow has illustrated core algorithmic approaches while incrementally removing assumptions. This work addresses two further requirements for solving geologic fracture flow systems with quantum algorithms: efficient system state preparation and efficient information extraction. Our approach to addressing each is consistent with an overall exponential speed-up.

Quantum algorithms for geologic fracture networks.

Solving large systems of equations is a challenge for modeling natural phenomena, such as simulating subsurface flow. To avoid systems that are intractable on current computers, it is often necessary to neglect information at small scales, an approach known as coarse-graining. For many practical applications, such as flow in porous, homogenous materials, coarse-graining offers a sufficiently-accurate approximation of the solution. Unfortunately, fractured systems cannot be accurately coarse-grained, as critical network topology exists at the smallest scales, including topology that can push the network across a percolation threshold. Therefore, new techniques are necessary to accurately model important fracture systems. Quantum algorithms for solving linear systems offer a theoretically-exponential improvement over their classical counterparts, and in this work we introduce two quantum algorithms for fractured flow. The first algorithm, designed for future quantum computers which operate without error, has enormous potential, but we demonstrate that current hardware is too noisy for adequate performance. The second algorithm, designed to be noise resilient, already performs well for problems of small to medium size (order 10-1000 nodes), which we demonstrate experimentally and explain theoretically. We expect further improvements by leveraging quantum error mitigation and preconditioning.

Physics-embedded inverse analysis with algorithmic differentiation for the earth's subsurface.

Inverse analysis has been utilized to understand unknown underground geological properties by matching the observational data with simulators. To overcome the underconstrained nature of inverse problems and achieve good performance, an approach is presented with embedded physics and a technique known as algorithmic differentiation. We use a physics-embedded generative model, which takes statistically simple parameters as input and outputs subsurface properties (e.g., permeability or P-wave velocity), that embeds physical knowledge of the subsurface properties into inverse analysis and improves its performance. We tested the application of this approach on four geologic problems: two heterogeneous hydraulic conductivity fields, a hydraulic fracture network, and a seismic inversion for P-wave velocity. This physics-embedded inverse analysis approach consistently characterizes these geological problems accurately. Furthermore, the excellent performance in matching the observational data demonstrates the reliability of the proposed method. Moreover, the application of algorithmic differentiation makes this an easy and fast approach to inverse analysis when dealing with complicated geological structures.

Identification and characterization of select oxysterols as ligands for GPR17.

BACKGROUND AND PURPOSE: G-protein coupled receptor 17 (GPR17) is an orphan receptor involved in the process of myelination, due to its ability to inhibit the maturation of oligodendrocyte progenitor cells (OPCs) into myelinating oligodendrocytes. Despite multiple claims that the biological ligand has been identified, it remains an orphan receptor. EXPERIMENTAL APPROACH: Seventy-seven oxysterols were screened in a cell-free [35 S]GTPγS binding assay using membranes from cells expressing GPR17. The positive hits were characterized using adenosine 3',5' cyclic monophosphate (cAMP), inositol monophosphate (IP1) and calcium mobilization assays, with results confirmed in rat primary oligodendrocytes. Rat and pig brain extracts were separated by high-performance liquid chromatography (HPLC) and endogenous activator(s) were identified in receptor activation assays. Gene expression studies of GPR17, and CYP46A1 (cytochrome P450 family 46 subfamily A member 1) enzymes responsible for the conversion of cholesterol into specific oxysterols, were performed using quantitative real-time PCR. KEY RESULTS: Five oxysterols were able to stimulate GPR17 activity, including the brain cholesterol, 24(S)-hydroxycholesterol (24S-HC). A specific brain fraction from rat and pig extracts containing 24S-HC activates GPR17 in vitro. Expression of Gpr17 during mouse brain development correlates with the expression of Cyp46a1 and the levels of 24S-HC itself. Other active oxysterols have low brain concentrations below effective ranges. CONCLUSIONS AND IMPLICATIONS: Oxysterols, including but not limited to 24S-HC, could be physiological activators for GPR17 and thus potentially regulate OPC differentiation and myelination through activation of the receptor.

Quantum computing and preconditioners for hydrological linear systems.

Modeling hydrological fracture networks is a hallmark challenge in computational earth sciences. Accurately predicting critical features of fracture systems, e.g. percolation, can require solving large linear systems far beyond current or future high performance capabilities. Quantum computers can theoretically bypass the memory and speed constraints faced by classical approaches, however several technical issues must first be addressed. Chief amongst these difficulties is that such systems are often ill-conditioned, i.e. small changes in the system can produce large changes in the solution, which can slow down the performance of linear solving algorithms. We test several existing quantum techniques to improve the condition number, but find they are insufficient. We then introduce the inverse Laplacian preconditioner, which improves the scaling of the condition number of the system from O(N) to [Formula: see text] and admits a quantum implementation. These results are a critical first step in developing a quantum solver for fracture systems, both advancing the state of hydrological modeling and providing a novel real-world application for quantum linear systems algorithms.

Physics-informed machine learning with differentiable programming for heterogeneous underground reservoir pressure management.

Avoiding over-pressurization in subsurface reservoirs is critical for applications like CO[Formula: see text] sequestration and wastewater injection. Managing the pressures by controlling injection/extraction are challenging because of complex heterogeneity in the subsurface. The heterogeneity typically requires high-fidelity physics-based models to make predictions on CO[Formula: see text] fate. Furthermore, characterizing the heterogeneity accurately is fraught with parametric uncertainty. Accounting for both, heterogeneity and uncertainty, makes this a computationally-intensive problem challenging for current reservoir simulators. To tackle this, we use differentiable programming with a full-physics model and machine learning to determine the fluid extraction rates that prevent over-pressurization at critical reservoir locations. We use DPFEHM framework, which has trustworthy physics based on the standard two-point flux finite volume discretization and is also automatically differentiable like machine learning models. Our physics-informed machine learning framework uses convolutional neural networks to learn an appropriate extraction rate based on the permeability field. We also perform a hyperparameter search to improve the model's accuracy. Training and testing scenarios are executed to evaluate the feasibility of using physics-informed machine learning to manage reservoir pressures. We constructed and tested a sufficiently accurate simulator that is 400 000 times faster than the underlying physics-based simulator, allowing for near real-time analysis and robust uncertainty quantification.

Enhancing high-fidelity nonlinear solver with reduced order model.

We propose the use of reduced order modeling (ROM) to reduce the computational cost and improve the convergence rate of nonlinear solvers of full order models (FOM) for solving partial differential equations. In this study, a novel ROM-assisted approach is developed to improve the computational efficiency of FOM nonlinear solvers by using ROM's prediction as an initial guess. We hypothesize that the nonlinear solver will take fewer steps to the converged solutions with an initial guess that is closer to the real solutions. To evaluate our approach, four physical problems with varying degrees of nonlinearity in flow and mechanics have been tested: Richards' equation of water flow in heterogeneous porous media, a contact problem in a hyperelastic material, two-phase flow in layered porous media, and fracture propagation in a homogeneous material. Overall, our approach maintains the FOM's accuracy while speeding up nonlinear solver by 18-73% (through suitable ROM-assisted FOMs). More importantly, the proximity of ROM's prediction to the solution space leads to the improved convergence of FOMs that would have otherwise diverged with default initial guesses. We demonstrate that the ROM's accuracy can impact the computational efficiency with more accurate ROM solutions, resulting in a better cost reduction. We also illustrate that this approach could be used in many FOM discretizations (e.g., finite volume, finite element, or a combination of those). Since our ROMs are data-driven and non-intrusive, the proposed procedure can easily lend itself to any nonlinear physics-based problem.

Reduced order modeling for flow and transport problems with Barlow Twins self-supervised learning.

We propose a unified data-driven reduced order model (ROM) that bridges the performance gap between linear and nonlinear manifold approaches. Deep learning ROM (DL-ROM) using deep-convolutional autoencoders (DC-AE) has been shown to capture nonlinear solution manifolds but fails to perform adequately when linear subspace approaches such as proper orthogonal decomposition (POD) would be optimal. Besides, most DL-ROM models rely on convolutional layers, which might limit its application to only a structured mesh. The proposed framework in this study relies on the combination of an autoencoder (AE) and Barlow Twins (BT) self-supervised learning, where BT maximizes the information content of the embedding with the latent space through a joint embedding architecture. Through a series of benchmark problems of natural convection in porous media, BT-AE performs better than the previous DL-ROM framework by providing comparable results to POD-based approaches for problems where the solution lies within a linear subspace as well as DL-ROM autoencoder-based techniques where the solution lies on a nonlinear manifold; consequently, bridges the gap between linear and nonlinear reduced manifolds. We illustrate that a proficient construction of the latent space is key to achieving these results, enabling us to map these latent spaces using regression models. The proposed framework achieves a relative error of 2% on average and 12% in the worst-case scenario (i.e., the training data is small, but the parameter space is large.). We also show that our framework provides a speed-up of [Formula: see text] times, in the best case, and [Formula: see text] times on average compared to a finite element solver. Furthermore, this BT-AE framework can operate on unstructured meshes, which provides flexibility in its application to standard numerical solvers, on-site measurements, experimental data, or a combination of these sources.

Quantum annealing algorithms for Boolean tensor networks.

Quantum annealers manufactured by D-Wave Systems, Inc., are computational devices capable of finding high-quality heuristic solutions of NP-hard problems. In this contribution, we explore the potential and effectiveness of such quantum annealers for computing Boolean tensor networks. Tensors offer a natural way to model high-dimensional data commonplace in many scientific fields, and representing a binary tensor as a Boolean tensor network is the task of expressing a tensor containing categorical (i.e., [Formula: see text]) values as a product of low dimensional binary tensors. A Boolean tensor network is computed by Boolean tensor decomposition, and it is usually not exact. The aim of such decomposition is to minimize the given distance measure between the high-dimensional input tensor and the product of lower-dimensional (usually three-dimensional) tensors and matrices representing the tensor network. In this paper, we introduce and analyze three general algorithms for Boolean tensor networks: Tucker, Tensor Train, and Hierarchical Tucker networks. The computation of a Boolean tensor network is reduced to a sequence of Boolean matrix factorizations, which we show can be expressed as a quadratic unconstrained binary optimization problem suitable for solving on a quantum annealer. By using a novel method we introduce called parallel quantum annealing, we demonstrate that Boolean tensor's with up to millions of elements can be decomposed efficiently using a DWave 2000Q quantum annealer.

A machine learning framework for rapid forecasting and history matching in unconventional reservoirs.

We present a novel workflow for forecasting production in unconventional reservoirs using reduced-order models and machine-learning. Our physics-informed machine-learning workflow addresses the challenges to real-time reservoir management in unconventionals, namely the lack of data (i.e., the time-frame for which the wells have been producing), and the significant computational expense of high-fidelity modeling. We do this by applying the machine-learning paradigm of transfer learning, where we combine fast, but less accurate reduced-order models with slow, but accurate high-fidelity models. We use the Patzek model (Proc Natl Acad Sci 11:19731-19736, https://doi.org/10.1073/pnas.1313380110 , 2013) as the reduced-order model to generate synthetic production data and supplement this data with synthetic production data obtained from high-fidelity discrete fracture network simulations of the site of interest. Our results demonstrate that training with low-fidelity models is not sufficient for accurate forecasting, but transfer learning is able to augment the knowledge and perform well once trained with the small set of results from the high-fidelity model. Such a physics-informed machine-learning (PIML) workflow, grounded in physics, is a viable candidate for real-time history matching and production forecasting in a fractured shale gas reservoir.

Discovery and Preclinical Pharmacology of an Oral Bromodomain and Extra-Terminal (BET) Inhibitor Using Scaffold-Hopping and Structure-Guided Drug Design.

Inhibition of the bromodomain and extra-terminal (BET) family of adaptor proteins is an attractive strategy for targeting transcriptional regulation of key oncogenes, such as c-MYC. Starting with the screening hit 1, a combination of structure-activity relationship and protein structure-guided drug design led to the discovery of a differently oriented carbazole 9 with favorable binding to the tryptophan, proline, and phenylalanine (WPF) shelf conserved in the BET family. Identification of an additional lipophilic pocket and functional group optimization to optimize pharmacokinetic (PK) properties culminated in the discovery of 18 (BMS-986158) with excellent potency in binding and functional assays. On the basis of its favorable PK profile and robust in vivo activity in a panel of hematologic and solid tumor models, BMS-986158 was selected as a candidate for clinical evaluation.

Substituted diaryl ether compounds as retinoic acid-related orphan Receptor-γt (RORγt) agonists.

The discovery of a series of substituted diarylether compounds as retinoic acid related orphan receptor γt (RORγt) agonists is described. Compound 1 was identified from deck mining as a RORγt agonist. Hit-to-lead optimization led to the identification of lead compound 5, which possesses improved potency (10x). Extensive SAR exploration led to the identification of a potent and selective compound 22, that demonstrated an improved pharmacokinetic profile and a dose-dependent pharmacodynamic response. However, when dosed in a MC38 syngeneic tumor model, no evidence of efficacy was observed. ©2020 Elsevier Science Ltd. All rights reserved.

Reverse annealing for nonnegative/binary matrix factorization.

It was recently shown that quantum annealing can be used as an effective, fast subroutine in certain types of matrix factorization algorithms. The quantum annealing algorithm performed best for quick, approximate answers, but performance rapidly plateaued. In this paper, we utilize reverse annealing instead of forward annealing in the quantum annealing subroutine for nonnegative/binary matrix factorization problems. After an initial global search with forward annealing, reverse annealing performs a series of local searches that refine existing solutions. The combination of forward and reverse annealing significantly improves performance compared to forward annealing alone for all but the shortest run times.

A framework for data-driven solution and parameter estimation of PDEs using conditional generative adversarial networks.

Here we employ and adapt the image-to-image translation concept based on conditional generative adversarial networks (cGAN) for learning a forward and an inverse solution operator of partial differential equations (PDEs). We focus on steady-state solutions of coupled hydromechanical processes in heterogeneous porous media and present the parameterization of the spatially heterogeneous coefficients, which is exceedingly difficult using standard reduced-order modeling techniques. We show that our framework provides a speed-up of at least 2,000 times compared to a finite-element solver and achieves a relative root-mean-square error (r.m.s.e.) of less than 2% for forward modeling. For inverse modeling, the framework estimates the heterogeneous coefficients, given an input of pressure and/or displacement fields, with a relative r.m.s.e. of less than 7%, even for cases where the input data are incomplete and contaminated by noise. The framework also provides a speed-up of 120,000 times compared to a Gaussian prior-based inverse modeling approach while also delivering more accurate results.

Transient flow modeling in fractured media using graphs.

We describe a method to simulate transient fluid flows in fractured media using an approach based on graph theory. Our approach builds on past work where the graph-based approach was successfully used to simulate steady-state fluid flows in fractured media. We find a mean computational speedup of the order of 1400 from an ensemble of a 100 discrete fracture networks in contrast to the O(10^{4}) speedup that was obtained for steady-state flows earlier. However, the transient flows considered here involve an additional degree of complexity that was not present in the steady-state flows considered previously with a graph-based approach, that of time marching and solution of the flow equations within a time-stepping scheme. We verify our method with an analytical test case and demonstrate its use on a practical problem related to fluid flows in hydraulically fractured reservoirs. By enabling the study of transient flows, we create an opportunity for a wide set of possibilities where a steady-state approximation is not sufficient, such as the example motivated by hydraulic fracturing that we present here. This work validates the concept that graphs are able to reliably capture the topological properties of the fracture network and serve as effective surrogates in an uncertainty-quantification framework.

Substituted benzyloxytricyclic compounds as retinoic acid-related orphan receptor gamma t (RORγt) agonists.

Substituted benzyloxy aryl compound 2 was identified as an RORγt agonist. Structure based drug design efforts resulted in a potent and selective tricyclic compound 19 which, when administered orally in an MC38 mouse tumor model, demonstrated a desired pharmacokinetic profile as well as a dose-dependent pharmacodynamic response. However, no perceptible efficacy was observed in this tumor model at the doses investigated.