Model-Informed Reinforcement Learning for Enabling Precision Dosing Via Adaptive Dosing.

Precision dosing, the tailoring of drug doses to optimize therapeutic benefits and minimize risks in each patient, is essential for drugs with a narrow therapeutic window and severe adverse effects. Adaptive dosing strategies extend the precision dosing concept to time-varying treatments which require sequential dose adjustments based on evolving patient conditions. Reinforcement learning (RL) naturally fits this paradigm: it perfectly mimics the sequential decision-making process where clinicians adapt dose administration based on patient response and evolution monitoring. This paper aims to investigate the potentiality of coupling RL with population PK/PD models to develop precision dosing algorithms, reviewing the most relevant works in the field. Case studies in which PK/PD models were integrated within RL algorithms as simulation engine to predict consequences of any dosing action have been considered and discussed. They mainly concern propofol-induced anesthesia, anticoagulant therapy with warfarin and a variety of anticancer treatments differing for administered agents and/or monitored biomarkers. The resulted picture highlights a certain heterogeneity in terms of precision dosing approaches, applied methodologies, and degree of adherence to the clinical domain. In addition, a tutorial on how a precision dosing problem should be formulated in terms of the key elements composing the RL framework (i.e., system state, agent actions and reward function), and on how PK/PD models could enhance RL approaches is proposed for readers interested in delving in this field. Overall, the integration of PK/PD models into a RL-framework holds great promise for precision dosing, but further investigations and advancements are still needed to address current limitations and extend the applicability of this methodology to drugs requiring adaptive dosing strategies.

Predicting ADMET Properties from Molecule SMILE: A Bottom-Up Approach Using Attention-Based Graph Neural Networks.

Understanding the pharmacokinetics, safety and efficacy of candidate drugs is crucial for their success. One key aspect is the characterization of absorption, distribution, metabolism, excretion and toxicity (ADMET) properties, which require early assessment in the drug discovery and development process. This study aims to present an innovative approach for predicting ADMET properties using attention-based graph neural networks (GNNs). The model utilizes a graph-based representation of molecules directly derived from Simplified Molecular Input Line Entry System (SMILE) notation. Information is processed sequentially, from substructures to the whole molecule, employing a bottom-up approach. The developed GNN is tested and compared with existing approaches using six benchmark datasets and by encompassing regression (lipophilicity and aqueous solubility) and classification (CYP2C9, CYP2C19, CYP2D6 and CYP3A4 inhibition) tasks. Results show the effectiveness of our model, which bypasses the computationally expensive retrieval and selection of molecular descriptors. This approach provides a valuable tool for high-throughput screening, facilitating early assessment of ADMET properties and enhancing the likelihood of drug success in the development pipeline.

Reinforcement Learning and PK-PD Models Integration to Personalize the Adaptive Dosing Protocol of Erdafitinib in Patients with Metastatic Urothelial Carcinoma.

The integration of pharmacokinetic-pharmacodynamic (PK-PD) modeling and simulations with artificial intelligence/machine learning algorithms is one of the most attractive areas of the pharmacometric research. These hybrid techniques are currently under investigation to perform several tasks, among which precision dosing. In this scenario, this paper presents and evaluates a new framework embedding PK-PD models into a reinforcement learning (RL) algorithm, Q-learning (QL), to personalize pharmacological treatment. Each patient is represented with a set of PK-PD parameters and has a personal QL agent which optimizes the individual treatment. In the training phase, leveraging PK-PD simulations, the QL agent assesses different actions, defined consistently with the clinical knowledge to consider only plausible dose-adjustments, in order to find the optimal rules. The proposed framework was evaluated to optimize the erdafitinib treatment in patients with metastatic urothelial carcinoma. This drug was approved by the US Food and Drug Administration (FDA) with a dose-adaptive protocol based on monitoring the levels of serum phosphate, which represent a biomarker of both treatment efficacy and toxicity. To evaluate the flexibility of the methodology, a heterogeneous virtual population of 141 patients was generated using an erdafitinib population PK (PopPK)-PD literature model. For each patient, treatment response was simulated by using both QL-optimized protocol and the clinical one. QL agents outperform the approved dose-adaptive rules, increasing more than 10% the efficacy and the safety of treatment at each end point. Results confirm the great potentialities of the integration of PopPK-PD models and RL algorithms to optimize precision dosing tasks.

A two-stages global sensitivity analysis by using the δ sensitivity index in presence of correlated inputs: application on a tumor growth inhibition model based on the dynamic energy budget theory.

Global sensitivity analysis (GSA) evaluates the impact of variability and/or uncertainty of the model parameters on given model outputs. GSA is useful for assessing the quality of Pharmacometric model inference. Indeed, model parameters can be affected by high (estimation) uncertainty due to the sparsity of data. Independence between model parameters is a common assumption of GSA methods. However, ignoring (known) correlations between parameters may alter model predictions and, then, GSA results. To address this issue, a novel two-stages GSA technique based on the δ index, which is well-defined also in presence of correlated parameters, is here proposed. In the first step, statistical dependencies are neglected to identify parameters exerting causal effects. Correlations are introduced in the second step to consider the real distribution of the model output and investigate also the 'indirect' effects due to the correlation structure. The proposed two-stages GSA strategy was applied, as case study, to a preclinical tumor-in-host-growth inhibition model based on the Dynamic Energy Budget theory. The aim is to evaluate the impact of the model parameter estimate uncertainty (including correlations) on key model-derived metrics: the drug threshold concentration for tumor eradication, the tumor volume doubling time and a new index evaluating the drug efficacy-toxicity trade-off. This approach allowed to rank parameters according to their impact on the output, discerning whether a parameter mainly exerts a causal or 'indirect' effect. Thus, it was possible to identify uncertainties that should be necessarily reduced to obtain robust predictions for the outputs of interest.

mvLognCorrEst: an R package for sampling from multivariate lognormal distributions and estimating correlations from uncomplete correlation matrix.

BACKGROUND AND OBJECTIVE: Pharmacometrics (PMX) is a quantitative discipline which supports decision-making processes in all stages of drug development. PMX leverages Modeling and Simulations (M&S), which represents a powerful tool to characterize and predict the behavior and the effect of a drug. M&S-based methods, such as Sensitivity Analysis (SA) and Global Sensitivity Analysis (GSA), are gaining interest in PMX as they allow the evaluation of model-informed inference quality. Simulations should be correctly designed to obtain reliable results. Neglecting correlations between model parameters can significantly alter the results of simulations. However, the introduction of a correlation structure between model parameters can cause some issues. Sampling from a multivariate lognormal distribution, which is the typically distribution assumed for PMX model parameters, is not straightforward when a correlation structure is introduced. Indeed, correlations need to respect some constraints which depend by the CVs (i.e., coefficients of variation) of lognormal variables. In addition, when correlation matrices have some unspecified values, they should be properly fixed preserving the positive semi-definiteness of the correlation structure. In this paper, we present mvLognCorrEst, an R package developed to address these issues. METHODS: The proposed sampling strategy was based on reconducting the extraction from the multivariate lognormal distribution of interest to the underlying Normal distribution. However, with high lognormal CVs, a positive semi-definite Normal covariance matrix cannot be obtained due to the violation of some theoretical constraints. In these cases, the Normal covariance matrix was approximated to its nearest positive definite matrix using Frobenius norm as matrix distance. For the estimation of unknown correlations terms, the graph theory was used to represent the correlation structure as weighed undirected graph. Plausible value ranges for the unspecified correlations were derived considering the paths between variables. Then, their estimation was performed by solving a constrained optimization problem. RESULTS: Package functions are presented and applied on a real case study, that is the GSA of a PMX model that has been recently developed to support preclinical oncological studies. CONCLUSIONS: mvLognCorrEst package is an R tool to support simulation-based analysis for which sampling from multivariate lognormal distributions with correlated variables and/or estimation of partially defined correlation matrix are required.

A Dynamic Energy Budget (DEB) based modeling framework to describe tumor-in-host growth inhibition and cachexia onset during anticancer treatment in in vivo xenograft studies.

Cancer anorexia-cachexia syndrome (CACS) is a very severe complication of cancer for which an adequate therapeutic strategy has not yet been defined. Recently, a notable number of new animal models of human CACS has been made available for translational purposes. Under the assumption that tumor-induced adaptations of host metabolism and tumor-host energetic competition play a major role in CACS (together with possible toxicities induced by the anticancer treatment), we developed a new Dynamic Energy Budget (DEB)-based framework, modeling tumor-in-host growth dynamics and cachexia onset in preclinical animal models during anticancer treatments. The tumor-in-host modeling approach was successfully applied on a multitude of in vivo preclinical studies involving different host species, tumor cell lines, type of anticancer agents and experimental settings among which standard xenograft studies. Obtained results strongly suggested the adoption of the tumor-in-host DEB-based approach in the preclinical oncological setting for a joint assessment of drug efficacy and toxicity and for a better design of the experiments. Further applications of the DEB-based approach to the context of poly-targeted combination therapy, anti-cachectic treatments and preclinical to clinical translation are under investigation with extremely encouraging preliminary results.

Staggered Herringbone Microfluid Device for the Manufacturing of Chitosan/TPP Nanoparticles: Systematic Optimization and Preliminary Biological Evaluation.

Chitosan nanoparticles (CS NPs) showed promising results in drug, vaccine and gene delivery for the treatment of various diseases. The considerable attention towards CS was owning to its outstanding biological properties, however, the main challenge in the application of CS NPs was faced during their size-controlled synthesis. Herein, ionic gelation reaction between CS and sodium tripolyphosphate (TPP), a widely used and safe CS cross-linker for biomedical application, was exploited by a microfluidic approach based on a staggered herringbone micromixer (SHM) for the synthesis of TPP cross-linked CS NPs (CS/TPP NPs). Screening design of experiments was applied to systematically evaluate the main process and formulative factors affecting CS/TPP NPs physical properties (mean size and size distribution). Effectiveness of the SHM-assisted manufacturing process was confirmed by the preliminary evaluation of the biological performance of the optimized CS/TPP NPs that were internalized in the cytosol of human mesenchymal stem cells through clathrin-mediated mechanism. Curcumin, selected as a challenging model drug, was successfully loaded into CS/TPP NPs (EE% > 70%) and slowly released up to 48 h via the diffusion mechanism. Finally, the comparison with the conventional bulk mixing method corroborated the efficacy of the microfluidics-assisted method due to the precise control of mixing at microscales.