Reinforcement learning informs optimal treatment strategies to limit antibiotic resistance.

Antimicrobial resistance was estimated to be associated with 4.95 million deaths worldwide in 2019. It is possible to frame the antimicrobial resistance problem as a feedback-control problem. If we could optimize this feedback-control problem and translate our findings to the clinic, we could slow, prevent, or reverse the development of high-level drug resistance. Prior work on this topic has relied on systems where the exact dynamics and parameters were known a priori. In this study, we extend this work using a reinforcement learning (RL) approach capable of learning effective drug cycling policies in a system defined by empirically measured fitness landscapes. Crucially, we show that it is possible to learn effective drug cycling policies despite the problems of noisy, limited, or delayed measurement. Given access to a panel of 15 [Formula: see text]-lactam antibiotics with which to treat the simulated Escherichia coli population, we demonstrate that RL agents outperform two naive treatment paradigms at minimizing the population fitness over time. We also show that RL agents approach the performance of the optimal drug cycling policy. Even when stochastic noise is introduced to the measurements of population fitness, we show that RL agents are capable of maintaining evolving populations at lower growth rates compared to controls. We further tested our approach in arbitrary fitness landscapes of up to 1,024 genotypes. We show that minimization of population fitness using drug cycles is not limited by increasing genome size. Our work represents a proof-of-concept for using AI to control complex evolutionary processes.

Frequency-dependent ecological interactions increase the prevalence, and shape the distribution, of pre-existing drug resistance.

The evolution of resistance remains one of the primary challenges for modern medicine from infectious diseases to cancers. Many of these resistance-conferring mutations often carry a substantial fitness cost in the absence of treatment. As a result, we would expect these mutants to undergo purifying selection and be rapidly driven to extinction. Nevertheless, pre-existing resistance is frequently observed from drug-resistant malaria to targeted cancer therapies in non-small cell lung cancer (NSCLC) and melanoma. Solutions to this apparent paradox have taken several forms from spatial rescue to simple mutation supply arguments. Recently, in an evolved resistant NSCLC cell line, we found that frequency-dependent ecological interactions between ancestor and resistant mutant ameliorate the cost of resistance in the absence of treatment. Here, we hypothesize that frequency-dependent ecological interactions in general play a major role in the prevalence of pre-existing resistance. We combine numerical simulations with robust analytical approximations to provide a rigorous mathematical framework for studying the effects of frequency-dependent ecological interactions on the evolutionary dynamics of pre-existing resistance. First, we find that ecological interactions significantly expand the parameter regime under which we expect to observe pre-existing resistance. Next, even when positive ecological interactions between mutants and ancestors are rare, these resistant clones provide the primary mode of evolved resistance because even weak positive interaction leads to significantly longer extinction times. We then find that even in the case where mutation supply alone is sufficient to predict pre-existing resistance, frequency-dependent ecological forces still contribute a strong evolutionary pressure that selects for increasingly positive ecological effects (negative frequency-dependent selection). Finally, we genetically engineer several of the most common clinically observed resistance mechanisms to targeted therapies in NSCLC, a treatment notorious for pre-existing resistance. We find that each engineered mutant displays a positive ecological interaction with their ancestor. As a whole, these results suggest that frequency-dependent ecological effects can play a crucial role in shaping the evolutionary dynamics of pre-existing resistance.

The evolution of diverse antimicrobial responses in vancomycin-intermediate Staphylococcus aureus and its therapeutic implications.

Staphylococcus aureus causes endocarditis, osteomyelitis, and bacteremia. Clinicians often prescribe vancomycin as an empiric therapy to account for methicillin-resistant S. aureus (MRSA) and narrow treatment based on culture susceptibility results. However, these results reflect a single time point before empiric treatment and represent a limited subset of the total bacterial population within the patient. Thus, while they may indicate that the infection is susceptible to a particular drug, this recommendation may no longer be accurate during therapy. Here, we addressed how antibiotic susceptibility changes over time by accounting for evolution. We evolved 18 methicillin-susceptible S. aureus (MSSA) populations under increasing vancomycin concentrations until they reached intermediate resistance levels. Sequencing revealed parallel mutations that affect cell membrane stress response and cell-wall biosynthesis. The populations exhibited repeated cross-resistance to daptomycin and varied responses to meropenem, gentamicin, and nafcillin. We accounted for this variability by deriving likelihood estimates that express a population's probability of exhibiting a drug response following vancomycin treatment. Our results suggest antistaphylococcal penicillins are preferable first-line treatments for MSSA infections but also highlight the inherent uncertainty that evolution poses to effective therapies. Infections may take varied evolutionary paths; therefore, considering evolution as a probabilistic process should inform our therapeutic choices.

Drug dependence in cancer is exploitable by optimally constructed treatment holidays.

Cancers with acquired resistance to targeted therapy can become simultaneously dependent on the presence of the targeted therapy drug for survival, suggesting that intermittent therapy may slow resistance. However, relatively little is known about which tumours are likely to become dependent and how to schedule intermittent therapy optimally. Here we characterized drug dependence across a panel of over 75 MAPK-inhibitor-resistant BRAFV600E mutant melanoma models at the population and single-clone levels. Melanocytic differentiated models exhibited a much greater tendency to give rise to drug-dependent progeny than their dedifferentiated counterparts. Mechanistically, acquired loss of microphthalmia-associated transcription factor in differentiated melanoma models drives ERK-JunB-p21 signalling to enforce drug dependence. We identified the optimal scheduling of 'drug holidays' using simple mathematical models that we validated across short and long timescales. Without detailed knowledge of tumour characteristics, we found that a simple adaptive therapy protocol can produce near-optimal outcomes using only measurements of total population size. Finally, a spatial agent-based model showed that optimal schedules derived from exponentially growing cells in culture remain nearly optimal in the context of tumour cell turnover and limited environmental carrying capacity. These findings may guide the implementation of improved evolution-inspired treatment strategies for drug-dependent cancers.

Reinforcement Learning informs optimal treatment strategies to limit antibiotic resistance.

Antimicrobial resistance was estimated to be associated with 4.95 million deaths worldwide in 2019. It is possible to frame the antimicrobial resistance problem as a feedback-control problem. If we could optimize this feedback-control problem and translate our findings to the clinic, we could slow, prevent or reverse the development of high-level drug resistance. Prior work on this topic has relied on systems where the exact dynamics and parameters were known a priori. In this study, we extend this work using a reinforcement learning (RL) approach capable of learning effective drug cycling policies in a system defined by empirically measured fitness landscapes. Crucially, we show that is possible to learn effective drug cycling policies despite the problems of noisy, limited, or delayed measurement. Given access to a panel of 15 ß-lactam antibiotics with which to treat the simulated E. coli population, we demonstrate that RL agents outperform two naive treatment paradigms at minimizing the population fitness over time. We also show that RL agents approach the performance of the optimal drug cycling policy. Even when stochastic noise is introduced to the measurements of population fitness, we show that RL agents are capable of maintaining evolving populations at lower growth rates compared to controls. We further tested our approach in arbitrary fitness landscapes of up to 1024 genotypes. We show that minimization of population fitness using drug cycles is not limited by increasing genome size. Our work represents a proof-of-concept for using AI to control complex evolutionary processes.

Measuring competitive exclusion in non-small cell lung cancer.

In this study, we experimentally measure the frequency-dependent interactions between a gefitinib-resistant non-small cell lung cancer population and its sensitive ancestor via the evolutionary game assay. We show that cost of resistance is insufficient to accurately predict competitive exclusion and that frequency-dependent growth rate measurements are required. Using frequency-dependent growth rate data, we then show that gefitinib treatment results in competitive exclusion of the ancestor, while the absence of treatment results in a likely, but not guaranteed, exclusion of the resistant strain. Then, using simulations, we demonstrate that incorporating ecological growth effects can influence the predicted extinction time. In addition, we show that higher drug concentrations may not lead to the optimal reduction in tumor burden. Together, these results highlight the potential importance of frequency-dependent growth rate data for understanding competing populations, both in the laboratory and as we translate adaptive therapy regimens to the clinic.

UV decontamination of personal protective equipment with idle laboratory biosafety cabinets during the COVID-19 pandemic.

Personal protective equipment (PPE) is crucially important to the safety of both patients and medical personnel, particularly in the event of an infectious pandemic. As the incidence of Coronavirus Disease 2019 (COVID-19) increases exponentially in the United States and many parts of the world, healthcare provider demand for these necessities is currently outpacing supply. In the midst of the current pandemic, there has been a concerted effort to identify viable ways to conserve PPE, including decontamination after use. In this study, we outline a procedure by which PPE may be decontaminated using ultraviolet (UV) radiation in biosafety cabinets (BSCs), a common element of many academic, public health, and hospital laboratories. According to the literature, effective decontamination of N95 respirator masks or surgical masks requires UV-C doses of greater than 1 Jcm-2, which was achieved after 4.3 hours per side when placing the N95 at the bottom of the BSCs tested in this study. We then demonstrated complete inactivation of the human coronavirus NL63 on N95 mask material after 15 minutes of UV-C exposure at 61 cm (232 µWcm-2). Our results provide support to healthcare organizations looking for methods to extend their reserves of PPE.

Evolution in alternating environments with tunable interlandscape correlations.

Natural populations are often exposed to temporally varying environments. Evolutionary dynamics in varying environments have been extensively studied, although understanding the effects of varying selection pressures remains challenging. Here, we investigate how cycling between a pair of statistically related fitness landscapes affects the evolved fitness of an asexually reproducing population. We construct pairs of fitness landscapes that share global fitness features but are correlated with one another in a tunable way, resulting in landscape pairs with specific correlations. We find that switching between these landscape pairs, depending on the ruggedness of the landscape and the interlandscape correlation, can either increase or decrease steady-state fitness relative to evolution in single environments. In addition, we show that switching between rugged landscapes often selects for increased fitness in both landscapes, even in situations where the landscapes themselves are anticorrelated. We demonstrate that positively correlated landscapes often possess a shared maximum in both landscapes that allows the population to step through sub-optimal local fitness maxima that often trap single landscape evolution trajectories. Finally, we demonstrate that switching between anticorrelated paired landscapes leads to ergodic-like dynamics where each genotype is populated with nonzero probability, dramatically lowering the steady-state fitness in comparison to single landscape evolution.

Antibiotic interactions shape short-term evolution of resistance in E. faecalis.

Antibiotic combinations are increasingly used to combat bacterial infections. Multidrug therapies are a particularly important treatment option for E. faecalis, an opportunistic pathogen that contributes to high-inoculum infections such as infective endocarditis. While numerous synergistic drug combinations for E. faecalis have been identified, much less is known about how different combinations impact the rate of resistance evolution. In this work, we use high-throughput laboratory evolution experiments to quantify adaptation in growth rate and drug resistance of E. faecalis exposed to drug combinations exhibiting different classes of interactions, ranging from synergistic to suppressive. We identify a wide range of evolutionary behavior, including both increased and decreased rates of growth adaptation, depending on the specific interplay between drug interaction and drug resistance profiles. For example, selection in a dual ß-lactam combination leads to accelerated growth adaptation compared to selection with the individual drugs, even though the resulting resistance profiles are nearly identical. On the other hand, populations evolved in an aminoglycoside and ß-lactam combination exhibit decreased growth adaptation and resistant profiles that depend on the specific drug concentrations. We show that the main qualitative features of these evolutionary trajectories can be explained by simple rescaling arguments that correspond to geometric transformations of the two-drug growth response surfaces measured in ancestral cells. The analysis also reveals multiple examples where resistance profiles selected by drug combinations are nearly growth-optimized along a contour connecting profiles selected by the component drugs. Our results highlight trade-offs between drug interactions and resistance profiles during the evolution of multi-drug resistance and emphasize evolutionary benefits and disadvantages of particular drug pairs targeting enterococci.

Using Selection by Nonantibiotic Stressors to Sensitize Bacteria to Antibiotics.

Evolutionary adaptation of bacteria to nonantibiotic selective forces, such as osmotic stress, has been previously associated with increased antibiotic resistance, but much less is known about potentially sensitizing effects of nonantibiotic stressors. In this study, we use laboratory evolution to investigate adaptation of Enterococcus faecalis, an opportunistic bacterial pathogen, to a broad collection of environmental agents, ranging from antibiotics and biocides to extreme pH and osmotic stress. We find that nonantibiotic selection frequently leads to increased sensitivity to other conditions, including multiple antibiotics. Using population sequencing and whole-genome sequencing of single isolates from the evolved populations, we identify multiple mutations in genes previously linked with resistance to the selecting conditions, including genes corresponding to known drug targets or multidrug efflux systems previously tied to collateral sensitivity. Finally, we hypothesized based on the measured sensitivity profiles that sequential rounds of antibiotic and nonantibiotic selection may lead to hypersensitive populations by harnessing the orthogonal collateral effects of particular pairs of selective forces. To test this hypothesis, we show experimentally that populations evolved to a sequence of linezolid (an oxazolidinone antibiotic) and sodium benzoate (a common preservative) exhibit increased sensitivity to more stressors than adaptation to either condition alone. The results demonstrate how sequential adaptation to drug and nondrug environments can be used to sensitize bacteria to antibiotics and highlight new potential strategies for exploiting shared constraints governing adaptation to diverse environmental challenges.

Pervasive and diverse collateral sensitivity profiles inform optimal strategies to limit antibiotic resistance.

Evolved resistance to one antibiotic may be associated with "collateral" sensitivity to other drugs. Here, we provide an extensive quantitative characterization of collateral effects in Enterococcus faecalis, a gram-positive opportunistic pathogen. By combining parallel experimental evolution with high-throughput dose-response measurements, we measure phenotypic profiles of collateral sensitivity and resistance for a total of 900 mutant-drug combinations. We find that collateral effects are pervasive but difficult to predict because independent populations selected by the same drug can exhibit qualitatively different profiles of collateral sensitivity as well as markedly different fitness costs. Using whole-genome sequencing of evolved populations, we identified mutations in a number of known resistance determinants, including mutations in several genes previously linked with collateral sensitivity in other species. Although phenotypic drug sensitivity profiles show significant diversity, they cluster into statistically similar groups characterized by selecting drugs with similar mechanisms. To exploit the statistical structure in these resistance profiles, we develop a simple mathematical model based on a stochastic control process and use it to design optimal drug policies that assign a unique drug to every possible resistance profile. Stochastic simulations reveal that these optimal drug policies outperform intuitive cycling protocols by maintaining long-term sensitivity at the expense of short-term periods of high resistance. The approach reveals a new conceptual strategy for mitigating resistance by balancing short-term inhibition of pathogen growth with infrequent use of drugs intended to steer pathogen populations to a more vulnerable future state. Experiments in laboratory populations confirm that model-inspired sequences of four drugs reduce growth and slow adaptation relative to naive protocols involving the drugs alone, in pairwise cycles, or in a four-drug uniform cycle.

A metabolic interpretation for the response of cellular autofluorescence to chemical perturbations assessed using spectral phasor analysis.

Analytical approaches for sensing cellular NADH conformation from autofluorescence signals have significance because NADH is a metabolic indicator and endogenous biomarker. Recently, spectral detection of multiple cellular NADH forms during chemically-induced metabolic response was reported, however because NADH is solvatochromic and the spectral change is small, the possibility of a non-metabolic interpretation needs to be considered. Here we investigate the response of UV-excited autofluorescence to a range of well-known chemicals affecting fermentation, respiration, and oxidative-stress pathways in Saccharomyces cerevisiae. The two-component nature of the spectral response is assessed using phasor analysis. By considering a series of physically similar and dissimilar chemicals acting on multiple pathways, we show how the two-component nature of a spectral response is of metabolic origin, indicative of whether a single or several pathways have been affected.

Sensing NADH conformation using phasor analysis on fluorescence spectra.

Phasor analysis on fluorescence signals is a sensitive approach for analyzing multicomponent systems. Initially developed for time-resolved measurements, a spectral version has been used for the rapid identification of regions during the spectral imaging of biological systems. Here we show that quantitative information regarding conformation can be obtained from phasor analysis of fluorescence spectrum shape. Methanol denaturation of NADH and NADH binding to various dehydrogenase proteins are used as model reactions. Thermodynamic constants are calculated and compared with previous studies based on more direct measures of conformation. Next, the quantitative monitoring of UV-excited autofluorescence spectrum shape during chemically-induced metabolic transitions is presented and discussed in terms of NADH-utilizing pathways. Results show how phasor analysis is useful in assessing two-state behavior, and in interpreting autofluorescence as emission from an ensemble of cellular NADH forms.

Population Density Modulates Drug Inhibition and Gives Rise to Potential Bistability of Treatment Outcomes for Bacterial Infections.

The inoculum effect (IE) is an increase in the minimum inhibitory concentration (MIC) of an antibiotic as a function of the initial size of a microbial population. The IE has been observed in a wide range of bacteria, implying that antibiotic efficacy may depend on population density. Such density dependence could have dramatic effects on bacterial population dynamics and potential treatment strategies, but explicit measures of per capita growth as a function of density are generally not available. Instead, the IE measures MIC as a function of initial population size, and population density changes by many orders of magnitude on the timescale of the experiment. Therefore, the functional relationship between population density and antibiotic inhibition is generally not known, leaving many questions about the impact of the IE on different treatment strategies unanswered. To address these questions, here we directly measured real-time per capita growth of Enterococcus faecalis populations exposed to antibiotic at fixed population densities using multiplexed computer-automated culture devices. We show that density-dependent growth inhibition is pervasive for commonly used antibiotics, with some drugs showing increased inhibition and others decreased inhibition at high densities. For several drugs, the density dependence is mediated by changes in extracellular pH, a community-level phenomenon not previously linked with the IE. Using a simple mathematical model, we demonstrate how this density dependence can modulate population dynamics in constant drug environments. Then, we illustrate how time-dependent dosing strategies can mitigate the negative effects of density-dependence. Finally, we show that these density effects lead to bistable treatment outcomes for a wide range of antibiotic concentrations in a pharmacological model of antibiotic treatment. As a result, infections exceeding a critical density often survive otherwise effective treatments.

Autofluorescence from NADH Conformations Associated with Different Metabolic Pathways Monitored Using Nanosecond-Gated Spectroscopy and Spectral Phasor Analysis.

Cellular NADH conformation is increasingly recognized as an endogenous optical biomarker and metabolic indicator. Recently, we reported a real-time approach for tracking metabolism on the basis of the quantification of UV-excited autofluorescence spectrum shape. Here, we use nanosecond-gated spectral acquisition, combined with spectrum-shape quantification, to monitor the long excited-state lifetime autofluorescence (usually associated with protein-bound NADH conformations) separately from the autofluorescence signal as a whole. We observe that the autofluorescence response induced by two NADH-oxidation inhibitors­cyanide and ethanol­are similar in Saccharomyces cerevisiae when monitored using time-integrated detection but easily distinguished using time-gated detection. Results are consistent with the observation of multiple NADH conformations as assessed using spectral phasor analysis. Further, because well-known oxidation inhibitors are used, changes in spectrum shape can be associated with NADH conformations involved in the different metabolic pathways, giving bioanalytic utility to the spectral responses.

The real-time quantification of autofluorescence spectrum shape for the monitoring of mitochondrial metabolism.

The cellular proportion of free and protein-bound NADH complexes is increasingly recognized as a metabolic indicator and biomarker. Because free and bound forms exhibit different fluorescence spectra, we consider whether autofluorescence shape sufficiently correlates with mitochondrial metabolism to be useful for monitoring in cellular suspensions. Several computational approaches for rapidly quantifying spectrum shape are used to detect Saccharomyces cereviseae response to oxygenation, and to the addition of mitochondrial functional modifiers and metabolic substrates. Observed changes appear consistent with previous studies probing free/protein-bound proportions, making this a potentially useful approach for the real-time monitoring of metabolism. (© 2015 WILEY-VCH Verlag GmbH &Co. KGaA, Weinheim).

Note: A micro-perfusion system for use during real-time physiological studies under high pressure.

We construct a micro-perfusion system using piston screw pump generators for use during real-time, high-pressure physiological studies. Perfusion is achieved using two generators, with one generator being compressed while the other is retracted, thus maintaining pressurization while producing fluid flow. We demonstrate control over perfusion rates in the 10-µl/s range and the ability to change between fluid reservoirs at up to 50 MPa. We validate the screw-pump approach by monitoring the cyanide-induced response of UV-excited autofluorescence from Saccharomyces cerevisiae under pressurization.