Model selection in systems biology depends on experimental design.

Experimental design attempts to maximise the information available for modelling tasks. An optimal experiment allows the inferred models or parameters to be chosen with the highest expected degree of confidence. If the true system is faithfully reproduced by one of the models, the merit of this approach is clear - we simply wish to identify it and the true parameters with the most certainty. However, in the more realistic situation where all models are incorrect or incomplete, the interpretation of model selection outcomes and the role of experimental design needs to be examined more carefully. Using a novel experimental design and model selection framework for stochastic state-space models, we perform high-throughput in-silico analyses on families of gene regulatory cascade models, to show that the selected model can depend on the experiment performed. We observe that experimental design thus makes confidence a criterion for model choice, but that this does not necessarily correlate with a model's predictive power or correctness. Finally, in the special case of linear ordinary differential equation (ODE) models, we explore how wrong a model has to be before it influences the conclusions of a model selection analysis.

A framework for parameter estimation and model selection from experimental data in systems biology using approximate Bayesian computation.

As modeling becomes a more widespread practice in the life sciences and biomedical sciences, researchers need reliable tools to calibrate models against ever more complex and detailed data. Here we present an approximate Bayesian computation (ABC) framework and software environment, ABC-SysBio, which is a Python package that runs on Linux and Mac OS X systems and that enables parameter estimation and model selection in the Bayesian formalism by using sequential Monte Carlo (SMC) approaches. We outline the underlying rationale, discuss the computational and practical issues and provide detailed guidance as to how the important tasks of parameter inference and model selection can be performed in practice. Unlike other available packages, ABC-SysBio is highly suited for investigating, in particular, the challenging problem of fitting stochastic models to data. In order to demonstrate the use of ABC-SysBio, in this protocol we postulate the existence of an imaginary reaction network composed of seven interrelated biological reactions (involving a specific mRNA, the protein it encodes and a post-translationally modified version of the protein), a network that is defined by two files containing 'observed' data that we provide as supplementary information. In the first part of the PROCEDURE, ABC-SysBio is used to infer the parameters of this system, whereas in the second part we use ABC-SysBio's relevant functionality to discriminate between two different reaction network models, one of them being the 'true' one. Although computationally expensive, the additional insights gained in the Bayesian formalism more than make up for this cost, especially in complex problems.

Combined model of intrinsic and extrinsic variability for computational network design with application to synthetic biology.

Biological systems are inherently variable, with their dynamics influenced by intrinsic and extrinsic sources. These systems are often only partially characterized, with large uncertainties about specific sources of extrinsic variability and biochemical properties. Moreover, it is not yet well understood how different sources of variability combine and affect biological systems in concert. To successfully design biomedical therapies or synthetic circuits with robust performance, it is crucial to account for uncertainty and effects of variability. Here we introduce an efficient modeling and simulation framework to study systems that are simultaneously subject to multiple sources of variability, and apply it to make design decisions on small genetic networks that play a role of basic design elements of synthetic circuits. Specifically, the framework was used to explore the effect of transcriptional and post-transcriptional autoregulation on fluctuations in protein expression in simple genetic networks. We found that autoregulation could either suppress or increase the output variability, depending on specific noise sources and network parameters. We showed that transcriptional autoregulation was more successful than post-transcriptional in suppressing variability across a wide range of intrinsic and extrinsic magnitudes and sources. We derived the following design principles to guide the design of circuits that best suppress variability: (i) high protein cooperativity and low miRNA cooperativity, (ii) imperfect complementarity between miRNA and mRNA was preferred to perfect complementarity, and (iii) correlated expression of mRNA and miRNA--for example, on the same transcript--was best for suppression of protein variability. Results further showed that correlations in kinetic parameters between cells affected the ability to suppress variability, and that variability in transient states did not necessarily follow the same principles as variability in the steady state. Our model and findings provide a general framework to guide design principles in synthetic biology.

Elucidating the in vivo phosphorylation dynamics of the ERK MAP kinase using quantitative proteomics data and Bayesian model selection.

Ever since reversible protein phosphorylation was discovered, it has been clear that it plays a key role in the regulation of cellular processes. Proteins often undergo double phosphorylation, which can occur through two possible mechanisms: distributive or processive. Which phosphorylation mechanism is chosen for a particular cellular regulation bears biological significance, and it is therefore in our interest to understand these mechanisms. In this paper we study dynamics of the MEK/ERK phosphorylation. We employ a model selection algorithm based on approximate Bayesian computation to elucidate phosphorylation dynamics from quantitative time course data obtained from PC12 cells in vivo. The algorithm infers the posterior distribution over four proposed models for phosphorylation and dephosphorylation dynamics, and this distribution indicates the amount of support given to each model. We evaluate the robustness of our inferential framework by systematically exploring different ways of parameterizing the models and for different prior specifications. The models with the highest inferred posterior probability are the two models employing distributive dephosphorylation, whereas we are unable to choose decisively between the processive and distributive phosphorylation mechanisms.

Designing attractive models via automated identification of chaotic and oscillatory dynamical regimes.

Chaos and oscillations continue to capture the interest of both the scientific and public domains. Yet despite the importance of these qualitative features, most attempts at constructing mathematical models of such phenomena have taken an indirect, quantitative approach, for example, by fitting models to a finite number of data points. Here we develop a qualitative inference framework that allows us to both reverse-engineer and design systems exhibiting these and other dynamical behaviours by directly specifying the desired characteristics of the underlying dynamical attractor. This change in perspective from quantitative to qualitative dynamics, provides fundamental and new insights into the properties of dynamical systems.

From qualitative data to quantitative models: analysis of the phage shock protein stress response in Escherichia coli.

BACKGROUND: Bacteria have evolved a rich set of mechanisms for sensing and adapting to adverse conditions in their environment. These are crucial for their survival, which requires them to react to extracellular stresses such as heat shock, ethanol treatment or phage infection. Here we focus on studying the phage shock protein (Psp) stress response in Escherichia coli induced by a phage infection or other damage to the bacterial membrane. This system has not yet been theoretically modelled or analysed in silico. RESULTS: We develop a model of the Psp response system, and illustrate how such models can be constructed and analyzed in light of available sparse and qualitative information in order to generate novel biological hypotheses about their dynamical behaviour. We analyze this model using tools from Petri-net theory and study its dynamical range that is consistent with currently available knowledge by conditioning model parameters on the available data in an approximate Bayesian computation (ABC) framework. Within this ABC approach we analyze stochastic and deterministic dynamics. This analysis allows us to identify different types of behaviour and these mechanistic insights can in turn be used to design new, more detailed and time-resolved experiments. CONCLUSIONS: We have developed the first mechanistic model of the Psp response in E. coli. This model allows us to predict the possible qualitative stochastic and deterministic dynamic behaviours of key molecular players in the stress response. Our inferential approach can be applied to stress response and signalling systems more generally: in the ABC framework we can condition mathematical models on qualitative data in order to delimit e.g. parameter ranges or the qualitative system dynamics in light of available end-point or qualitative information.

Parameter inference and model selection in signaling pathway models.

To support and guide an extensive experimental research into systems biology of signaling pathways, increasingly more mechanistic models are being developed with hopes of gaining further insight into biological processes. In order to analyze these models, computational and statistical techniques are needed to estimate the unknown kinetic parameters. This chapter reviews methods from frequentist and Bayesian statistics for estimation of parameters and for choosing which model is best for modeling the underlying system. Approximate Bayesian computation techniques are introduced and employed to explore different hypothesis about the JAK-STAT signaling pathway.

ABC-SysBio--approximate Bayesian computation in Python with GPU support.

MOTIVATION: The growing field of systems biology has driven demand for flexible tools to model and simulate biological systems. Two established problems in the modeling of biological processes are model selection and the estimation of associated parameters. A number of statistical approaches, both frequentist and Bayesian, have been proposed to answer these questions. RESULTS: Here we present a Python package, ABC-SysBio, that implements parameter inference and model selection for dynamical systems in an approximate Bayesian computation (ABC) framework. ABC-SysBio combines three algorithms: ABC rejection sampler, ABC SMC for parameter inference and ABC SMC for model selection. It is designed to work with models written in Systems Biology Markup Language (SBML). Deterministic and stochastic models can be analyzed in ABC-SysBio. AVAILABILITY: http://abc-sysbio.sourceforge.net

Managing membrane stress: the phage shock protein (Psp) response, from molecular mechanisms to physiology.

The bacterial phage shock protein (Psp) response functions to help cells manage the impacts of agents impairing cell membrane function. The system has relevance to biotechnology and to medicine. Originally discovered in Escherichia coli, Psp proteins and homologues are found in Gram-positive and Gram-negative bacteria, in archaea and in plants. Study of the E. coli and Yersinia enterocolitica Psp systems provides insights into how membrane-associated sensory Psp proteins might perceive membrane stress, signal to the transcription apparatus and use an ATP-hydrolysing transcription activator to produce effector proteins to overcome the stress. Progress in understanding the mechanism of signal transduction by the membrane-bound Psp proteins, regulation of the psp gene-specific transcription activator and the cell biology of the system is presented and discussed. Many features of the action of the Psp system appear to be dominated by states of self-association of the master effector, PspA, and the transcription activator, PspF, alongside a signalling pathway that displays strong conditionality in its requirement.

Simulation-based model selection for dynamical systems in systems and population biology.

MOTIVATION: Computer simulations have become an important tool across the biomedical sciences and beyond. For many important problems several different models or hypotheses exist and choosing which one best describes reality or observed data is not straightforward. We therefore require suitable statistical tools that allow us to choose rationally between different mechanistic models of, e.g. signal transduction or gene regulation networks. This is particularly challenging in systems biology where only a small number of molecular species can be assayed at any given time and all measurements are subject to measurement uncertainty. RESULTS: Here, we develop such a model selection framework based on approximate Bayesian computation and employing sequential Monte Carlo sampling. We show that our approach can be applied across a wide range of biological scenarios, and we illustrate its use on real data describing influenza dynamics and the JAK-STAT signalling pathway. Bayesian model selection strikes a balance between the complexity of the simulation models and their ability to describe observed data. The present approach enables us to employ the whole formal apparatus to any system that can be (efficiently) simulated, even when exact likelihoods are computationally intractable.

The ABC of reverse engineering biological signalling systems.

Modelling biological systems would be straightforward if we knew the structure of the model and the parameters governing their dynamics. For the overwhelming majority of biological processes, however, such parameter values are unknown and often impossible to measure directly. This means that we have to estimate or infer these parameters from observed data. Here we argue that it is also important to appreciate the uncertainty inherent in these estimates. We discuss a statistical approach--approximate Bayesian computation (ABC)--which allows us to approximate the posterior distribution over parameters and show how this can add insights into our understanding of the system dynamics. We illustrate the application of this approach and how the resulting posterior distribution can be analyzed in the context of the mitogen-activated protein kinase phosphorylation cascade. Our analysis also highlights the added benefit of using the distribution of parameters rather than point estimates of parameter values when considering the notion of sloppy models in systems biology.

Model-based evolutionary analysis: the natural history of phage-shock stress response.

The evolution of proteins is inseparably linked to their function. Because most biological processes involve a number of different proteins, it may become impossible to study the evolutionary properties of proteins in isolation. In the present article, we show how simple mechanistic models of biological processes can complement conventional comparative analyses of biological traits. We use the specific example of the phage-shock stress response, which has been well characterized in Escherichia coli, to elucidate patterns of gene sharing and sequence conservation across bacterial species.

Approximate Bayesian computation scheme for parameter inference and model selection in dynamical systems.

Approximate Bayesian computation (ABC) methods can be used to evaluate posterior distributions without having to calculate likelihoods. In this paper, we discuss and apply an ABC method based on sequential Monte Carlo (SMC) to estimate parameters of dynamical models. We show that ABC SMC provides information about the inferability of parameters and model sensitivity to changes in parameters, and tends to perform better than other ABC approaches. The algorithm is applied to several well-known biological systems, for which parameters and their credible intervals are inferred. Moreover, we develop ABC SMC as a tool for model selection; given a range of different mathematical descriptions, ABC SMC is able to choose the best model using the standard Bayesian model selection apparatus.

Parameter inference for biochemical systems that undergo a Hopf bifurcation.

The increasingly widespread use of parametric mathematical models to describe biological systems means that the ability to infer model parameters is of great importance. In this study, we consider parameter inferability in nonlinear ordinary differential equation models that undergo a bifurcation, focusing on a simple but generic biochemical reaction model. We systematically investigate the shape of the likelihood function for the model's parameters, analyzing the changes that occur as the model undergoes a Hopf bifurcation. We demonstrate that there exists an intrinsic link between inference and the parameters' impact on the modeled system's dynamical stability, which we hope will motivate further research in this area.