How Modelers Model: the Overlooked Social and Human Dimensions in Model Intercomparison Studies.

There is a growing realization that the complexity of model ensemble studies depends not only on the models used but also on the experience and approach used by modelers to calibrate and validate results, which remain a source of uncertainty. Here, we applied a multi-criteria decision-making method to investigate the rationale applied by modelers in a model ensemble study where 12 process-based different biogeochemical model types were compared across five successive calibration stages. The modelers shared a common level of agreement about the importance of the variables used to initialize their models for calibration. However, we found inconsistency among modelers when judging the importance of input variables across different calibration stages. The level of subjective weighting attributed by modelers to calibration data decreased sequentially as the extent and number of variables provided increased. In this context, the perceived importance attributed to variables such as the fertilization rate, irrigation regime, soil texture, pH, and initial levels of soil organic carbon and nitrogen stocks was statistically different when classified according to model types. The importance attributed to input variables such as experimental duration, gross primary production, and net ecosystem exchange varied significantly according to the length of the modeler's experience. We argue that the gradual access to input data across the five calibration stages negatively influenced the consistency of the interpretations made by the modelers, with cognitive bias in "trial-and-error" calibration routines. Our study highlights that overlooking human and social attributes is critical in the outcomes of modeling and model intercomparison studies. While complexity of the processes captured in the model algorithms and parameterization is important, we contend that (1) the modeler's assumptions on the extent to which parameters should be altered and (2) modeler perceptions of the importance of model parameters are just as critical in obtaining a quality model calibration as numerical or analytical details.

Improving model prediction of soil N2O emissions through Bayesian calibration.

The biogeochemical processes that lead to the production of N2O in arable soils are controlled by temporally and spatially varying drivers. The need for prediction of soil N2O emissions across scales means that agroecosystem biogeochemistry models are widely used to simulate N2O emissions. Due to the parameter-dense nature of agroecosystem models their parameters have to be calibrated according to the soil and climatic conditions of the intended area of application. Bayesian calibration is considered one of the most advanced ways to complete this task. In this study, we calibrate nine parameters of the Landscape-DNDC process-based agroecosystem model, which are key to its N2O prediction. The Metropolis-Hastings algorithm is used at four separate implementations in order to estimate parameter posterior distributions at four arable sites in the UK. The results of this process are visualised, summarised and assessed against measured N2O data from ten independent arable sites. The study shows that, in many cases, soil N2O emission peaks that were not predicted with the default model parameters were predicted after calibration. Overall, the prediction of soil N2O fluxes across all the sites that were considered was improved by 33% when using the calibrated parameters.

Assessing uncertainties in crop and pasture ensemble model simulations of productivity and N2 O emissions.

Simulation models are extensively used to predict agricultural productivity and greenhouse gas emissions. However, the uncertainties of (reduced) model ensemble simulations have not been assessed systematically for variables affecting food security and climate change mitigation, within multi-species agricultural contexts. We report an international model comparison and benchmarking exercise, showing the potential of multi-model ensembles to predict productivity and nitrous oxide (N2 O) emissions for wheat, maize, rice and temperate grasslands. Using a multi-stage modelling protocol, from blind simulations (stage 1) to partial (stages 2-4) and full calibration (stage 5), 24 process-based biogeochemical models were assessed individually or as an ensemble against long-term experimental data from four temperate grassland and five arable crop rotation sites spanning four continents. Comparisons were performed by reference to the experimental uncertainties of observed yields and N2 O emissions. Results showed that across sites and crop/grassland types, 23%-40% of the uncalibrated individual models were within two standard deviations (SD) of observed yields, while 42 (rice) to 96% (grasslands) of the models were within 1 SD of observed N2 O emissions. At stage 1, ensembles formed by the three lowest prediction model errors predicted both yields and N2 O emissions within experimental uncertainties for 44% and 33% of the crop and grassland growth cycles, respectively. Partial model calibration (stages 2-4) markedly reduced prediction errors of the full model ensemble E-median for crop grain yields (from 36% at stage 1 down to 4% on average) and grassland productivity (from 44% to 27%) and to a lesser and more variable extent for N2 O emissions. Yield-scaled N2 O emissions (N2 O emissions divided by crop yields) were ranked accurately by three-model ensembles across crop species and field sites. The potential of using process-based model ensembles to predict jointly productivity and N2 O emissions at field scale is discussed.