Learning from mistakes-Assessing the performance and uncertainty in process-based models.

Typical applications of process- or physically-based models aim to gain a better process understanding or provide the basis for a decision-making process. To adequately represent the physical system, models should include all essential processes. However, model errors can still occur. Other than large systematic observation errors, simplified, misrepresented, inadequately parametrised or missing processes are potential sources of errors. This study presents a set of methods and a proposed workflow for analysing errors of process-based models as a basis for relating them to process representations. The evaluated approach consists of three steps: (1) training a machine-learning (ML) error model using the input data of the process-based model and other available variables, (2) estimation of local explanations (i.e., contributions of each variable to an individual prediction) for each predicted model error using SHapley Additive exPlanations (SHAP) in combination with principal component analysis, (3) clustering of SHAP values of all predicted errors to derive groups with similar error generation characteristics. By analysing these groups of different error-variable association, hypotheses on error generation and corresponding processes can be formulated. That can ultimately lead to improvements in process understanding and prediction. The approach is applied to a process-based stream water temperature model HFLUX in a case study for modelling an alpine stream in the Canadian Rocky Mountains. By using available meteorological and hydrological variables as inputs, the applied ML model is able to predict model residuals. Clustering of SHAP values results in three distinct error groups that are mainly related to shading and vegetation-emitted long wave radiation. Model errors are rarely random and often contain valuable information. Assessing model error associations is ultimately a way of enhancing trust in implemented processes and of providing information on potential areas of improvement to the model.

Automatic Regionalization of Model Parameters for Hydrological Models.

Parameter estimation is one of the most challenging tasks in large-scale distributed modeling, because of the high dimensionality of the parameter space. Relating model parameters to catchment/landscape characteristics reduces the number of parameters, enhances physical realism, and allows the transfer of hydrological model parameters in time and space. This study presents the first large-scale application of automatic parameter transfer function (TF) estimation for a complex hydrological model. The Function Space Optimization (FSO) method can automatically estimate TF structures and coefficients for distributed models. We apply FSO to the mesoscale Hydrologic Model (mHM, mhm-ufz.org), which is the only available distributed model that includes a priori defined TFs for all its parameters. FSO is used to estimate new TFs for the parameters "saturated hydraulic conductivity" and "field capacity," which both influence a range of hydrological processes. The setup of mHM from a previous study serves as a benchmark. The estimated TFs resulted in predictions in 222 validation basins with a median NSE of 0.68, showing that even with 5 years of calibration data, high performance in ungauged basins can be achieved. The performance is similar to the benchmark results, showing that the automatic TFs can achieve comparable results to TFs that were developed over years using expert knowledge. In summary, the findings present a step toward automatic TF estimation of model parameters for distributed models.

Systematic analysis and comparison of commercial seizure-detection software.

OBJECTIVE: To determine if three different commercially available seizure-detection software packages (Besa 2.0, Encevis 1.7, and Persyst 13) accurately detect seizures with high sensitivity, high specificity, and short detection delay in epilepsy patients undergoing long-term video-electroencephalography (EEG) monitoring (VEM). METHODS: Comparison of sensitivity (detection rate), specificity (false alarm rate), and detection delay of three commercially available seizure-detection software packages in 81 randomly selected patients with epilepsy undergoing long-term VEM. RESULTS: Detection rates on a per-patient basis were not significantly different between Besa (mean 67.6%, range 0-100%), Encevis (77.8%, 0-100%) and Persyst (81%, 0-100%; P = .059). False alarm rate (per hour) was significantly different between Besa (mean 0.7/h, range 0.01-6.2/h), Encevis (0.2/h, 0.01-0.5/h), and Persyst (0.9/h, 0.04-6.5/h; P < .001). Detection delay was significantly different between Besa (mean 30 s, range 0-431 s), Encevis (25 s, 2-163 s), and Persyst (20 s, 0-167 s; P = .007). Kappa statistics showed moderate to substantial agreement between the reference standard and each seizure-detection software (Besa: 0.47, 95% confidence interval [CI] 0.36-0.59; Encevis: 0.59, 95% CI 0.47-0.7; Persyst: 0.63, 95% CI 0.51-0.74). SIGNIFICANCE: Three commercially available seizure-detection software packages showed similar, reasonable sensitivities on the same data set, but differed in false alarm rates and detection delay. Persyst 13 showed the highest detection rate and false alarm rate with the shortest detection delay, whereas Encevis 1.7 had a slightly lower sensitivity, the lowest false alarm rate, and longer detection delay.

Genome-wide association study of germline copy number variations reveals an association with prostate cancer aggressiveness.

Prostate cancer is a major health burden, being the second most commonly diagnosed malignancy in men worldwide. Overtreatment represents a major problem in prostate cancer therapy, leading to significant long-term quality-of-life effects for patients and a broad socio-ecological burden. Biomarkers that could facilitate risk stratification of prostate cancer aggressiveness at the time of diagnosis may help to guide clinical treatment decisions and reduce overtreatment. Previous research on genetic variations in prostate cancer has shown that germline copy number variations as well as somatic copy number alterations are commonly present in cancer patients, altering a greater portion of the cancer genome than any other type of genetic variation. To investigate the effect of germline copy number variations on cancer aggressiveness we have compared genome-wide screening data from genomic DNA isolated from the blood of 120 patients with aggressive prostate cancer, 231 patients with non-aggressive prostate cancer and 87 controls with benign prostatic hyperplasia from the Prostate Cancer Study of Austria biobank using the Affymetrix SNP 6.0 array. We could show that patients with an aggressive form of prostate cancer had a higher frequency of copy number variations [mean count of copy number segments (CNS) = 12.9, median count of CNS = 9] compared to patients with non-aggressive prostate cancer (mean count of CNS = 10.4, median count of CNS = 8) or control patients diagnosed with benign prostatic hyperplasia (mean count of CNS = 9.3, median count of CNS = 8). In general, we observed that copy number gain is a rarer event, compared to copy number loss within all three patient groups. Furthermore, we could show a significant effect of copy number losses located on chromosomes 8, 9 and 10 on prostate cancer aggressiveness (P = 0.040, P = 0.037 and P = 0.005, respectively). Applying a cross-validation analysis yielded an area under the curve of 0.63. Our study reports promising findings suggesting that copy number losses might play an important role in the establishment of novel biomarkers to predict prostate cancer aggressiveness at the time of diagnosis. Such markers could be used to facilitate risk stratification to reduce overtreatment of prostate cancer patients.