Domain randomization using synthetic electrocardiograms for training neural networks.

We present a method for training neural networks with synthetic electrocardiograms that mimic signals produced by a wearable single lead electrocardiogram monitor. We use domain randomization where the synthetic signal properties such as the waveform shape, RR-intervals and noise are varied for every training example. Models trained with synthetic data are compared to their counterparts trained with real data. Detection of r-waves in electrocardiograms recorded during different physical activities and in atrial fibrillation is used to assess the performance. By allowing the randomization of the synthetic signals to increase beyond what is typically observed in the real-world data the performance is on par or superseding the performance of networks trained with real data. Experiments show robust model performance using different seeds and on different unseen test sets that were fully separated from the training phase. The ability of the model to generalize well to hidden test sets without any specific tuning provides a simple and explainable alternative to more complex adversarial domain adaptation methods for model generalization. This method opens up the possibility of extending the use of synthetic data towards domain insensitive cardiac disease classification when disease specific a priori information is used in the electrocardiogram generation. Additionally, the method provides training with free-to-collect data with accurate labels, control of the data distribution eliminating class imbalances that are typically observed in health-related data, and the generated data is inherently private.

Budget-Based Classification of Parkinson's Disease From Resting State EEG.

Early detection is vital for future neuroprotective treatments of Parkinson's disease (PD). Resting state electroencephalographic (EEG) recording has shown potential as a cost-effective means to aid in detection of neurological disorders such as PD. In this study, we investigated how the number and placement of electrodes affects classifying PD patients and healthy controls using machine learning based on EEG sample entropy. We used a custom budget-based search algorithm for selecting optimized sets of channels for classification, and iterated over variable channel budgets to investigate changes in classification performance. Our data consisted of 60-channel EEG collected at three different recording sites, each of which included observations collected both eyes open (total N = 178) and eyes closed (total N = 131). Our results with the data recorded eyes open demonstrated reasonable classification performance (ACC = .76; AUC = .76) with only 5 channels placed far away from each other, the selected regions including right-frontal, left-temporal and midline-occipital sites. Comparison to randomly selected subsets of channels indicated improved classifier performance only with relatively small channel-budgets. The results with the data recorded eyes closed demonstrated consistently worse classification performance (when compared to eyes open data), and classifier performance improved more steadily as a function of number of channels. In summary, our results suggest that a small subset of electrodes of an EEG recording can suffice for detecting PD with a classification performance on par with a full set of electrodes. Furthermore our results demonstrate that separately collected EEG data sets can be used for pooled machine learning based PD detection with reasonable classification performance.

Machine Learning Approach for Preterm Birth Prediction Using Health Records: Systematic Review.

BACKGROUND: Preterm birth (PTB), a common pregnancy complication, is responsible for 35% of the 3.1 million pregnancy-related deaths each year and significantly affects around 15 million children annually worldwide. Conventional approaches to predict PTB lack reliable predictive power, leaving >50% of cases undetected. Recently, machine learning (ML) models have shown potential as an appropriate complementary approach for PTB prediction using health records (HRs). OBJECTIVE: This study aimed to systematically review the literature concerned with PTB prediction using HR data and the ML approach. METHODS: This systematic review was conducted in accordance with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) statement. A comprehensive search was performed in 7 bibliographic databases until May 15, 2021. The quality of the studies was assessed, and descriptive information, including descriptive characteristics of the data, ML modeling processes, and model performance, was extracted and reported. RESULTS: A total of 732 papers were screened through title and abstract. Of these 732 studies, 23 (3.1%) were screened by full text, resulting in 13 (1.8%) papers that met the inclusion criteria. The sample size varied from a minimum value of 274 to a maximum of 1,400,000. The time length for which data were extracted varied from 1 to 11 years, and the oldest and newest data were related to 1988 and 2018, respectively. Population, data set, and ML models' characteristics were assessed, and the performance of the model was often reported based on metrics such as accuracy, sensitivity, specificity, and area under the receiver operating characteristic curve. CONCLUSIONS: Various ML models used for different HR data indicated potential for PTB prediction. However, evaluation metrics, software and package used, data size and type, selected features, and importantly data management method often remain unjustified, threatening the reliability, performance, and internal or external validity of the model. To understand the usefulness of ML in covering the existing gap, future studies are also suggested to compare it with a conventional method on the same data set.

Adaptive risk prediction system with incremental and transfer learning.

Currently, popular methods for prenatal risk assessment of fetal aneuploidies are based on multivariate probabilistic modelling, that are built on decades of scientific research and large-scale multi-center clinical studies. These static models that are deployed to screening labs are rarely updated or adapted to local population characteristics. In this article, we propose an adaptive risk prediction system or ARPS, which considers these changing characteristics and automatically deploys updated risk models. 8 years of real-life Down syndrome screening data was used to firstly develop a distribution shift detection method that captures significant changes in the patient population and secondly a probabilistic risk modelling system that adapts to new data when these changes are detected. Various candidate systems that utilize transfer -and incremental learning that implement different levels of plasticity were tested. Distribution shift detection using a windowed approach provides a computationally less expensive alternative to fitting models at every data block step while not sacrificing performance. This was possible when utilizing transfer learning. Deploying an ARPS to a lab requires careful consideration of the parameters regarding the distribution shift detection and model updating, as they are affected by lab throughput and the incidence of the screened rare disorder. When this is done, ARPS could be also utilized for other population screening problems. We demonstrate with a large real-life dataset that our best performing novel Incremental-Learning-Population-to-Population-Transfer-Learning design can achieve on par prediction performance without human intervention, when compared to a deployed risk screening algorithm that has been manually updated over several years.

Synthetic minority oversampling of vital statistics data with generative adversarial networks.

OBJECTIVE: Minority oversampling is a standard approach used for adjusting the ratio between the classes on imbalanced data. However, established methods often provide modest improvements in classification performance when applied to data with extremely imbalanced class distribution and to mixed-type data. This is usual for vital statistics data, in which the outcome incidence dictates the amount of positive observations. In this article, we developed a novel neural network-based oversampling method called actGAN (activation-specific generative adversarial network) that can derive useful synthetic observations in terms of increasing prediction performance in this context. MATERIALS AND METHODS: From vital statistics data, the outcome of early stillbirth was chosen to be predicted based on demographics, pregnancy history, and infections. The data contained 363 560 live births and 139 early stillbirths, resulting in class imbalance of 99.96% and 0.04%. The hyperparameters of actGAN and a baseline method SMOTE-NC (Synthetic Minority Over-sampling Technique-Nominal Continuous) were tuned with Bayesian optimization, and both were compared against a cost-sensitive learning-only approach. RESULTS: While SMOTE-NC provided mixed results, actGAN was able to improve true positive rate at a clinically significant false positive rate and area under the curve from the receiver-operating characteristic curve consistently. DISCUSSION: Including an activation-specific output layer to a generator network of actGAN enables the addition of information about the underlying data structure, which overperforms the nominal mechanism of SMOTE-NC. CONCLUSIONS: actGAN provides an improvement to the prediction performance for our learning task. Our developed method could be applied to other mixed-type data prediction tasks that are known to be afflicted by class imbalance and limited data availability.

Algebraic shortcuts for leave-one-out cross-validation in supervised network inference.

Supervised machine learning techniques have traditionally been very successful at reconstructing biological networks, such as protein-ligand interaction, protein-protein interaction and gene regulatory networks. Many supervised techniques for network prediction use linear models on a possibly nonlinear pairwise feature representation of edges. Recently, much emphasis has been placed on the correct evaluation of such supervised models. It is vital to distinguish between using a model to either predict new interactions in a given network or to predict interactions for a new vertex not present in the original network. This distinction matters because (i) the performance might dramatically differ between the prediction settings and (ii) tuning the model hyperparameters to obtain the best possible model depends on the setting of interest. Specific cross-validation schemes need to be used to assess the performance in such different prediction settings. In this work we discuss a state-of-the-art kernel-based network inference technique called two-step kernel ridge regression. We show that this regression model can be trained efficiently, with a time complexity scaling with the number of vertices rather than the number of edges. Furthermore, this framework leads to a series of cross-validation shortcuts that allow one to rapidly estimate the model performance for any relevant network prediction setting. This allows computational biologists to fully assess the capabilities of their models. The machine learning techniques with the algebraic shortcuts are implemented in the RLScore software package: https://github.com/aatapa/RLScore.

Prostate Cancer Risk Stratification in Men With a Clinical Suspicion of Prostate Cancer Using a Unique Biparametric MRI and Expression of 11 Genes in Apparently Benign Tissue: Evaluation Using Machine-Learning Techniques.

BACKGROUND: Accurate risk stratification of men with a clinical suspicion of prostate cancer (cSPCa) remains challenging despite the increasing use of MRI. PURPOSE: To evaluate the diagnostic accuracy of a unique biparametric MRI protocol (IMPROD bpMRI) combined with clinical and molecular markers in men with cSPCa. STUDY TYPE: Prospective single-institutional clinical trial (NCT01864135). SUBJECTS: Eighty men with cSPCa. FIELD STRENGTH/SEQUENCE: 3T, surface array coils. Two T2 -weighted and three diffusion-weighted imaging (DWI) acquisitions: 1) b-values 0, 100, 200, 300, 500 s/mm2 ; 2) b-values 0,1500 s/mm2 ; 3) b-values 0, 2000 s/mm2 . ASSESSMENT: IMPROD bpMRI examinations were qualitatively (IMPROD bpMRI Likert score) and quantitatively (DWI-based Gleason grade score) prospectively reported. Men with IMPROD bpMRI Likert 3-5 had two targeted biopsies followed by 12-core systematic biopsies (SB); those with IMPROD bpMRI Likert 1-2 had only SB. Additionally, 2-core from normal-appearing prostate areas were obtained for the mRNA expression of ACSM1, AMACR, CACNA1D, DLX1, PCA3, PLA2G7, RHOU, SPINK1, SPON2, TMPRSS2-ERG, and TDRD1 measured by quantitative reverse-transcription polymerase chain reaction. STATISTICAL TESTS: Univariate and multivariate analysis using regularized least-squares, feature selection and tournament leave-pair-out cross-validation (TLPOCV), as well as 10 random splits of the data in training-testing sets, were used to evaluate the mRNA, clinical and IMPROD bpMRI parameters in detecting clinically significant prostate cancer (SPCa) defined as Gleason score ≥ 3 + 4. The evaluation metric was the area under the curve (AUC). RESULTS: IMPROD bpMRI Likert demonstrated the highest TLPOCV AUC of 0.92. The tested clinical variables had AUC 0.56-0.73, while the mRNA and additional IMPROD bpMRI parameters had AUC 0.50-0.67 and 0.65-0.89 respectively. The combination of clinical and mRNA biomarkers produced TLPOCV AUC of 0.87, the highest TLPOCV performance without including IMPROD bpMRI Likert. DATA CONCLUSION: The qualitative IMPROD bpMRI Likert score demonstrated the highest accuracy for SPCa detection compared with the tested clinical variables and mRNA biomarkers. LEVEL OF EVIDENCE: 1 Technical Efficacy Stage: 2 J. Magn. Reson. Imaging 2020;51:1540-1553.

Cost-effective survival prediction for patients with advanced prostate cancer using clinical trial and real-world hospital registry datasets.

INTRODUCTION: Predictive survival modeling offers systematic tools for clinical decision-making and individualized tailoring of treatment strategies to improve patient outcomes while reducing overall healthcare costs. In 2015, a number of machine learning and statistical models were benchmarked in the DREAM 9.5 Prostate Cancer Challenge, based on open clinical trial data for metastatic castration resistant prostate cancer (mCRPC). However, applying these models into clinical practice poses a practical challenge due to the inclusion of a large number of model variables, some of which are not routinely monitored or are expensive to measure. OBJECTIVES: To develop cost-specified variable selection algorithms for constructing cost-effective prognostic models of overall survival that still preserve sufficient model performance for clinical decision making. METHODS: Penalized Cox regression models were used for the survival prediction. For the variable selection, we implemented two algorithms: (i) LASSO regularization approach; and (ii) a greedy cost-specified variable selection algorithm. The models were compared in three cohorts of mCRPC patients from randomized clinical trials (RCT), as well as in a real-world cohort (RWC) of advanced prostate cancer patients treated at the Turku University Hospital. Hospital laboratory expenses were utilized as a reference for computing the costs of introducing new variables into the models. RESULTS: Compared to measuring the full set of clinical variables, economic costs could be reduced by half without a significant loss of model performance. The greedy algorithm outperformed the LASSO-based variable selection with the lowest tested budgets. The overall top performance was higher with the LASSO algorithm. CONCLUSION: The cost-specified variable selection offers significant budget optimization capability for the real-world survival prediction without compromising the predictive power of the model.

Tournament leave-pair-out cross-validation for receiver operating characteristic analysis.

Receiver operating characteristic analysis is widely used for evaluating diagnostic systems. Recent studies have shown that estimating an area under receiver operating characteristic curve with standard cross-validation methods suffers from a large bias. The leave-pair-out cross-validation has been shown to correct this bias. However, while leave-pair-out produces an almost unbiased estimate of area under receiver operating characteristic curve, it does not provide a ranking of the data needed for plotting and analyzing the receiver operating characteristic curve. In this study, we propose a new method called tournament leave-pair-out cross-validation. This method extends leave-pair-out by creating a tournament from pair comparisons to produce a ranking for the data. Tournament leave-pair-out preserves the advantage of leave-pair-out for estimating area under receiver operating characteristic curve, while it also allows performing receiver operating characteristic analyses. We have shown using both synthetic and real-world data that tournament leave-pair-out is as reliable as leave-pair-out for area under receiver operating characteristic curve estimation and confirmed the bias in leave-one-out cross-validation on low-dimensional data. As a case study on receiver operating characteristic analysis, we also evaluate how reliably sensitivity and specificity can be estimated from tournament leave-pair-out receiver operating characteristic curves.

Learning with multiple pairwise kernels for drug bioactivity prediction.

Motivation: Many inference problems in bioinformatics, including drug bioactivity prediction, can be formulated as pairwise learning problems, in which one is interested in making predictions for pairs of objects, e.g. drugs and their targets. Kernel-based approaches have emerged as powerful tools for solving problems of that kind, and especially multiple kernel learning (MKL) offers promising benefits as it enables integrating various types of complex biomedical information sources in the form of kernels, along with learning their importance for the prediction task. However, the immense size of pairwise kernel spaces remains a major bottleneck, making the existing MKL algorithms computationally infeasible even for small number of input pairs. Results: We introduce pairwiseMKL, the first method for time- and memory-efficient learning with multiple pairwise kernels. pairwiseMKL first determines the mixture weights of the input pairwise kernels, and then learns the pairwise prediction function. Both steps are performed efficiently without explicit computation of the massive pairwise matrices, therefore making the method applicable to solving large pairwise learning problems. We demonstrate the performance of pairwiseMKL in two related tasks of quantitative drug bioactivity prediction using up to 167 995 bioactivity measurements and 3120 pairwise kernels: (i) prediction of anticancer efficacy of drug compounds across a large panel of cancer cell lines; and (ii) prediction of target profiles of anticancer compounds across their kinome-wide target spaces. We show that pairwiseMKL provides accurate predictions using sparse solutions in terms of selected kernels, and therefore it automatically identifies also data sources relevant for the prediction problem. Availability and implementation: Code is available at https://github.com/aalto-ics-kepaco. Supplementary information: Supplementary data are available at Bioinformatics online.

A Comparative Study of Pairwise Learning Methods Based on Kernel Ridge Regression.

Many machine learning problems can be formulated as predicting labels for a pair of objects. Problems of that kind are often referred to as pairwise learning, dyadic prediction, or network inference problems. During the past decade, kernel methods have played a dominant role in pairwise learning. They still obtain a state-of-the-art predictive performance, but a theoretical analysis of their behavior has been underexplored in the machine learning literature. In this work we review and unify kernel-based algorithms that are commonly used in different pairwise learning settings, ranging from matrix filtering to zero-shot learning. To this end, we focus on closed-form efficient instantiations of Kronecker kernel ridge regression. We show that independent task kernel ridge regression, two-step kernel ridge regression, and a linear matrix filter arise naturally as a special case of Kronecker kernel ridge regression, implying that all these methods implicitly minimize a squared loss. In addition, we analyze universality, consistency, and spectral filtering properties. Our theoretical results provide valuable insights into assessing the advantages and limitations of existing pairwise learning methods.

Fast Kronecker Product Kernel Methods via Generalized Vec Trick.

Kronecker product kernel provides the standard approach in the kernel methods' literature for learning from graph data, where edges are labeled and both start and end vertices have their own feature representations. The methods allow generalization to such new edges, whose start and end vertices do not appear in the training data, a setting known as zero-shot or zero-data learning. Such a setting occurs in numerous applications, including drug-target interaction prediction, collaborative filtering, and information retrieval. Efficient training algorithms based on the so-called vec trick that makes use of the special structure of the Kronecker product are known for the case where the training data are a complete bipartite graph. In this paper, we generalize these results to noncomplete training graphs. This allows us to derive a general framework for training Kronecker product kernel methods, as specific examples we implement Kronecker ridge regression and support vector machine algorithms. Experimental results demonstrate that the proposed approach leads to accurate models, while allowing order of magnitude improvements in training and prediction time.

A Community Challenge for Inferring Genetic Predictors of Gene Essentialities through Analysis of a Functional Screen of Cancer Cell Lines.

We report the results of a DREAM challenge designed to predict relative genetic essentialities based on a novel dataset testing 98,000 shRNAs against 149 molecularly characterized cancer cell lines. We analyzed the results of over 3,000 submissions over a period of 4 months. We found that algorithms combining essentiality data across multiple genes demonstrated increased accuracy; gene expression was the most informative molecular data type; the identity of the gene being predicted was far more important than the modeling strategy; well-predicted genes and selected molecular features showed enrichment in functional categories; and frequently selected expression features correlated with survival in primary tumors. This study establishes benchmarks for gene essentiality prediction, presents a community resource for future comparison with this benchmark, and provides insights into factors influencing the ability to predict gene essentiality from functional genetic screens. This study also demonstrates the value of releasing pre-publication data publicly to engage the community in an open research collaboration.

Assessment of metal ion concentration in water with structured feature selection.

We propose a cost-effective system for the determination of metal ion concentration in water, addressing a central issue in water resources management. The system combines novel luminometric label array technology with a machine learning algorithm that selects a minimal number of array reagents (modulators) and liquid sample dilutions, such that enable accurate quantification. The algorithm is able to identify the optimal modulators and sample dilutions leading to cost reductions since less manual labour and resources are needed. Inferring the ion detector involves a unique type of a structured feature selection problem, which we formalize in this paper. We propose a novel Cartesian greedy forward feature selection algorithm for solving the problem. The novel algorithm was evaluated in the concentration assessment of five metal ions and the performance was compared to two known feature selection approaches. The results demonstrate that the proposed system can assist in lowering the costs with minimal loss in accuracy.

Computational-experimental approach to drug-target interaction mapping: A case study on kinase inhibitors.

Due to relatively high costs and labor required for experimental profiling of the full target space of chemical compounds, various machine learning models have been proposed as cost-effective means to advance this process in terms of predicting the most potent compound-target interactions for subsequent verification. However, most of the model predictions lack direct experimental validation in the laboratory, making their practical benefits for drug discovery or repurposing applications largely unknown. Here, we therefore introduce and carefully test a systematic computational-experimental framework for the prediction and pre-clinical verification of drug-target interactions using a well-established kernel-based regression algorithm as the prediction model. To evaluate its performance, we first predicted unmeasured binding affinities in a large-scale kinase inhibitor profiling study, and then experimentally tested 100 compound-kinase pairs. The relatively high correlation of 0.77 (p < 0.0001) between the predicted and measured bioactivities supports the potential of the model for filling the experimental gaps in existing compound-target interaction maps. Further, we subjected the model to a more challenging task of predicting target interactions for such a new candidate drug compound that lacks prior binding profile information. As a specific case study, we used tivozanib, an investigational VEGF receptor inhibitor with currently unknown off-target profile. Among 7 kinases with high predicted affinity, we experimentally validated 4 new off-targets of tivozanib, namely the Src-family kinases FRK and FYN A, the non-receptor tyrosine kinase ABL1, and the serine/threonine kinase SLK. Our sub-sequent experimental validation protocol effectively avoids any possible information leakage between the training and validation data, and therefore enables rigorous model validation for practical applications. These results demonstrate that the kernel-based modeling approach offers practical benefits for probing novel insights into the mode of action of investigational compounds, and for the identification of new target selectivities for drug repurposing applications.

Prediction of overall survival for patients with metastatic castration-resistant prostate cancer: development of a prognostic model through a crowdsourced challenge with open clinical trial data.

BACKGROUND: Improvements to prognostic models in metastatic castration-resistant prostate cancer have the potential to augment clinical trial design and guide treatment strategies. In partnership with Project Data Sphere, a not-for-profit initiative allowing data from cancer clinical trials to be shared broadly with researchers, we designed an open-data, crowdsourced, DREAM (Dialogue for Reverse Engineering Assessments and Methods) challenge to not only identify a better prognostic model for prediction of survival in patients with metastatic castration-resistant prostate cancer but also engage a community of international data scientists to study this disease. METHODS: Data from the comparator arms of four phase 3 clinical trials in first-line metastatic castration-resistant prostate cancer were obtained from Project Data Sphere, comprising 476 patients treated with docetaxel and prednisone from the ASCENT2 trial, 526 patients treated with docetaxel, prednisone, and placebo in the MAINSAIL trial, 598 patients treated with docetaxel, prednisone or prednisolone, and placebo in the VENICE trial, and 470 patients treated with docetaxel and placebo in the ENTHUSE 33 trial. Datasets consisting of more than 150 clinical variables were curated centrally, including demographics, laboratory values, medical history, lesion sites, and previous treatments. Data from ASCENT2, MAINSAIL, and VENICE were released publicly to be used as training data to predict the outcome of interest-namely, overall survival. Clinical data were also released for ENTHUSE 33, but data for outcome variables (overall survival and event status) were hidden from the challenge participants so that ENTHUSE 33 could be used for independent validation. Methods were evaluated using the integrated time-dependent area under the curve (iAUC). The reference model, based on eight clinical variables and a penalised Cox proportional-hazards model, was used to compare method performance. Further validation was done using data from a fifth trial-ENTHUSE M1-in which 266 patients with metastatic castration-resistant prostate cancer were treated with placebo alone. FINDINGS: 50 independent methods were developed to predict overall survival and were evaluated through the DREAM challenge. The top performer was based on an ensemble of penalised Cox regression models (ePCR), which uniquely identified predictive interaction effects with immune biomarkers and markers of hepatic and renal function. Overall, ePCR outperformed all other methods (iAUC 0·791; Bayes factor >5) and surpassed the reference model (iAUC 0·743; Bayes factor >20). Both the ePCR model and reference models stratified patients in the ENTHUSE 33 trial into high-risk and low-risk groups with significantly different overall survival (ePCR: hazard ratio 3·32, 95% CI 2·39-4·62, p<0·0001; reference model: 2·56, 1·85-3·53, p<0·0001). The new model was validated further on the ENTHUSE M1 cohort with similarly high performance (iAUC 0·768). Meta-analysis across all methods confirmed previously identified predictive clinical variables and revealed aspartate aminotransferase as an important, albeit previously under-reported, prognostic biomarker. INTERPRETATION: Novel prognostic factors were delineated, and the assessment of 50 methods developed by independent international teams establishes a benchmark for development of methods in the future. The results of this effort show that data-sharing, when combined with a crowdsourced challenge, is a robust and powerful framework to develop new prognostic models in advanced prostate cancer. FUNDING: Sanofi US Services, Project Data Sphere.

Circulating Cytokines Predict the Development of Insulin Resistance in a Prospective Finnish Population Cohort.

CONTEXT: Metabolic inflammation contributes to the development of insulin resistance (IR), but the roles of different inflammatory and other cytokines in this process remain unclear. OBJECTIVE: We aimed at analyzing the value of different cytokines in predicting future IR. DESIGN, SETTING, AND PARTICIPANTS: We measured the serum concentrations of 48 cytokines from a nationwide cohort of 2200 Finns (the Cardiovascular Risk in Young Finns Study), and analyzed their role as independent risk factors for predicting the development of IR 4 years later. MAIN OUTCOME MEASURES: We used cross-sectional regression analysis adjusted for known IR risk factors (high age, body mass index, systolic blood pressure, triglycerides, smoking, physical inactivity, and low high-density lipoprotein cholesterol), C-reactive protein and 37 cytokines to find the determinants of continuous baseline IR (defined by homeostatic model assessment). A logistic regression model adjusted for the known risk factors, baseline IR, and 37 cytokines was used to predict the future IR. RESULTS: Several cytokines, often in a sex-dependent manner, remained as independent determinants of current IR. In men, none of the cytokines was an independent predictive risk marker of future IR. In women, in contrast, IL-17 (odds ratio, 1.42 for 1-SD change in ln-transformed IL-17) and IL-18 (odds ratio, 1.37) were independently associated with the future IR. IL-17 levels also independently predicted the development of incident future IR (odds ratio, 1.48). CONCLUSIONS: The systemic levels of the T helper 1 cell cytokine IL-18 and the T helper 17 cell cytokine IL-17 thus may have value in predicting future insulin sensitivity in women independently of classical IR risk factors.

Comparison of automatic summarisation methods for clinical free text notes.

OBJECTIVE: A major source of information available in electronic health record (EHR) systems are the clinical free text notes documenting patient care. Managing this information is time-consuming for clinicians. Automatic text summarisation could assist clinicians in obtaining an overview of the free text information in ongoing care episodes, as well as in writing final discharge summaries. We present a study of automated text summarisation of clinical notes. It looks to identify which methods are best suited for this task and whether it is possible to automatically evaluate the quality differences of summaries produced by different methods in an efficient and reliable way. METHODS AND MATERIALS: The study is based on material consisting of 66,884 care episodes from EHRs of heart patients admitted to a university hospital in Finland between 2005 and 2009. We present novel extractive text summarisation methods for summarising the free text content of care episodes. Most of these methods rely on word space models constructed using distributional semantic modelling. The summarisation effectiveness is evaluated using an experimental automatic evaluation approach incorporating well-known ROUGE measures. We also developed a manual evaluation scheme to perform a meta-evaluation on the ROUGE measures to see if they reflect the opinions of health care professionals. RESULTS: The agreement between the human evaluators is good (ICC=0.74, p<0.001), demonstrating the stability of the proposed manual evaluation method. Furthermore, the correlation between the manual and automated evaluations are high (> 0.90 Spearman's rho). Three of the presented summarisation methods ('Composite', 'Case-Based' and 'Translate') significantly outperform the other methods for all ROUGE measures (p<0.05, Wilcoxon signed-rank test and Bonferroni correction). CONCLUSION: The results indicate the feasibility of the automated summarisation of care episodes. Moreover, the high correlation between manual and automated evaluations suggests that the less labour-intensive automated evaluations can be used as a proxy for human evaluations when developing summarisation methods. This is of significant practical value for summarisation method development, because manual evaluation cannot be afforded for every variation of the summarisation methods. Instead, one can resort to automatic evaluation during the method development process.

Toward more realistic drug-target interaction predictions.

A number of supervised machine learning models have recently been introduced for the prediction of drug-target interactions based on chemical structure and genomic sequence information. Although these models could offer improved means for many network pharmacology applications, such as repositioning of drugs for new therapeutic uses, the prediction models are often being constructed and evaluated under overly simplified settings that do not reflect the real-life problem in practical applications. Using quantitative drug-target bioactivity assays for kinase inhibitors, as well as a popular benchmarking data set of binary drug-target interactions for enzyme, ion channel, nuclear receptor and G protein-coupled receptor targets, we illustrate here the effects of four factors that may lead to dramatic differences in the prediction results: (i) problem formulation (standard binary classification or more realistic regression formulation), (ii) evaluation data set (drug and target families in the application use case), (iii) evaluation procedure (simple or nested cross-validation) and (iv) experimental setting (whether training and test sets share common drugs and targets, only drugs or targets or neither). Each of these factors should be taken into consideration to avoid reporting overoptimistic drug-target interaction prediction results. We also suggest guidelines on how to make the supervised drug-target interaction prediction studies more realistic in terms of such model formulations and evaluation setups that better address the inherent complexity of the prediction task in the practical applications, as well as novel benchmarking data sets that capture the continuous nature of the drug-target interactions for kinase inhibitors.