Development and validation of clinical prediction models: marginal differences between logistic regression, penalized maximum likelihood estimation, and genetic programming.

OBJECTIVE: Many prediction models are developed by multivariable logistic regression. However, there are several alternative methods to develop prediction models. We compared the accuracy of a model that predicts the presence of deep venous thrombosis (DVT) when developed by four different methods. STUDY DESIGN AND SETTING: We used the data of 2,086 primary care patients suspected of DVT, which included 21 candidate predictors. The cohort was split into a derivation set (1,668 patients, 329 with DVT) and a validation set (418 patients, 86 with DVT). Also, 100 cross-validations were conducted in the full cohort. The models were developed by logistic regression, logistic regression with shrinkage by bootstrapping techniques, logistic regression with shrinkage by penalized maximum likelihood estimation, and genetic programming. The accuracy of the models was tested by assessing discrimination and calibration. RESULTS: There were only marginal differences in the discrimination and calibration of the models in the validation set and cross-validations. CONCLUSION: The accuracy measures of the models developed by the four different methods were only slightly different, and the 95% confidence intervals were mostly overlapped. We have shown that models with good predictive accuracy are most likely developed by sensible modeling strategies rather than by complex development methods.

[Hypoxaemia and bradycardia in children during guillotine adenotonsillectomy without intubation]. / Hypoxemie en bradycardie bij kinderen tijdens adenotonsillectomie volgens Sluder zonder intubatie.

OBJECTIVE: To study the incidence of hypoxaemia and bradycardia in children who undergo guillotine adenotonsillectomy in a sitting position, without intubation and under inhalation anaesthesia. DESIGN: Retrospective study. METHOD: Analysis of age, weight, sex, oxygen saturation, heart rate and subsequent bleeding in all children up to the age of 11 years who underwent guillotine adenotonsillectomy in the period December 1999 to December 2007. Hypoxaemia was defined as oxygen saturation of less than 85% for longer than 60 s. Bradycardia was defined as a heart rate of less than 60/min for longer than 30 s. RESULTS: We analysed data from 2963 patients. The mean age was 4.7 years and mean weight 18.8 kg. There was no significant relationship between age, weight and the onset of incidental desaturation or bradycardia. A total of 132 patients (4.5%) had hypoxaemia and 280 patients (9.4%) had bradycardia. Twenty-five patients had both hypoxaemia and bradycardia, of whom 3 (0.1%) had bradycardia immediately following hypoxaemia. In none of the recorded episodes of hypoxaemia and bradycardia did this lead to peri- or postoperative complications. CONCLUSION: Hypoxaemia and bradycardia occurred during guillotine adenotonsillectomy in non-intubated children in a sitting position under inhalation anaesthesia. The simultaneous onset of hypoxaemia and bradycardia is rare, however, and does not lead to perioperative complications. A further study is required using adenotonsillectomy with a large number of intubated and non-intubated children in order to compare the incidence of hypoxaemia and bradycardia and the occurrence of complications.

A software application for comparing large numbers of high resolution MALDI-FTICR MS spectra demonstrated by searching candidate biomarkers for glioma blood vessel formation.

BACKGROUND: A Java application is presented, which compares large numbers (n > 100) of raw FTICR mass spectra from patients and controls. Two peptide profile matrices can be produced simultaneously, one with occurrences of peptide masses in samples and another with the intensity of common peak masses in all the measured samples, using the peak- and background intensities of the raw data. In latter way, more significantly differentially expressed peptides are found between groups than just using the presence or absence in samples of common peak masses. The software application is tested by searching angiogenesis related proteins in glioma by comparing laser capture micro dissected- and enzymatic by trypsin digested tissue sections. RESULTS: By hierarchical clustering of the presence-absence matrix, it appears that proteins, such as hemoglobin alpha and delta subunit, fibrinogen beta and gamma chain precursor, tubulin specific chaperone A, epidermal fatty acid binding protein, neutrophil gelatinase-associated lipocalin precursor, peptidyl tRNA hydrolase 2 mitochondrial precursor, placenta specific growth hormone, and zinc finger CCHC domain containing protein 13 are significantly different expressed in glioma vessels. The up-regulated proteins in the glioma vessels with respect to the normal vessels determined by the Wilcoxon-Mann-Whitney test on the intensity matrix are vimentin, glial fibrillary acidic protein, serum albumin precursor, annexin A5, alpha cardiac and beta actin, type I cytoskeletal 10 keratin, calcium binding protein p22, and desmin. Peptide masses of calcium binding protein p22, Cdc42 effector protein 3, fibronectin precursor, and myosin-9 are exclusively present in glioma vessels. Some peptide fragments of non-muscular myosin-9 at the C-terminus are strongly up-regulated in the glioma vessels with respect to the normal vessels. CONCLUSION: The less rigorous than in general used commercial propriety software de-isotope algorithm results in more mono-isotopic peptide masses and consequently more proteins. Centroiding of peptide masses takes place by taking the average over more spectra in the profile matrix. Cytoskeleton proteins and proteins involved in the calcium signaling pathway seem to be most up-regulated in glioma vessels. The finding that peptides at the C-terminus of myosin-9 are up-regulated could be ascribed to splicing or fragmentation by proteases.

Identification of differentially regulated splice variants and novel exons in glial brain tumors using exon expression arrays.

Aberrant splice variants are involved in the initiation and/or progression of glial brain tumors. We therefore set out to identify splice variants that are differentially expressed between histologic subgroups of gliomas. Splice variants were identified using a novel platform that profiles the expression of virtually all known and predicted exons present in the human genome. Exon-level expression profiling was done on 26 glioblastomas, 22 oligodendrogliomas, and 6 control brain samples. Our results show that Human Exon arrays can identify subgroups of gliomas based on their histologic appearance and genetic aberrations. We next used our expression data to identify differentially expressed splice variants. In two independent approaches, we identified 49 and up to 459 exons that are differentially spliced between glioblastomas and oligodendrogliomas, a subset of which (47% and 33%) were confirmed by reverse transcription-PCR (RT-PCR). In addition, exon level expression profiling also identified >700 novel exons. Expression of approximately 67% of these candidate novel exons was confirmed by RT-PCR. Our results indicate that exon level expression profiling can be used to molecularly classify brain tumor subgroups, can identify differentially regulated splice variants, and can identify novel exons. The splice variants identified by exon level expression profiling may help to detect the genetic changes that cause or maintain gliomas and may serve as novel treatment targets.

A database application for pre-processing, storage and comparison of mass spectra derived from patients and controls.

BACKGROUND: Statistical comparison of peptide profiles in biomarker discovery requires fast, user-friendly software for high throughput data analysis. Important features are flexibility in changing input variables and statistical analysis of peptides that are differentially expressed between patient and control groups. In addition, integration the mass spectrometry data with the results of other experiments, such as microarray analysis, and information from other databases requires a central storage of the profile matrix, where protein id's can be added to peptide masses of interest. RESULTS: A new database application is presented, to detect and identify significantly differentially expressed peptides in peptide profiles obtained from body fluids of patient and control groups. The presented modular software is capable of central storage of mass spectra and results in fast analysis. The software architecture consists of 4 pillars, 1) a Graphical User Interface written in Java, 2) a MySQL database, which contains all metadata, such as experiment numbers and sample codes, 3) a FTP (File Transport Protocol) server to store all raw mass spectrometry files and processed data, and 4) the software package R, which is used for modular statistical calculations, such as the Wilcoxon-Mann-Whitney rank sum test. Statistic analysis by the Wilcoxon-Mann-Whitney test in R demonstrates that peptide-profiles of two patient groups 1) breast cancer patients with leptomeningeal metastases and 2) prostate cancer patients in end stage disease can be distinguished from those of control groups. CONCLUSION: The database application is capable to distinguish patient Matrix Assisted Laser Desorption Ionization (MALDI-TOF) peptide profiles from control groups using large size datasets. The modular architecture of the application makes it possible to adapt the application to handle also large sized data from MS/MS- and Fourier Transform Ion Cyclotron Resonance (FT-ICR) mass spectrometry experiments. It is expected that the higher resolution and mass accuracy of the FT-ICR mass spectrometry prevents the clustering of peaks of different peptides and allows the identification of differentially expressed proteins from the peptide profiles.

MALDI-TOF mass spectrometry analysis of cerebrospinal fluid tryptic peptide profiles to diagnose leptomeningeal metastases in patients with breast cancer.

Leptomeningeal metastasis (LM) is a devastating complication that occurs in 5% of patients with breast cancer. Early diagnosis and initiation of treatment are essential to prevent neurological deterioration. However, early diagnosis of LM remains challenging because 25% of cerebrospinal fluid (CSF) samples produce false-negative results at first cytological examination. We developed a new, MS-based method to investigate the protein expression patterns present in the CSF from patients with breast cancer with and without LM. CSF samples from 106 patients with active breast cancer (54 with LM and 52 without LM) and 45 control subjects were digested with trypsin. The resulting peptides were measured by MALDI-TOF MS. Then, the mass spectra were analyzed and compared between patient groups using newly developed bioinformatics tools. A total of 895 possible peak positions was detected, and 164 of these peaks discriminated between the patient groups (Kruskal-Wallis, p<0.01). The discriminatory masses were clustered, and a classifier was built to distinguish patients with breast cancer with and without LM. After bootstrap validation, the classifier had a maximum accuracy of 77% with a sensitivity of 79% and a specificity of 76%. Direct MALDI-TOF analysis of tryptic digests of CSF gives reproducible peptide profiles that can assist in diagnosing LM in patients with breast cancer. The same method can be used to develop diagnostic assays for other neurological disorders.

A new method to analyze matrix-assisted laser desorption/ionization time-of-flight peptide profiling mass spectra.

In protein and peptide mass spectrometry in which profiling of peaks is involved, their masses and intensities are important characteristics. Because of the relative low reproducibility of peak intensities associated with complex samples in matrix-assisted laser desorption/ionization time-of-flight mass spectrometry (MALDI-TOFMS), it is difficult to accurately assess the number of peaks and their intensities. In this study we evaluate these two characteristics for tryptic digests of cerebro-spinal fluid. We observed that the reproducibility of peak intensities was relatively poor (CV = 42%) and that additional normalization or spiking did not lead to a large improvement (CV = 30%). Moreover, at least seven mass spectra per sample were required to obtain a reliable peak list. An improvement of the sensitivity (i.e., eventually more peaks are detected) is observed if more replicates per sample are measured. We conclude that the reproducibility and sensitivity of peptide profiling can be significantly improved by a combination of measuring at least seven spectra per sample with a dichotomous scoring of the intensities. This approach will aid the analysis of large numbers of mass spectra of patient samples in a reproducible way for the detection and validation of candidate biomarkers.

Genetic programming outperformed multivariable logistic regression in diagnosing pulmonary embolism.

OBJECTIVE: Genetic programming is a search method that can be used to solve complex associations between large numbers of variables. It has been used, for example, for myoelectrical signal recognition, but its value for medical prediction as in diagnostic and prognostic settings, has not been documented. STUDY DESIGN AND SETTING: We compared genetic programming and the commonly used logistic regression technique in the development of a prediction model using empirical data from a study on diagnosis of pulmonary embolism. Using part (67%) of the data, we developed and internally validated (using bootstrapping techniques) a diagnostic prediction model by genetic programming and by logistic regression, and compared both on their predictive ability in the remaining data (validation set). RESULTS: In the validation set, the area under the ROC curve of the genetic programming model was significantly larger (0.73; 95%CI: 0.64-0.82) than that of the logistic regression model (0.68; 0.59-0.77). The calibration of both models was similar, indicating a similar amount of overoptimism. CONCLUSION: Although the interpretation of a genetic programming model is less intuitive and this is the first empirical study quantifying its value for medical prediction, genetic programming seems a promising technique to develop prediction rules for diagnostic and prognostic purposes.

Genomic profiling of gastric cancer predicts lymph node status and survival.

Gastric carcinogenesis is driven by an accumulation of genetic changes that to a large extent occur at the chromosomal level. We analysed the patterns of chromosomal instability in 35 gastric carcinomas and their clinical correlations. With microarray competitive genomic hybridization, genomewide chromosomal copy number changes can be studied with high resolution and sensitivity. A genomewide scanning array with 2275 BAC and P1 clones spotted in triplicate was used. This array provided an average resolution of 1.4 Mb across the genome. Patterns of chromosomal aberrations were analysed by hierarchical cluster analysis of the normalized log(2) tumour to normal fluorescence ratios of all clones, and cluster membership was correlated to clinicopathological data including survival. Hierarchical cluster analysis revealed three groups with different genomic profiles that correlated significantly with lymph node status (P=0.02). Moreover, gastric cancer cases from cluster 3 showed a significantly better prognosis than those from clusters 1 and 2 (P=0.02). Genomic profiling of gastric adenocarcinomas based on microarray analysis of chromosomal copy number changes predicted lymph node status and survival. The possibility to discriminate between patients with a high risk of lymph node metastasis could clinically be helpful for selecting patients for extended lymph node resection.