Discussion on "Optimal test procedures for multiple hypotheses controlling the familywise expected loss" by Willi Maurer, Frank Bretz, and Xiaolei Xun.

We discuss three issues. In the first part, we discuss the criteria emphasized by Maurer, Bretz, and Xun, warning that it modifies the per comparison error rate that does not address the concerns raised by multiple testing. In the second part, we strengthen the optimality results developed in the paper, based on our recent results. In the third part, we highlight the potentially important role that the use of weights may have in practice and discuss the difficulties in assigning weights that convey the importance in the gain and loss functions, especially as it pertains to multiple endpoints.

Optimal multiple testing and design in clinical trials.

A central goal in designing clinical trials is to find the test that maximizes power (or equivalently minimizes required sample size) for finding a false null hypothesis subject to the constraint of type I error. When there is more than one test, such as in clinical trials with multiple endpoints, the issues of optimal design and optimal procedures become more complex. In this paper, we address the question of how such optimal tests should be defined and how they can be found. We review different notions of power and how they relate to study goals, and also consider the requirements of type I error control and the nature of the procedures. This leads us to an explicit optimization problem with objective and constraints that describe its specific desiderata. We present a complete solution for deriving optimal procedures for two hypotheses, which have desired monotonicity properties, and are computationally simple. For some of the optimization formulations this yields optimal procedures that are identical to existing procedures, such as Hommel's procedure or the procedure of Bittman et al. (2009), while for other cases it yields completely novel and more powerful procedures than existing ones. We demonstrate the nature of our novel procedures and their improved power extensively in a simulation and on the APEX study (Cohen et al., 2016).

Relación entre el mes de nacimiento y la prevalencia de enfermedades crónicas / On relationship between patients' month of birth and the prevalence of chronic diseases

No disponible

FWER and FDR control when testing multiple mediators.

Motivation: The biological pathways linking exposures and disease risk are often poorly understood. To gain insight into these pathways, studies may try to identify biomarkers that mediate the exposure/disease relationship. Such studies often simultaneously test hundreds or thousands of biomarkers. Results: We consider a set of m biomarkers and a corresponding set of null hypotheses, where the jth null hypothesis states that biomarker j does not mediate the exposure/disease relationship. We propose a Multiple Comparison Procedure (MCP) that rejects a set of null hypotheses or, equivalently, identifies a set of mediators, while asymptotically controlling the Family-Wise Error Rate (FWER) or False Discovery Rate (FDR). We use simulations to show that, compared to currently available methods, our proposed method has higher statistical power to detect true mediators. We then apply our method to a breast cancer study and identify nine metabolites that may mediate the known relationship between an increased BMI and an increased risk of breast cancer. Availability and implementation: R package MultiMed on https://github.com/SiminaB/MultiMed. Supplementary information: Supplementary data are available at Bioinformatics online.

Reproducibility and replicability of rodent phenotyping in preclinical studies.

The scientific community is increasingly concerned with the proportion of published "discoveries" that are not replicated in subsequent studies. The field of rodent behavioral phenotyping was one of the first to raise this concern, and to relate it to other methodological issues: the complex interaction between genotype and environment; the definitions of behavioral constructs; and the use of laboratory mice and rats as model species for investigating human health and disease mechanisms. In January 2015, researchers from various disciplines gathered at Tel Aviv University to discuss these issues. The general consensus was that the issue is prevalent and of concern, and should be addressed at the statistical, methodological and policy levels, but is not so severe as to call into question the validity and the usefulness of model organisms as a whole. Well-organized community efforts, coupled with improved data and metadata sharing, have a key role in identifying specific problems and promoting effective solutions. Replicability is closely related to validity, may affect generalizability and translation of findings, and has important ethical implications.

Discrete False-Discovery Rate Improves Identification of Differentially Abundant Microbes.

Differential abundance testing is a critical task in microbiome studies that is complicated by the sparsity of data matrices. Here we adapt for microbiome studies a solution from the field of gene expression analysis to produce a new method, discrete false-discovery rate (DS-FDR), that greatly improves the power to detect differential taxa by exploiting the discreteness of the data. Additionally, DS-FDR is relatively robust to the number of noninformative features, and thus removes the problem of filtering taxonomy tables by an arbitrary abundance threshold. We show by using a combination of simulations and reanalysis of nine real-world microbiome data sets that this new method outperforms existing methods at the differential abundance testing task, producing a false-discovery rate that is up to threefold more accurate, and halves the number of samples required to find a given difference (thus increasing the efficiency of microbiome experiments considerably). We therefore expect DS-FDR to be widely applied in microbiome studies. IMPORTANCE DS-FDR can achieve higher statistical power to detect significant findings in sparse and noisy microbiome data compared to the commonly used Benjamini-Hochberg procedure and other FDR-controlling procedures.

Dietary Flavonoid Intake Reduces the Risk of Head and Neck but Not Esophageal or Gastric Cancer in US Men and Women.

Background: Flavonoids are bioactive polyphenolic compounds found in fruits, vegetables, and beverages of plant origin. Previous studies have shown that flavonoid intake reduces the risk of certain cancers; however, few studies to date have examined associations of flavonoids with upper gastrointestinal cancers or used prospective cohorts.Objective: Our study examined the association between intake of flavonoids (anthocyanidins, flavan-3-ols, flavanones, flavones, flavonols, and isoflavones) and risk of head and neck, esophageal, and gastric cancers.Methods: The NIH-AARP Diet and Health Study is a prospective cohort study that consists of 469,008 participants. Over a mean 12-y follow-up, 2453 head and neck (including 1078 oral cavity, 424 pharyngeal, and 817 laryngeal), 1165 esophageal (890 adenocarcinoma and 275 squamous cell carcinoma), and 1297 gastric (625 cardia and 672 noncardia) cancer cases were identified. We used Cox proportional hazards regression models to estimate HRs and CIs for the associations between flavonoid intake assessed at study baseline and cancer outcomes. For 56 hypotheses examined, P-trend values were adjusted using the Benjamini-Hochberg (BH) procedure for false discovery rate control.Results: The highest quintile of total flavonoid intake was associated with a 24% lower risk of head and neck cancer (HR: 0.76; 95% CI: 0.66, 0.86; BH-adjusted 95% CI: 0.63, 0.91; P-trend = 0.02) compared with the lowest quintile. Notably, anthocyanidins were associated with a 28% lower risk of head and neck cancer (HR: 0.72; 95% CI: 0.62, 0.82; BH-adjusted 95% CI: 0.59, 0.87; P-trend = 0.0005), and flavanones were associated with a 22% lower risk of head and neck cancer (HR: 0.78; 95% CI: 0.68, 0.89; BH-adjusted 95% CI: 0.64, 0.94; P-trend: 0.02). No associations between flavonoid intake and risk of esophageal or gastric cancers were found.Conclusions: Our results indicate that flavonoid intake is associated with lower head and neck cancer risk. These associations suggest a protective effect of dietary flavonoids on head and neck cancer risk, and thus potential as a risk reduction strategy.

Prevalence of sexual dimorphism in mammalian phenotypic traits.

The role of sex in biomedical studies has often been overlooked, despite evidence of sexually dimorphic effects in some biological studies. Here, we used high-throughput phenotype data from 14,250 wildtype and 40,192 mutant mice (representing 2,186 knockout lines), analysed for up to 234 traits, and found a large proportion of mammalian traits both in wildtype and mutants are influenced by sex. This result has implications for interpreting disease phenotypes in animal models and humans.

A powerful statistical framework for generalization testing in GWAS, with application to the HCHS/SOL.

In genome-wide association studies (GWAS), "generalization" is the replication of genotype-phenotype association in a population with different ancestry than the population in which it was first identified. Current practices for declaring generalizations rely on testing associations while controlling the family-wise error rate (FWER) in the discovery study, then separately controlling error measures in the follow-up study. This approach does not guarantee control over the FWER or false discovery rate (FDR) of the generalization null hypotheses. It also fails to leverage the two-stage design to increase power for detecting generalized associations. We provide a formal statistical framework for quantifying the evidence of generalization that accounts for the (in)consistency between the directions of associations in the discovery and follow-up studies. We develop the directional generalization FWER (FWERg ) and FDR (FDRg ) controlling r-values, which are used to declare associations as generalized. This framework extends to generalization testing when applied to a published list of Single Nucleotide Polymorphism-(SNP)-trait associations. Our methods control FWERg or FDRg under various SNP selection rules based on P-values in the discovery study. We find that it is often beneficial to use a more lenient P-value threshold than the genome-wide significance threshold. In a GWAS of total cholesterol in the Hispanic Community Health Study/Study of Latinos (HCHS/SOL), when testing all SNPs with P-values <5×10-8 (15 genomic regions) for generalization in a large GWAS of whites, we generalized SNPs from 15 regions. But when testing all SNPs with P-values <6.6×10-5 (89 regions), we generalized SNPs from 27 regions.

Improving the Identification of Phenotypic Abnormalities and Sexual Dimorphism in Mice When Studying Rare Event Categorical Characteristics.

Biological research frequently involves the study of phenotyping data. Many of these studies focus on rare event categorical data, and functional genomics studies typically study the presence or absence of an abnormal phenotype. With the growing interest in the role of sex, there is a need to assess the phenotype for sexual dimorphism. The identification of abnormal phenotypes for downstream research is challenged by the small sample size, the rare event nature, and the multiple testing problem, as many variables are monitored simultaneously. Here, we develop a statistical pipeline to assess statistical and biological significance while managing the multiple testing problem. We propose a two-step pipeline to initially assess for a treatment effect, in our case example genotype, and then test for an interaction with sex. We compare multiple statistical methods and use simulations to investigate the control of the type-one error rate and power. To maximize the power while addressing the multiple testing issue, we implement filters to remove data sets where the hypotheses to be tested cannot achieve significance. A motivating case study utilizing a large scale high-throughput mouse phenotyping data set from the Wellcome Trust Sanger Institute Mouse Genetics Project, where the treatment is a gene ablation, demonstrates the benefits of the new pipeline on the downstream biological calls.

Function of cancer associated genes revealed by modern univariate and multivariate association tests.

Copy number variation (CNV) plays a role in pathogenesis of many human diseases, especially cancer. Several whole genome CNV association studies have been performed for the purpose of identifying cancer associated CNVs. Here we undertook a novel approach to whole genome CNV analysis, with the goal being identification of associations between CNV of different genes (CNV-CNV) across 60 human cancer cell lines. We hypothesize that these associations point to the roles of the associated genes in cancer, and can be indicators of their position in gene networks of cancer-driving processes. Recent studies show that gene associations are often non-linear and non-monotone. In order to obtain a more complete picture of all CNV associations, we performed omnibus univariate analysis by utilizing dCov, MIC, and HHG association tests, which are capable of detecting any type of association, including non-monotone relationships. For comparison we used Spearman and Pearson association tests, which detect only linear or monotone relationships. Application of dCov, MIC and HHG tests resulted in identification of twice as many associations compared to those found by Spearman and Pearson alone. Interestingly, most of the new associations were detected by the HHG test. Next, we utilized dCov's and HHG's ability to perform multivariate analysis. We tested for association between genes of unknown function and known cancer-related pathways. Our results indicate that multivariate analysis is much more effective than univariate analysis for the purpose of ascribing biological roles to genes of unknown function. We conclude that a combination of multivariate and univariate omnibus association tests can reveal significant information about gene networks of disease-driving processes. These methods can be applied to any large gene or pathway dataset, allowing more comprehensive analysis of biological processes.

Is this the right normalization? A diagnostic tool for ChIP-seq normalization.

BACKGROUND: Chip-seq experiments are becoming a standard approach for genome-wide profiling protein-DNA interactions, such as detecting transcription factor binding sites, histone modification marks and RNA Polymerase II occupancy. However, when comparing a ChIP sample versus a control sample, such as Input DNA, normalization procedures have to be applied in order to remove experimental source of biases. Despite the substantial impact that the choice of the normalization method can have on the results of a ChIP-seq data analysis, their assessment is not fully explored in the literature. In particular, there are no diagnostic tools that show whether the applied normalization is indeed appropriate for the data being analyzed. RESULTS: In this work we propose a novel diagnostic tool to examine the appropriateness of the estimated normalization procedure. By plotting the empirical densities of log relative risks in bins of equal read count, along with the estimated normalization constant, after logarithmic transformation, the researcher is able to assess the appropriateness of the estimated normalization constant. We use the diagnostic plot to evaluate the appropriateness of the estimates obtained by CisGenome, NCIS and CCAT on several real data examples. Moreover, we show the impact that the choice of the normalization constant can have on standard tools for peak calling such as MACS or SICER. Finally, we propose a novel procedure for controlling the FDR using sample swapping. This procedure makes use of the estimated normalization constant in order to gain power over the naive choice of constant (used in MACS and SICER), which is the ratio of the total number of reads in the ChIP and Input samples. CONCLUSIONS: Linear normalization approaches aim to estimate a scale factor, r, to adjust for different sequencing depths when comparing ChIP versus Input samples. The estimated scaling factor can easily be incorporated in many peak caller algorithms to improve the accuracy of the peak identification. The diagnostic plot proposed in this paper can be used to assess how adequate ChIP/Input normalization constants are, and thus it allows the user to choose the most adequate estimate for the analysis.

Deciding whether follow-up studies have replicated findings in a preliminary large-scale omics study.

We propose a formal method to declare that findings from a primary study have been replicated in a follow-up study. Our proposal is appropriate for primary studies that involve large-scale searches for rare true positives (i.e., needles in a haystack). Our proposal assigns an r value to each finding; this is the lowest false discovery rate at which the finding can be called replicated. Examples are given and software is available.

repfdr: a tool for replicability analysis for genome-wide association studies.

MOTIVATION: Identification of single nucleotide polymorphisms that are associated with a phenotype in more than one study is of great scientific interest in the genome-wide association studies (GWAS) research. The empirical Bayes approach for discovering whether results have been replicated across studies was shown to be a reliable method, and close to optimal in terms of power. RESULTS: The R package repfdr provides a flexible implementation of the empirical Bayes approach for replicability analysis and meta-analysis, to be used when several studies examine the same set of null hypotheses. The usefulness of the package for the GWAS community is discussed. AVAILABILITY AND IMPLEMENTATION: The R package repfdr can be downloaded from CRAN. CONTACT: ruheller@gmail.com SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

Outbreaks of severe enteric disease associated with Eimeria furonis infection in ferrets (Mustela putorius furo) of 3 densely populated groups.

CASE DESCRIPTION: 3 unrelated, densely populated, dynamic ferret populations with severe outbreaks of enteric coccidiosis were evaluated. CLINICAL FINDINGS: In each outbreak, morbidity rate was high, there were an appreciable number of deaths, and ferrets of all ages were affected. Affected individuals had acute onset of diarrhea, and feces often contained frank or digested blood. Other clinical signs included dehydration, weakness, lethargy, and weight loss. Fecal examinations of affected ferrets revealed sporadic and inconsistent shedding of coccidial oocysts. Necropsy findings included moderate to marked atrophic enteritis associated with numerous intraepithelial and fewer extracellular coccidial life stages. Sporulated oocysts isolated from feces were consistent with Eimeria furonis. A PCR assay was performed on formalin-fixed, paraffin-embedded sections of intestine for the gene encoding the small subunit of rRNA yielded products with sequences identical to those described for E furonis. TREATMENT AND OUTCOME: Supportive care and treatment with sulfadimethoxine over the course of these outbreaks was palliative, but long-term treatment was required and failed to completely eradicate infection as identified by the subsequent finding of oocysts in fecal samples. CLINICAL RELEVANCE: Enteric coccidiosis due to infection with E furonis has typically been reported to be subclinical rather than to cause severe gastrointestinal disease in ferrets. This report indicated that infection with E furonis may have contributed to severe enteric disease with high morbidity and mortality rates in 3 densely populated, dynamic groups of ferrets. Furthermore, long-term treatment with anti-coccidials may be required in outbreak situations, but may be ineffectual in completely eradicating infection.

Selective inference in complex research.

We explain the problem of selective inference in complex research using a recently published study: a replicability study of the associations in order to reveal and establish risk loci for type 2 diabetes. The false discovery rate approach to such problems will be reviewed, and we further address two problems: (i) setting confidence intervals on the size of the risk at the selected locations and (ii) selecting the replicable results.

A flexible two-stage procedure for identifying gene sets that are differentially expressed.

MOTIVATION: Microarray data analysis has expanded from testing individual genes for differential expression to testing gene sets for differential expression. The tests at the gene set level may focus on multivariate expression changes or on the differential expression of at least one gene in the gene set. These tests may be powerful at detecting subtle changes in expression, but findings at the gene set level need to be examined further to understand whether they are informative and if so how. RESULTS: We propose to first test for differential expression at the gene set level but then proceed to test for differential expression of individual genes within discovered gene sets. We introduce the overall false discovery rate (OFDR) as an appropriate error rate to control when testing multiple gene sets and genes. We illustrate the advantage of this procedure over procedures that only test gene sets or individual genes.