Design, analysis, and interpretation of treatment response heterogeneity in personalized nutrition and obesity treatment research.

It is increasingly assumed that there is no one-size-fits-all approach to dietary recommendations for the management and treatment of chronic diseases such as obesity. This phenomenon that not all individuals respond uniformly to a given treatment has become an area of research interest given the rise of personalized and precision medicine. To conduct, interpret, and disseminate this research rigorously and with scientific accuracy, however, requires an understanding of treatment response heterogeneity. Here, we define treatment response heterogeneity as it relates to clinical trials, provide statistical guidance for measuring treatment response heterogeneity, and highlight study designs that can quantify treatment response heterogeneity in nutrition and obesity research. Our goal is to educate nutrition and obesity researchers in how to correctly identify and consider treatment response heterogeneity when analyzing data and interpreting results, leading to rigorous and accurate advancements in the field of personalized medicine.

Randomization, design and analysis for interdependency in aging research: no person or mouse is an island.

Investigators traditionally use randomized designs and corresponding analysis procedures to make causal inferences about the effects of interventions, assuming independence between an individual's outcome and treatment assignment and the outcomes of other individuals in the study. Often, such independence may not hold. We provide examples of interdependency in model organism studies and human trials and group effects in aging research and then discuss methodologic issues and solutions. We group methodologic issues as they pertain to (1) single-stage individually randomized trials; (2) cluster-randomized controlled trials; (3) pseudo-cluster-randomized trials; (4) individually randomized group treatment; and (5) two-stage randomized designs. Although we present possible strategies for design and analysis to improve the rigor, accuracy and reproducibility of the science, we also acknowledge real-world constraints. Consequences of nonadherence, differential attrition or missing data, unintended exposure to multiple treatments and other practical realities can be reduced with careful planning, proper study designs and best practices.

Estimating the range of obesity treatment response variability in humans: methods and illustrations.

BACKGROUND/AIMS: The rising prevalence of human obesity worldwide has focused research on a variety of interventions that result in highly varied degrees of weight loss (WL). The advent of genomic testing has quantified estimates of both the contribution of genetic factors to the development of obesity as well as racial/ethnic variation of risk alleles across subpopulations. More recent studies have examined genetic associations with effectiveness of WL interventions, but to date are unable to explain a large proportion of the variance observed. METHODS: We describe and provide two illustrations of statistical methods to estimate upper and lower bounds of WL treatment response heterogeneity (TRH) in the absence of genotypic data, using published summary statistics and a raw data set from WL studies. RESULTS: Thirty-two studies had some evidence of a positive mean treatment effect with respect to the control intervention. Twelve of these 32 studies reported WL TRH. Of these 12, 3 demonstrated an estimated proportion of >5% of the sampled population having an outcome opposite the mean effect. In the raw data set, bounds estimations for change in waist circumference revealed tighter ranges in men than women. CONCLUSION: Future studies may be able to take advantage of multiple approaches, including the method we describe, to identify and quantify the presence of TRH in studies of WL or related outcomes.

Treatment Heterogeneity and Individual Qualitative Interaction.

Plausibility of high variability in treatment effects across individuals has been recognized as an important consideration in clinical studies. Surprisingly, little attention has been given to evaluating this variability in design of clinical trials or analyses of resulting data. High variation in a treatment's efficacy or safety across individuals (referred to herein as treatment heterogeneity) may have important consequences because the optimal treatment choice for an individual may be different from that suggested by a study of average effects. We call this an individual qualitative interaction (IQI), borrowing terminology from earlier work - referring to a qualitative interaction (QI) being present when the optimal treatment varies across a"groups" of individuals. At least three techniques have been proposed to investigate treatment heterogeneity: techniques to detect a QI, use of measures such as the density overlap of two outcome variables under different treatments, and use of cross-over designs to observe "individual effects." We elucidate underlying connections among them, their limitations and some assumptions that may be required. We do so under a potential outcomes framework that can add insights to results from usual data analyses and to study design features that improve the capability to more directly assess treatment heterogeneity.

Inappropriate fiddling with statistical analyses to obtain a desirable p-value: tests to detect its presence in published literature.

Much has been written regarding p-values below certain thresholds (most notably 0.05) denoting statistical significance and the tendency of such p-values to be more readily publishable in peer-reviewed journals. Intuition suggests that there may be a tendency to manipulate statistical analyses to push a "near significant p-value" to a level that is considered significant. This article presents a method for detecting the presence of such manipulation (herein called "fiddling") in a distribution of p-values from independent studies. Simulations are used to illustrate the properties of the method. The results suggest that the method has low type I error and that power approaches acceptable levels as the number of p-values being studied approaches 1000.

Submitted for your consideration: potential advantages of a novel clinical trial design and initial patient reaction.

In many circumstances, individuals do not respond identically to the same treatment. This phenomenon, which is called treatment response heterogeneity (TRH), appears to be present in treatments for many conditions, including obesity. Estimating the total amount of TRH, predicting an individual's response, and identifying the mediators of TRH are of interest to biomedical researchers. Clinical investigators and physicians commonly postulate that some of these mediators could be genetic. Current designs can estimate TRH as a function of specific, measurable observed factors; however, they cannot estimate the total amount of TRH, nor provide reliable estimates of individual persons' responses. We propose a new repeated randomizations design (RRD), which can be conceived as a generalization of the Balaam design, that would allow estimates of that variability and facilitate estimation of the total amount of TRH, prediction of an individual's response, and identification of the mediators of TRH. In a pilot study, we asked 118 subjects entering a weight loss trial for their opinion of the RRD, and they stated a preference for the RRD over the conventional two-arm parallel groups design. Research is needed as to how the RRD will work in practice and its relative statistical properties, and we invite dialog about it.

Validity and power of missing data imputation for extreme sampling and terminal measures designs in mediation analysis.

Several authors have acknowledged that testing mediational hypotheses between treatments, genes, physiological measures, and behaviors may substantially advance our understanding of how these associations operate. In psychiatric research, the costs of measuring the putative mediator or the outcome can be prohibitive. Extreme sampling designs have been validated as methods for reducing study costs by increasing power per subject measured on the more expensive variable when assessing bivariate relationships. However, there exist concerns about how missing data can potentially bias the results. Additionally, most mediation analysis techniques presuppose the joint measurement of mediators and outcomes for all subjects. There have been limited methodological developments for techniques that can evaluate putative mediators in studies that have employed extreme sampling, resulting in missing data. We demonstrate that extreme (selective) sampling strategies can be beneficial in the context of mediation analyses. Handling the missing data with maximum likelihood (ML) resulted in minimal power loss and unbiased parameter estimates. We must be cautious, though, in recommending the ML approach for extreme sampling designs because it yielded inflated Type 1 error rates under some null conditions. Yet, the use of extreme sampling designs and methods to handle the resultant missing data presents a viable research strategy.

Genes and biochemical pathways in human skeletal muscle affecting resting energy expenditure and fuel partitioning.

Genes influencing resting energy expenditure (REE) and respiratory quotient (RQ) represent candidate genes for obesity and the metabolic syndrome because of the involvement of these traits in energy balance and substrate oxidation. We aim to explore the molecular basis for individual variation in REE and fuel partitioning as reflected by RQ. We performed microarray studies in human vastus lateralis muscle biopsies from 40 healthy subjects with measured REE and RQ values. We identified 2,392 and 1,115 genes significantly correlated with REE and RQ, respectively. Genes correlated with REE and RQ encompass a broad array of functions, including carbohydrate and lipid metabolism, gene expression, mitochondrial processes, and membrane transport. Microarray pathway analysis revealed that REE was positively correlated with upregulation of G protein-coupled receptor signaling (meet criteria/total genes: 65 of 283) involved in autonomic nervous system functions, including those receptors mediating adrenergic, dopamine, γ-aminobutyric acid (GABA), neuropeptide Y (NPY), and serotonin action (meet criteria/total genes: 46 of 176). Reduced REE was associated with an increase in genes participating in ubiquitin-proteasome-dependent proteolytic pathways (58 of 232). Serine-type peptidase activity (9 of 76) was positively correlated with RQ, while genes involved in the protein phosphatase type 2A complex (4 of 9), mitochondrial function and cellular respiration (38 of 315), and unfolded protein binding (19 of 97) were associated with reduced RQ values and a preference for lipid fuel metabolism. Individual variations in whole body REE and RQ are regulated by differential expressions of specific genes and pathways intrinsic to skeletal muscle.

Illustrations on Using the Distribution of a P-value in High Dimensional Data Analyses.

Several statistical methods have recently been developed that use the distribution of P-values from multiple tests of hypotheses to analyze data from high-dimensional experiments. These methods are only as valid as the P-values that were derived from test statistics. If an incorrect distribution for a test statistic was used, the P-value will not be valid and the distribution of P-values from multiple test statistics could give misleading results. Moreover, if the correct distribution of a test statistic is used, a distribution of P-values may still give misleading results if P-values are correlated. A primary focus of this paper is on the distribution of a P-value under a null hypothesis, and the test statistic that is considered is the number of rejected null hypotheses. Two issues are demonstrated using six data examples, two that are simulated and four from actual microarray experiments. The results provide some insight into how much of an effect might be introduced into a distribution of P-values if invalid P-values are computed or if P-values are correlated. Additional illustration is given regarding the distribution of a P-value under an alternative hypothesis and some approaches to modeling it are presented.

Challenges and approaches to statistical design and inference in high-dimensional investigations.

Advances in modern technologies have facilitated high-dimensional experiments (HDEs) that generate tremendous amounts of genomic, proteomic, and other "omic" data. HDEs involving whole-genome sequences and polymorphisms, expression levels of genes, protein abundance measurements, and combinations thereof have become a vanguard for new analytic approaches to the analysis of HDE data. Such situations demand creative approaches to the processes of statistical inference, estimation, prediction, classification, and study design. The novel and challenging biological questions asked from HDE data have resulted in many specialized analytic techniques being developed. This chapter discusses some of the unique statistical challenges facing investigators studying high-dimensional biology and describes some approaches being developed by statistical scientists. We have included some focus on the increasing interest in questions involving testing multiple propositions simultaneously, appropriate inferential indicators for the types of questions biologists are interested in, and the need for replication of results across independent studies, investigators, and settings. A key consideration inherent throughout is the challenge in providing methods that a statistician judges to be sound and a biologist finds informative.

Evaluating statistical methods using plasmode data sets in the age of massive public databases: an illustration using false discovery rates.

Plasmode is a term coined several years ago to describe data sets that are derived from real data but for which some truth is known. Omic techniques, most especially microarray and genomewide association studies, have catalyzed a new zeitgeist of data sharing that is making data and data sets publicly available on an unprecedented scale. Coupling such data resources with a science of plasmode use would allow statistical methodologists to vet proposed techniques empirically (as opposed to only theoretically) and with data that are by definition realistic and representative. We illustrate the technique of empirical statistics by consideration of a common task when analyzing high dimensional data: the simultaneous testing of hundreds or thousands of hypotheses to determine which, if any, show statistical significance warranting follow-on research. The now-common practice of multiple testing in high dimensional experiment (HDE) settings has generated new methods for detecting statistically significant results. Although such methods have heretofore been subject to comparative performance analysis using simulated data, simulating data that realistically reflect data from an actual HDE remains a challenge. We describe a simulation procedure using actual data from an HDE where some truth regarding parameters of interest is known. We use the procedure to compare estimates for the proportion of true null hypotheses, the false discovery rate (FDR), and a local version of FDR obtained from 15 different statistical methods.

Application of potential outcomes to an intentional weight loss latent variable problem.

Studies that explore the link between weight loss among obese individuals and mortality have met with mixed results. One possible explanation is that total weight loss may have contributions from weight loss that is intentional and weight loss that is unintentional. The latter may be due to some underlying condition that has a deleterious effect on subsequent mortality. Some studies have then focused on subjects who intend to lose weight. However, in a population there is no guarantee that weight loss among these individuals is due only to their intention. This paper extends the work of Coffey et al., (2005) who treated intentional weight loss as a latent variable. In particular, the problem is reformulated using potential outcomes. This formulation more clearly identifies a nonestimable correlation that arises because of the latent variable, and it allows for the incorporation of covariate information that can tighten estimable bounds for this correlation. We show in a data set from an experiment on mice that substantial tightening of bounds is possible with a covariate that is predictive of weight loss. These bounds can then, in turn, be used to estimate bounds on a causal parameter in a linear model.

Epistemological issues in omics and high-dimensional biology: give the people what they want.

Gene expression microarrays have been the vanguard of new analytic approaches in high-dimensional biology. Draft sequences of several genomes coupled with new technologies allow study of the influences and responses of entire genomes rather than isolated genes. This has opened a new realm of highly dimensional biology where questions involve multiplicity at unprecedented scales: thousands of genetic polymorphisms, gene expression levels, protein measurements, genetic sequences, or any combination of these and their interactions. Such situations demand creative approaches to the processes of inference, estimation, prediction, classification, and study design. Although bench scientists intuitively grasp the need for flexibility in the inferential process, the elaboration of formal supporting statistical frameworks is just at the very start. Here, we will discuss some of the unique statistical challenges facing investigators studying high-dimensional biology, describe some approaches being developed by statistical scientists, and offer an epistemological framework for the validation of proffered statistical procedures. A key theme will be the challenge in providing methods that a statistician judges to be sound and a biologist finds informative. The shift from family-wise error rate control to false discovery rate estimation and to assessment of ranking and other forms of stability will be portrayed as illustrative of approaches to this challenge.

The PowerAtlas: a power and sample size atlas for microarray experimental design and research.

BACKGROUND: Microarrays permit biologists to simultaneously measure the mRNA abundance of thousands of genes. An important issue facing investigators planning microarray experiments is how to estimate the sample size required for good statistical power. What is the projected sample size or number of replicate chips needed to address the multiple hypotheses with acceptable accuracy? Statistical methods exist for calculating power based upon a single hypothesis, using estimates of the variability in data from pilot studies. There is, however, a need for methods to estimate power and/or required sample sizes in situations where multiple hypotheses are being tested, such as in microarray experiments. In addition, investigators frequently do not have pilot data to estimate the sample sizes required for microarray studies. RESULTS: To address this challenge, we have developed a Microrarray PowerAtlas. The atlas enables estimation of statistical power by allowing investigators to appropriately plan studies by building upon previous studies that have similar experimental characteristics. Currently, there are sample sizes and power estimates based on 632 experiments from Gene Expression Omnibus (GEO). The PowerAtlas also permits investigators to upload their own pilot data and derive power and sample size estimates from these data. This resource will be updated regularly with new datasets from GEO and other databases such as The Nottingham Arabidopsis Stock Center (NASC). CONCLUSION: This resource provides a valuable tool for investigators who are planning efficient microarray studies and estimating required sample sizes.

HDBStat!: a platform-independent software suite for statistical analysis of high dimensional biology data.

BACKGROUND: Many efforts in microarray data analysis are focused on providing tools and methods for the qualitative analysis of microarray data. HDBStat! (High-Dimensional Biology-Statistics) is a software package designed for analysis of high dimensional biology data such as microarray data. It was initially developed for the analysis of microarray gene expression data, but it can also be used for some applications in proteomics and other aspects of genomics. HDBStat! provides statisticians and biologists a flexible and easy-to-use interface to analyze complex microarray data using a variety of methods for data preprocessing, quality control analysis and hypothesis testing. RESULTS: Results generated from data preprocessing methods, quality control analysis and hypothesis testing methods are output in the form of Excel CSV tables, graphs and an Html report summarizing data analysis. CONCLUSION: HDBStat! is a platform-independent software that is freely available to academic institutions and non-profit organizations. It can be downloaded from our website http://www.soph.uab.edu/ssg_content.asp?id=1164.

The effects of intentional weight loss as a latent variable problem.

Although obesity is associated with increased mortality rate and short-term weight loss improves risk factors for mortality, it has not been convincingly shown that weight loss among obese people results in reduced mortality rate. When considering the human literature, it has been pointed out that weight loss is often a sign of illness and that investigators therefore need to separate intentional from unintentional weight loss. It has generally been assumed that among people who state that they do not intend to lose weight, weight change subsequently observed is unintentional. Among such people, weight loss has been consistently associated with increased mortality rate. Complementarily, it has generally been assumed that among people who state that they do intend to lose weight, weight change subsequently observed is intentional. In these people who are intending to lose weight, some studies show apparent benefits of weight loss, some are neutral, and some show deleterious effects. The overall conclusion that some reviewers have drawn from this literature is that intentional weight loss (IWL) is at best not beneficial and may even be harmful with respect to mortality rate. We believe that this conclusion is drawn by inappropriately conflating weight loss (or more generally weight change) among people intending to lose weight with IWL (or change). Herein, under certain assumptions, we: (1) show that the association between mortality rate and weight loss among people intending to lose weight and between mortality rate and IWL are two different things; (2) show that the association between IWL and mortality rate is an inherently unobservable entity; (3) derive a method for estimating the plausible range of true effect of IWL on mortality rate if one is willing to make a number of restrictive, but perhaps reasonable assumptions; and (4) illustrate the method by application to a data set involving middle-age onset calorie restriction in mice.

Assessing treatment effect heterogeneity in clinical trials with blocked binary outcomes.

This paper addresses treatment effect heterogeneity (also referred to, more compactly, as 'treatment heterogeneity') in the context of a controlled clinical trial with binary endpoints. Treatment heterogeneity, variation in the true (causal) individual treatment effects, is explored using the concept of the potential outcome. This framework supposes the existance of latent responses for each subject corresponding to each possible treatment. In the context of a binary endpoint, treatment heterogeniety may be represented by the parameter, pi2, the probability that an individual would have a failure on the experimental treatment, if received, and would have a success on control, if received. Previous research derived bounds for pi2 based on matched pairs data. The present research extends this method to the blocked data context. Estimates (and their variances) and confidence intervals for the bounds are derived. We apply the new method to data from a renal disease clinical trial. In this example, bounds based on the blocked data are narrower than the corresponding bounds based only on the marginal success proportions. Some remaining challenges (including the possibility of further reducing bound widths) are discussed.

Results of soy-based meal replacement formula on weight, anthropometry, serum lipids & blood pressure during a 40-week clinical weight loss trial.

BACKGROUND: To evaluate the intermediate-term health outcomes associated with a soy-based meal replacement, and to compare the weight loss efficacy of two distinct patterns of caloric restriction. METHODS: Ninety overweight/obese (28 < BMI < or = 41 kg/m2) adults received a single session of dietary counseling and were randomized to either 12 weeks at 1200 kcal/day, 16 weeks at 1500 kcal/d and 12 weeks at 1800 kcal/d (i.e., the 12/15/18 diet group), or 28 weeks at 1500 kcal/d and 12 weeks at 1800 kcal/d (i.e., the 15/18 diet group). Weight, body fat, waist circumference, blood pressure and serum lipid concentrations were measured at 4-week intervals throughout the 40-week trial. RESULTS: Subjects in both treatments showed statistically significant improvements in outcomes. A regression model for weight change suggests that subjects with larger baseline weights tended to lose more weight and subjects in the 12/15/18 group tended to experience, on average, an additional 0.9 kg of weight loss compared with subjects in the 15/18 group. CONCLUSION: Both treatments using the soy-based meal replacement program were associated with significant and comparable weight loss and improvements on selected health variables.

Cause-effect relationships in analytical surveys: an illustration of statistical issues.

Establishing cause-effect is critical in the field of natural resources where one may want to know the impact of management practices, wildfires, drought, etc. on water quality and quantity, wildlife, growth and survival of desirable trees for timber production, etc. Yet, key obstacles exist when trying to establish cause-effect in such contexts. Issues involved with identifying a causal hypothesis, and conditions needed to estimate a causal effect or to establish cause-effect are considered. Ideally one conducts an experiment and follows with a survey, or vice versa. In an experiment, the population of inference may be quite limited and in surveys, the probability distribution of treatment assignments is generally unknown and, if not accounted for, can cause serious errors when estimating causal effects. The latter is illustrated in simulation experiments of artificially generated forest populations using annual plot mortality as the response, drought as the cause, and age as a covariate that is correlated with mortality. We also consider the role of a vague unobservable covariate such as 'drought susceptibility'. Recommendations are made designed to maximize the possibility of identifying cause-effect relationships in large-scale natural resources surveys.