Asymptotic versus exact methods in the analysis of contingency tables: Evidence-based practical recommendations.

Controversy over the validity of significance tests in the analysis of contingency tables is motivated by the disagreement between asymptotic and exact p values and its dependence on the magnitude of expected frequencies. Variants of Pearson's X2 statistic and their asymptotic distributions were proposed to overcome the difficulties, but several approaches also exist to conduct exact tests. This paper shows that discrepant asymptotic and exact results may or may not occur whether expected frequencies are large or small: Eventual inaccuracy of asymptotic p values is instead caused by idiosyncrasies of the discrete distribution of X2. More importantly, discrepancies are also artificially created by the hypergeometric sampling model used to perform exact tests. Exact computations under the alternative full-multinomial or product-multinomial models require eliminating nuisance parameters and we propose a novel method that integrates them out. The resultant exact distributions are very accurately approximated by the asymptotic distribution, which eliminates concerns about the accuracy of the latter. We also discuss that the two-stage approach that tests for significance of residuals conditional on a significant X2 test is inadvisable and that an alternative single-stage test preserves Type-I error rates and further eliminates concerns about asymptotic accuracy.

Nonparametric tests for equality of psychometric functions.

Many empirical studies measure psychometric functions (curves describing how observers' performance varies with stimulus magnitude) because these functions capture the effects of experimental conditions. To assess these effects, parametric curves are often fitted to the data and comparisons are carried out by testing for equality of mean parameter estimates across conditions. This approach is parametric and, thus, vulnerable to violations of the implied assumptions. Furthermore, testing for equality of means of parameters may be misleading: Psychometric functions may vary meaningfully across conditions on an observer-by-observer basis with no effect on the mean values of the estimated parameters. Alternative approaches to assess equality of psychometric functions per se are thus needed. This paper compares three nonparametric tests that are applicable in all situations of interest: The existing generalized Mantel-Haenszel test, a generalization of the Berry-Mielke test that was developed here, and a split variant of the generalized Mantel-Haenszel test also developed here. Their statistical properties (accuracy and power) are studied via simulation and the results show that all tests are indistinguishable as to accuracy but they differ non-uniformly as to power. Empirical use of the tests is illustrated via analyses of published data sets and practical recommendations are given. The computer code in MATLAB and R to conduct these tests is available as Electronic Supplemental Material.

Error-rate estimation in discriminant analysis of non-linear longitudinal data: A comparison of resampling methods.

Consider longitudinal observations across different subjects such that the underlying distribution is determined by a non-linear mixed-effects model. In this context, we look at the misclassification error rate for allocating future subjects using cross-validation, bootstrap algorithms (parametric bootstrap, leave-one-out, .632 and [Formula: see text]), and bootstrap cross-validation (which combines the first two approaches), and conduct a numerical study to compare the performance of the different methods. The simulation and comparisons in this study are motivated by real observations from a pregnancy study in which one of the main objectives is to predict normal versus abnormal pregnancy outcomes based on information gathered at early stages. Since in this type of studies it is not uncommon to have insufficient data to simultaneously solve the classification problem and estimate the misclassification error rate, we put special attention to situations when only a small sample size is available. We discuss how the misclassification error rate estimates may be affected by the sample size in terms of variability and bias, and examine conditions under which the misclassification error rate estimates perform reasonably well.

Conditional overdispersed models: Application to count area data.

This paper proposes alternative models for the analysis of count data featuring a given spatial structure, which corresponds to geographical areas. We assume that the overdispersion data structure partially results from the existing and well justified spatial correlation between geographical adjacent regions, so an extension of existing overdispersion models that include spatial neighborhood structures within a Bayesian framework is proposed. These models allow practitioners to quantify the association explained by the considered neighborhood structures and the one modelled by additional factors. Finally, using the information provided by the Colombian National Demographic and Health Survey, the usefulness of the proposed models is illustrated by fitting them to infant mortality rates and to data including the proportion of mothers who, after giving birth to their last child, underwent a postnatal screening period in Colombia.

Analysis of residuals in contingency tables: another nail in the coffin of conditional approaches to significance testing.

Omnibus tests of significance in contingency tables use statistics of the chi-square type. When the null is rejected, residual analyses are conducted to identify cells in which observed frequencies differ significantly from expected frequencies. Residual analyses are thus conditioned on a significant omnibus test. Conditional approaches have been shown to substantially alter type I error rates in cases involving t tests conditional on the results of a test of equality of variances, or tests of regression coefficients conditional on the results of tests of heteroscedasticity. We show that residual analyses conditional on a significant omnibus test are also affected by this problem, yielding type I error rates that can be up to 6 times larger than nominal rates, depending on the size of the table and the form of the marginal distributions. We explored several unconditional approaches in search for a method that maintains the nominal type I error rate and found out that a bootstrap correction for multiple testing achieved this goal. The validity of this approach is documented for two-way contingency tables in the contexts of tests of independence, tests of homogeneity, and fitting psychometric functions. Computer code in MATLAB and R to conduct these analyses is provided as Supplementary Material.

Statistical approaches to analyse patient-reported outcomes as response variables: an application to health-related quality of life.

Patient-reported outcomes (PRO) are used as primary endpoints in medical research and their statistical analysis is an important methodological issue. Theoretical assumptions of the selected methodology and interpretation of its results are issues to take into account when selecting an appropriate statistical technique to analyse data. We present eight methods of analysis of a popular PRO tool under different assumptions that lead to different interpretations of the results. All methods were applied to responses obtained from two of the health dimensions of the SF-36 Health Survey. The proposed methods are: multiple linear regression (MLR), with least square and bootstrap estimations, tobit regression, ordinal logistic and probit regressions, beta-binomial regression (BBR), binomial-logit-normal regression (BLNR) and coarsening. Selection of an appropriate model depends not only on its distributional assumptions but also on the continuous or ordinal features of the response and the fact that they are constrained to a bounded interval. The BBR approach renders satisfactory results in a broad number of situations. MLR is not recommended, especially with skewed outcomes. Ordinal methods are only appropriate for outcomes with a few number of categories. Tobit regression is an acceptable option under normality assumptions and in the presence of moderate ceiling or floor effect. The BLNR and coarsening proposals are also acceptable, but only under certain distributional assumptions that are difficult to test a priori. Interpretation of the results is more convenient when using the BBR, BLNR and ordinal logistic regression approaches.

Statistical inference involving binomial and negative binomial parameters.

Statistical inference about two binomial parameters implies that they are both estimated by binomial sampling. There are occasions in which one aims at testing the equality of two binomial parameters before and after the occurrence of the first success along a sequence of Bernoulli trials. In these cases, the binomial parameter before the first success is estimated by negative binomial sampling whereas that after the first success is estimated by binomial sampling, and both estimates are related. This paper derives statistical tools to test two hypotheses, namely, that both binomial parameters equal some specified value and that both parameters are equal though unknown. Simulation studies are used to show that in small samples both tests are accurate in keeping the nominal Type-I error rates, and also to determine sample size requirements to detect large, medium, and small effects with adequate power. Additional simulations also show that the tests are sufficiently robust to certain violations of their assumptions.

Statistical Inference Involving Binomial and Negative Binomial Parameters

Statistical inference about two binomial parameters implies that they are both estimated by binomial sampling. There are occasions in which one aims at testing the equality of two binomial parameters before and after the occurrence of the first success along a sequence of Bernoulli trials. In these cases, the binomial parameter before the first success is estimated by negative binomial sampling whereas that after the first success is estimated by binomial sampling, and both estimates are related. This paper derives statistical tools to test two hypotheses, namely, that both binomial parameters equal some specified value and that both parameters are equal though unknown. Simulation studies are used to show that in small samples both tests are accurate in keeping the nominal Type-I error rates, and also to determine sample size requirements to detect large, medium, and small effects with adequate power. Additional simulations also show that the tests are sufficiently robust to certain violations of their assumptions (AU)

El contraste de hipótesis acerca de dos proporciones supone que cada una de ellas se ha estimado mediante muestreo binomial, pero hay ocasiones en que interesa evaluar la hipótesis de que la probabilidad de éxito a medida que se repite una determinada tarea varía una vez que se ha obtenido el primer éxito. En estos casos, la probabilidad de éxito antes de que ocurra el primer éxito se estima mediante muestreo binomial negativo, en tanto que la probabilidad de éxito después del primer éxito se estima mediante muestreo binomial, y ambas estimaciones están relacionadas. En este trabajo se presentan procedimientos para contrastar dos hipótesis aplicables a esta situación. Una es la de que las dos probabilidades son iguales y tienen un determinado valor; la otra es más general y sólo expresa que las dos probabilidades son iguales. El comportamiento de estos dos contrastes en muestras finitas se analiza mediante simulaciones cuyos resultados muestran que en ambos casos se preserva adecuadamente la tasa nominal de error de tipo I. También se ha determinado mediante simulación los tamaños muestrales necesarios para detectar efectos grandes, medianos o pequeños con potencia suficiente. Finalmente, otro grupo de simulaciones muestra que ambos contrastes son suficientemente robustos ante violaciones de sus supuestos (AU)

Analysis of the short form-36 (SF-36): the beta-binomial distribution approach.

Health-related quality of life (HRQoL) is an important indicator of health status and the Short Form-36 (SF-36) is a generic instrument to measure it. Multiple linear regression (MLR) is often used to study the relationship of HRQoL with patients' characteristics, though HRQoL outcomes tend to be not normally distributed, skewed and bounded (e.g. between 0 and 100). A sample of 193 patients with eating disorders has been analysed to assess the performance of the MLR under non-normality conditions. Normal distribution was rejected for seven out of the eight domains. A beta-binomial distribution is suggested to fit the SF-36 scores. The beta-binomial distribution is not rejected for five out of the eight domains. Thus, a beta-binomial regression (BBR) is suggested to analyse the SF-36 scores. Results using MLR and BBR have been compared for real and simulated data. Performance of the BBR is shown to be better than MLR in the HRQoL domains with few ordered categories and very similar to MLR in the more continuous domains. Moreover, the interpretation of the estimates obtained with BBR is clinically more meaningful. A common technique of statistical analysis is preferable for all the HRQoL dimensions. Therefore, the BBR approach is recommended not only to detect significant predictors of HRQoL when SF-36 is used, but also to analyse and interpret the effect of several explanatory variables on HRQoL. Further work is required to test the better performance of BBR against standard methods for other HRQoL outcomes, populations or interventions.

On the chi-square approximation to the exact distribution of goodness-of-fit statistics in multinomial models with composite hypotheses.

Multinomial models are increasingly being used in psychology, and this use always requires estimating model parameters and testing goodness of fit with a composite null hypothesis. Goodness of fit is customarily tested with recourse to the asymptotic approximation to the distribution of the statistics. An assessment of the quality of this approximation requires a comparison with the exact distribution, but how to compute this exact distribution when parameters are estimated from the data appears never to have been defined precisely. The main goal of this paper is to compare two different approaches to defining this exact distribution. One of the approaches uses the marginal distribution and is, therefore, independent of the data; the other approach uses the conditional distribution of the statistics given the estimated parameters and, therefore, is data-dependent. We carried out a thorough study involving various parameter estimation methods and goodness-of-fit statistics, all of them members of the general class of power-divergence measures. Included in the study were multinomial models with three to five cells and up to three parameters. Our results indicate that the asymptotic distribution is rarely a good approximation to the exact marginal distribution of the statistics, whereas it is a good approximation to the exact conditional distribution only when the vector of expected frequencies is interior to the sample space of the multinomial distribution.

Censored partial regression.

In this work we study the effect of several covariates on a censored response variable with unknown probability distribution. A semiparametric model is proposed to consider situations where the functional form of the effect of one or more covariates is unknown, as is the case in the application presented in this work. We provide its estimation procedure and, in addition, a bootstrap technique to make inference on the parameters. A simulation study has been carried out to show the good performance of the proposed estimation process and to analyse the effect of the censorship. Finally, we present the results when the methodology is applied to AIDS diagnosed patients.

Comparing proportional hazards and accelerated failure time models for survival analysis.

This paper describes a method proposed for a censored linear regression model that can be used in the context of survival analysis. The method has the important characteristic of allowing estimation and inference without knowing the distribution of the duration variable. Moreover, it does not need the assumption of proportional hazards. Therefore, it can be an interesting alternative to the Cox proportional hazards models when this assumption does not hold. In addition, implementation and interpretation of the results is simple. In order to analyse the performance of this methodology, we apply it to two real examples and we carry out a simulation study. We present its results together with those obtained with the traditional Cox model and AFT parametric models. The new proposal seems to lead to more precise results.