Frequentist and Bayesian Hypothesis Testing: An Intuitive Guide for Urologists and Clinicians / Pruebas de hipótesis frecuentista y bayesiana: Una Guía intuitiva para urólogos y clínicos

Given the limitations of frequentist method for null hypothesis significance testing, different authors recommend alternatives such as Bayesian inference. A poor understanding of both statistical frameworks is common among clinicians. The present is a gentle narrative review of the frequentist and Bayesian methods intended for physicians not familiar with mathematics. The frequentist p-value is the probability of finding a value equal to or higher than that observed in a study, assuming that the null hypothesis (H0) is true. The H0 is rejected or not based on a p threshold of 0.05, and this dichotomous approach does not express the probability that the alternative hypothesis (H1) is true. The Bayesian method calculates the probability of H1 and H0 considering prior odds and the Bayes factor (Bf). Prior odds are the researcher's belief about the probability of H1, and the Bf quantifies how consistent the data is concerning H1 and H0. The Bayesian prediction is not dichotomous but is expressed in continuous scales of the Bf and of the posterior odds. The JASP software enables the performance of both frequentist and Bayesian analyses in a friendly and intuitive way, and its application is displayed at the end of the paper. In conclusion, the frequentist method expresses how consistent the data is with H0 in terms of p-values, with no consideration of the probability of H1. The Bayesian model is a more comprehensive prediction because it quantifies in continuous scales the evidence for H1 versus H0 in terms of the Bf and the

Dadas las limitaciones del método de significancia frecuentista basado en la hipótesis nula, diferentes autores recomiendan alternativas como la inferencia bayesiana. Es común entre los médicos una comprensión deficiente de ambos marcos estadísticos. Esta es una revisión narrativa amigable de los métodos frecuentista y bayesiano dirigida quienes no están familiarizados con las matemáticas. El valor de p frecuentista es la probabilidad de encontrar un valor igual o superior al observado en un estudio, asumiendo que la hipótesis nula (H0) es cierta. La H0 se rechaza o no con base en un umbral p de 0.05, y este enfoque dicotómico no expresa la probabilidad de que la hipótesis alternativa (H1) sea verdadera. El método bayesiano calcula la probabilidad de H1 y H0 considerando las probabilidades a priori y el factor de Bayes (fB). Las probabilidades a priori son la creencia del investigador sobre la probabilidad de H1, y el fB cuantifica cuán consistentes son los datos con respecto a H1 y H0. La predicción bayesiana no es dicotómica, sino que se expresa en escalas continuas del fB y de las probabilidades a posteriori. El programa JASP permite realizar análisis frecuentista y bayesiano de una forma simple e intuitiva, y su aplicación se muestra al final del documento. En conclusión, el método frecuentista expresa cuán consistentes son los datos con H0 en términos de valores p, sin considerar la probabilidad de H1. El modelo bayesiano es una predicción más completa porque cuantifica en escalas continuas la evidencia de H1 versus H0 en términos del fB y de las probabilidades a posteriori.

Uso del factor Bayes durante el análisis estadístico: un ejemplo / Use of the Bayes factor during statistical analysis: an example

El factor de Bayes resulta una prueba recomendable para la comprobación de las hipótesis esta-dísticas atendiendo al estado de los p valores, empleando la escala de clasificación de Jeffreys preferiblemente

The Bayes factor is a recommended test for the verification of statistical hypotheses taking into account the state of the p values, preferably using the Jeffreys classification scale.

O valor de p é adequadamente interpretado pelos profissionais de terapia intensiva? Uma pesquisa on-line / Is the p-value properly interpreted by critical care professionals? Online survey

RESUMO Objetivo: Determinar a prevalência e os fatores de risco para conhecimento insuficiente sobre valores de p entre médicos e terapeutas respiratórios atuantes em terapia intensiva na Argentina. Métodos: Levantamento transversal on-line com 25 questões relativas às características dos participantes, autopercepção e conhecimento sobre valores de p (teoria e prática). Realizaram-se análises de estatística descritiva e regressão logística multivariada. Resultados: Analisaram-se 376 participantes. Não tinham conhecimento a respeito dos valores de p 237 participantes (63,1%). Segundo análise de regressão logística multivariada, falta de treinamento em metodologia científica (RC ajustadas 2,50; IC95% 1,37 - 4,53; p = 0,003) e a quantidade de leitura (< 6 artigos científicos por ano; RC ajustadas 3,27; IC95% 1,67 - 6,40; p = 0,001) foram identificados como independentemente associados com a falta de conhecimento sobre valores de p por parte dos participantes. Conclusão: A prevalência de conhecimento insuficiente com relação a valores de p entre médicos e terapeutas respiratórios na Argentina foi de 63%. Falta de treinamento em metodologia científica e quantidade de leitura (< 6 artigos científicos por ano) foram identificados como independentemente associados com a falta de conhecimento sobre valores de p por parte dos participantes.

ABSTRACT Objective: To determine the prevalence of and risk factors for insufficient knowledge related to p-values among critical care physicians and respiratory therapists in Argentina. Methods: This cross-sectional online survey contained 25 questions about respondents' characteristics, self-perception and p-value knowledge (theory and practice). Descriptive and multivariable logistic regression analyses were conducted. Results: Three hundred seventy-six respondents were analyzed. Two hundred thirty-seven respondents (63.1%) did not know about p-values. According to the multivariable logistic regression analysis, a lack of training on scientific research methodology (adjusted OR 2.50; 95%CI 1.37 - 4.53; p = 0.003) and the amount of reading (< 6 scientific articles per year; adjusted OR 3.27; 95%CI 1.67 - 6.40; p = 0.001) were found to be independently associated with the respondents' lack of p-value knowledge. Conclusion: The prevalence of insufficient knowledge regarding p-values among critical care physicians and respiratory therapists in Argentina was 63%. A lack of training on scientific research methodology and the amount of reading (< 6 scientific articles per year) were found to be independently associated with the respondents' lack of p-value knowledge.

Teste de hipóteses: perguntas que você sempre quis fazer, mas nunca teve coragem / Hypothesis testing: questions you have always wanted to ask, but never had the courage to

Dando continuidade aos artigos da série "Perguntas que você sempre quis fazer, mas nunca teve coragem", que tem como objetivo responder e sugerir referências para o melhor entendimento das principais dúvidas dos pesquisadores do Hospital de Clínicas de Porto Alegre sobre estatística, este segundo artigo se propõe a responder às principais dúvidas levantadas sobre Teste de Hipóteses. São discutidas questões referentes à metodologia de um teste de hipóteses na concepção clássica de Inferência Estatística, bem como tamanho de efeito, tipos de erros, valor de p e poder. Os conceitos são abordados numa linguagem acessível ao público leigo e diversas referências são sugeridas para os curiosos em relação ao tema. (AU)

Continuing the series of articles "Questions you have always wanted to ask, but never had the courage to", which aims to answer the most common questions of researchers at Hospital de Clínicas de Porto Alegre regarding statistics and to suggest references for a better understanding, this second article addresses the topic of hypothesis testing. The hypothesis testing method is discussed from a classical conception of statistical inference, including effect size, type of errors, p-value and power. The concepts are explained in plain language for lay readers and several references are suggested for those curious about the topic. (AU)

The Mind's Theorist / La Mente como Constructora de Teorías / O Teórico da Mente

Abstract The highest-order function of the mind is as a theorist. The memory system accumulates information about the outside world. The mind's theorist must sort through the information to formulate a theory about that world. The basic component of the system for theory building is a process called trolling. When the conscious mind is not being bombarded by external stimuli, or during certain stages of sleep, the mind's theorist trolls through memory searching for traces that contain similar information. If several traces are identified, analysis may yield information that was not evident when each was examined individually; reification of this sort can add new information to memory. The trolling process, and its ability to form new memory traces in the absence of external stimulation, is key to understanding many psychological phenomena.

Resumen La función superior de la mente es la de la construcción teórica. El sistema de memoria acumula información acerca del mundo externo y la mente constructora de teorías debe revisar dicha información para formular una teoría sobre el mundo. El componente básico del sistema de construcción teórica es un proceso llamado "trolling", que implica una búsqueda cuidadosa y sistemática. Cuando la mente consciente no está siendo bombardeada por estímulos externos o durante ciertas etapas del sueño, la mente teórica escudriña en la memoria buscando trazas que contengan información similar. Cuando se identifican varias trazas, es posible que el análisis arroje información que no era evidente al examinar cada traza de manera individual. Así, este tipo de reificación puede aportarle nueva información a la memoria. Dada su capacidad de formar nuevas trazas de memoria en ausencia de estímulos externos, el proceso de "trolling" es clave para la comprensión de muchos fenómenos psicológicos.

Resumo A maior função superior do cérebro é a de ser um teórico. O sistema da memória acumula informação sobre o mundo de fora. O teórico da mente deve investigar toda a informação para formular uma teoria sobre esse mundo. O componente básico do sistema para elaborar teorias é um processo chamado "trolling" (pesca de corrico). Quando a mente consciente não está sendo bombardeada por estímulos externos, ou durante certas etapas do sono, o teórico da mente "pesca" pelas memórias, procurando traços que contenham informações parecidas. Se vários traços são identificados, pode-se ter como resultado informação que não era evidente quando cada um foi analisado individualmente; reificação desse tipo pode adicionar nova informação à memória. O processo de "pescar" e sua habilidade para formar novos traços de memória na ausência de estímulos externos é fundamental para entender muitos fenômenos psicológicos.

Evaluación de una hipótesis general en investigación empírica utilizando el índice Dm / Testing a general hypothesis in empirical research using Dm-index

El índice Dm tiene aplicaciones en psicometría, específicamente como una medida de validez de constructo de los ítems. En el presente trabajo se expone una aplicación alternativa del índice Dm para la comprobación de hipótesis generales en investigación empírica cuando esta involucra un constructo general y no es posible obtener una puntuación total para realizar dicho contraste. Para ello, es considerada la información de la comprobación de cada hipótesis específica, y sistematizada en Dm. Se ofrecen ejemplos para el caso de correlaciones y comparación de grupos. Entonces, la aplicación de Dm en el contexto de la comprobación de hipótesis generales es promisoria.

The Dm-index have many applications in psychometrics, specifically as a item´s construct validity measure. In this paper, we expose an alternative application of index-Dm to testing general hypothesis in empirical research when it includes a general construct, and isn´t possible to have a global score to make this testing. For this purpose, is considered the information of every specific hypothesis testing, and systematize it on Dm. This work offers examples for the correlation and group comparison procedures, obtaining sounds results. Then, the application of Dm in context of general hypothesis testing is promissory

Desempeño de cuatro métodos estadísticos para evaluación de la concordancia prueba-reprueba de variables continuas en una muestra / Performance of four statistical methods for the assessment of test re-test reliability of continuous variables in a sample

Objetivo: Comparar el desempeño de cuatro pruebas estadísticas para la evaluación de la confiabilidad prueba/re-prueba de variables continuas. Métodos: estudio de simulación estadística desarrollado dentro en el marco de un estudio de pruebas diagnósticas in vitro en 120 dientes que cumplieron con los criterios de selección. Para cada diente posicionado en un dispositivo de estandarización se tomaron dos radiografías digitales (T0 y T1) a las cuales se evaluó la longitud dental. Los datos se analizaron con estadística descriptiva y luego la comparación estadística a través de "t" de Student pareada, coeficiente de correlación intraclase, coeficiente de correlación de Pearson y coeficiente de correlación y concordancia de Lin en el paquete Stat v.13.2 para Windows (StataCorp., TX., USA). Resultados: La media de longitud dental para T0 fue 21,15 mm y para T1 21,07 mm. La prueba "t" de Student reveló una diferencia de medias de 0,089 (P=0,00). El coeficiente de correlación intraclase fue 0,877 (IC 95%: 0,43 - 0,98), coeficiente de correlación de Pearson 0,93 y el coeficiente de correlación y concordancia de Lin 0,93 (IC 95%: 0,908 - 0,956). Conclusiones: La selección de una prueba estadística para evaluación de concordancia prueba/re-prueba debe hacerse teniendo en cuenta los objetivos del estudio en cada contexto y la posibilidad de cada método estadístico de valorar la presencia de error en los datos. Así, un método que actualmente cumple con este requerimiento esencial es el coeficiente de correlación y concordancia de Lin por lo cual se recomienda su uso en futuros estudios.

Objective: To compare the performance of four statistical tests in continuous variables test/retest reliability assessment. Methods: Statistical simulation study developed in the framework of an in vitro diagnostic test study including 120 teeth which met the inclusion criteria. Each tooth was positioned in a standardization device and was taken two digital x-rays (T0 and T1) in which we assessed tooth-length. Data were analyzed with descriptive statistics and then a statistical comparison was done with paired Student's "t" test, intraclass correlation coefficient, Pearson correlation coefficient and Lin's concordance correlation coefficient in Stata v.13.2 for Windows (StataCorp., TX., USA). Results: The average dental length for T0 was 21.15 mm and for T1 21.07 mm. Student's "t" test revealed an average difference of 0.089 (P=0.00). The intraclass correlation coefficient 0.877 (95% CI: 0.43 - 0.98), Pearson's productmoment correlation coefficient 0.93, and Lin's concordance correlation coefficient 0.93 (95% CI: 0.908 - 0.956). Conclusions: Selection of a statistical test for test/re-test reliability assessment should be made having in mind the research objectives in any context and the possibility of each method for error assessment. Thus, a method that currently complies with this essential requirement is Lin's concordance correlation coefficient, which is recommended for future test re-test research studies.

Test of association: which one is the most appropriate for my study?

AbstractBACKGROUND:Hypothesis tests are statistical tools widely used for assessing whether or not there is an association between two or more variables. These tests provide a probability of the type 1 error (p-value), which is used to accept or reject the null study hypothesis.OBJECTIVE:To provide a practical guide to help researchers carefully select the most appropriate procedure to answer the research question. We discuss the logic of hypothesis testing and present the prerequisites of each procedure based on practical examples.

Tópicos para uma boa análise de dados de estudos empíricos / Topic for a good analysis of empirical studies' data

Este artigo discute o processo de análise de dados em sete tópicos: a) modelos estatísticos, técnicas estatísticas e diferenciação de "receita estatística"; b) a associação da validade das conclusões estatísticas ao processo de delineamento e amostragem; c) a diferença e complementaridade da descrição de dados e do teste de hipóteses; d) como as hipóteses se relacionam com os testes estatísticos a realizar; e) a análise preliminar de dados e sua relevância; f) teste de hipótese na perspectiva de Fisher e de Neyman e Pearson; g) a apresentação dos dados como um processo comunicacional, e vantagens do uso de normas (APA).

This paper discusses the data analysis process around seven critical topics: a) Distinguishing between a statistical model, a statistical technique and a statistical receipt; b) Sampling and Design's implications for statistical validity; c) Complementarities and distinctions between data description and hypothesis testing; d) How statistical hypothesis are framed by the statistical model and vice versa; e) relevance of preliminary analysis of data; f) Fisher's and Neyman-Pearson'sperspectives of statistical inference; g) Data presentation as a communication matter.

Interpreting “statistical hypothesis testing” results in clinical research.

Difference between “Clinical Signifi cance and Statistical Signifi cance” should be kept in mind while interpreting “statistical hypothesis testing” results in clinical research. This fact is already known to many but again pointed out here as philosophy of “statistical hypothesis testing” is sometimes unnecessarily criticized mainly due to failure in considering such distinction. Randomized controlled trials are also wrongly criticized similarly. Some scientifi c method may not be applicable in some peculiar/particular situation does not mean that the method is useless. Also remember that “statistical hypothesis testing” is not for decision making and the fi eld of “decision analysis” is very much an integral part of science of statistics. It is not correct to say that “confi dence intervals have nothing to do with confi dence” unless one understands meaning of the word “confi dence” as used in context of confi dence interval. Interpretation of the results of every study should always consider all possible alternative explanations like chance, bias, and confounding. Statistical tests in inferential statistics are, in general, designed to answer the question “How likely is the difference found in random sample(s) is due to chance” and therefore limitation of relying only on statistical signifi cance in making clinical decisions should be avoided.

Uso inadecuado de la prueba de Ji al cuadrado y la omisión del control del sesgo de confusión / Improper use of the chi-square test and the omission of the confounding bias control

se presentan algunos elementos básicos sobre el uso y abuso de las pruebas de Ji al cuadrado de independencia y de homogeneidad en los informes finales de tesis de grado; así como la importancia del control del sesgo de confusión, la necesidad de tenerlo en cuenta en las investigaciones analíticas y algunos métodos para lograrlo.

Some basic elements on the use and misuse of independence and homogeneity chi-square tests in the final reports of theses are shown, as well as the importance of the confounding bias control, the need to take this into account in the analytic research and some methods to achieve it.

A critique of statistical hypothesis testing in clinical research.

Many have documented the difficulty of using the current paradigm of Randomized Controlled Trials (RCTs) to test and validate the effectiveness of alternative medical systems such as Ayurveda. This paper critiques the applicability of RCTs for all clinical knowledge-seeking endeavors, of which Ayurveda research is a part. This is done by examining statistical hypothesis testing, the underlying foundation of RCTs, from a practical and philosophical perspective. In the philosophical critique, the two main worldviews of probability are that of the Bayesian and the frequentist. The frequentist worldview is a special case of the Bayesian worldview requiring the unrealistic assumptions of knowing nothing about the universe and believing that all observations are unrelated to each other. Many have claimed that the first belief is necessary for science, and this claim is debunked by comparing variations in learning with different prior beliefs. Moving beyond the Bayesian and frequentist worldviews, the notion of hypothesis testing itself is challenged on the grounds that a hypothesis is an unclear distinction, and assigning a probability on an unclear distinction is an exercise that does not lead to clarity of action. This critique is of the theory itself and not any particular application of statistical hypothesis testing. A decisionmaking frame is proposed as a way of both addressing this critique and transcending ideological debates on probability. An example of a Bayesian decision-making approach is shown as an alternative to statistical hypothesis testing, utilizing data from a past clinical trial that studied the effect of Aspirin on heart attacks in a sample population of doctors. As a big reason for the prevalence of RCTs in academia is legislation requiring it, the ethics of legislating the use of statistical methods for clinical research is also examined.

O papel da plausibilidade na avaliação da pesquisa científica / The role of plausibility in the evaluation of scientific research / El papel de la plausibilidad en la evaluación de la investigación científica

O artigo discute o impacto da plausibilidade (probabilidade a priori) no resultado de pesquisas científicas, conforme abordagem de Ioannidis, referente ao percentual de hipóteses nulas erroneamente classificadas como "positivas" (estatisticamente significante). A questão "qual fração de resultados positivos é verdadeiramente positiva?", equivalente ao valor preditivo positivo, depende da combinação de hipóteses falsas e positivas em determinada área. Por exemplo, sejam 90 por cento das hipóteses falsas e α = 0,05, poder = 0,8: para cada 1.000 hipóteses, 45 (900 x 0,05) serão falso-positivos e 80 (100 x 0,8) verdadeiro-positivos. Assim, a probabilidade de que um resultado positivo seja um falso-positivo é de 45/125. Adicionalmente, o relato de estudos negativos como se fossem positivos contribuiria para a inflação desses valores. Embora essa análise seja de difícil quantificação e provavelmente superestimada, ela tem duas implicações: i) a plausibilidade deve ser considerada na análise da conformidade ética de uma pesquisa e ii) mecanismos de registro de estudo e protocolo devem ser estimulados.

The paper discusses the impact of plausibility (the a priori probability) on the results of scientific research, according to the approach proposed by Ioannidis, concerning the percentage of null hypotheses erroneously classified as "positive" (statistically significant). The question "what fraction of positive results are true-positives?", which is equivalent to the positive predictive value, is dependent on the combination of true and false hypotheses within a given area. For example, consider an area in which 90 percent of hypotheses are false and α = 0.05 and power = 0.8: for every 1,000 hypotheses, 45 (900 x 0.05) are false-positives and 80 (100 x 0.8) are true-positives. Therefore, the probability of a positive result being a false-positive is 45/125. In addition, the reporting of negative results as if they were positive would contribute towards an increase in this fraction. Although this analysis is difficult to quantify, and these results are likely be overestimated, it has two implications: i) plausibility should be considered in the analysis of the ethical adequacy of a research proposal, and ii) mechanisms aimed at registering studies and protocols should be encouraged.

El artículo discute el impacto de la plausibilidad (probabilidad a priori) en el resultado de investigaciones científicas, conforme abordaje de Ioannidis, relacionado con el porcentaje de hipótesis nulas erróneamente clasificadas como "positivas" (estadísticamente significativas). La interrogante "cuál fracción de resultados positivos es verdaderamente positiva?", equivalente al valor predictivo positivo, depende de la combinación de hipótesis falsas y positivas en determinada área. Por ejemplo, sea el 90 por ciento de las hipótesis falsas y α= 0,05, poder= 0,8: para cada 1000 hipótesis, 45 (900 x 0,05) serán falsos positivos, y 80 (100 x 0,8) verdaderos positivos. Así, la probabilidad de que un resultado sea un falso positivo es de 45/125. Adicionalmente, el relato de estudios negativos como si fueran positivos contribuiría a la inflación de esos valores. A pesar de que el análisis sea de difícil cuantificación y probablemente super-estimado, el mismo tiene dos implicaciones: i) la plausibilidad debe ser considerada en el análisis de la conformidad ética de una investigación y ii) mecanismos de registro de estudio y protocolo deben ser estimulados.