Small numbers are an opportunity, not a problem / Números pequenos são uma oportunidade, não um problema

Aims: outcomes of research in education and training are partly a function of the context in which that study takes place, the questions we ask, and what is feasible. Many questions are about learning, which involves repeated measurements in a particular time window, and the practical context is usually such that offering an intervention to some but not to all learners does not make sense or is unethical. For quality assurance and other purposes, education and training centers may have very locally oriented questions that they seek to answer, such as whether an intervention can be considered effective in their context of small numbers of learners. While the rationale behind the design and outcomes of this kind of studies may be of interest to a much wider community, for example to study the transferability of findings to other contexts, people are often discouraged to report on the outcomes of such studies at conferences or in educational research journals. The aim of this paper is to counter that discouragement and instead encourage people to see small numbers as an opportunity instead of as a problem. Method: a worked example of a parametric and a non-parametric method for this type of situation, using simulated data in the zero-cost Open Source statistical program R version 4.0.5. Results: contrary to the non-parametric method, the parametric method can provide estimates of intervention effectiveness for the individual participant, account for trends in different phases of a study. However, the non-parametric method provides a solution in several situations where the parametric method should be used. Conclusion: Given the costs of research, the lessons to be learned from research, and statistical methods available, small numbers should be considered an opportunity, not a problem.

Objetivo: os resultados da pesquisa em educação e treinamento são, em parte, uma função do contexto em que esse estudo ocorre, das perguntas que fazemos e do que é viável. Muitas perguntas são sobre a aprendizagem, que envolve medições repetidas em uma janela de tempo específica, e o contexto prático, geralmente, é tal, que oferecer uma intervenção a alguns, mas não a todos os alunos, não faz sentido ou é antiético. Para garantia de qualidade e outros propósitos, os centros de educação e treinamento podem ter perguntas orientadas localmente que procuram responder, como, por exemplo, se uma intervenção pode ser considerada eficaz em seu contexto de pequeno número de alunos. Embora a justificativa por trás do projeto e dos resultados deste tipo de estudos possa ser do interesse de uma comunidade muito mais ampla, por exemplo, para estudar a possibilidade de transferência de resultados para outros contextos, as pessoas são frequentemente desencorajadas a relatar os resultados de tais estudos em conferências ou em revistas de pesquisa educacional. O objetivo deste artigo é combater esse desânimo e, em vez disso, incentivar as pessoas a verem os pequenos números como uma oportunidade em vez de um problema. Método: realizado um exemplo de método paramétrico e não paramétrico para este tipo de situação, utilizando dados simulados no programa estatístico Open Source R versão 4.0.5 de custo zero. Resultados: ao contrário do método não paramétrico, o método paramétrico pode fornecer estimativas da eficácia da intervenção para o participante individual, levando em conta as tendências em diferentes fases de um estudo. No entanto, o método não paramétrico fornece uma solução em várias situações, onde o método paramétrico deve ser usado. Conclusão: dados os custos da pesquisa, as lições a serem aprendidas com a pesquisa e os métodos estatísticos disponíveis, pequenos números devem ser considerados uma oportunidade, não um problema.

Statistics for N = 1: a non-parametric Bayesian approach / Estatística para N = 1: uma abordagem Bayesiana não paramétrica

Research in education is often associated with comparing group averages and linear relations in sufficiently large samples and evidence-based practice is about using the outcomes of that research in the practice of education. However, there are questions that are important for the practice of education that cannot really be addressed by comparisons of group averages and linear relations, no matter how large the samples. Besides, different types of constraints including logistic, financial, and ethical ones may make larger-sample research unfeasible or at least questionable. What has remained less known in many fields is that there are study designs and statistical methods for research involving small samples or even individuals that allow us to address questions of importance for the practice of education. This article discusses one type of such situations and provides a simple coherent statistical approach that provides point and interval estimates of differences of interest regardless of the type of the outcome variable and that is of use in other types of studies involving large samples, small samples, and single individuals.

A pesquisa em educação é frequentemente associada à comparação de médias de grupo e relações lineares em amostras suficientemente grandes, e a prática baseada em evidências trata do uso dos resultados dessa pesquisa na prática educacional. No entanto, há questões importantes para a prática da educação que não podem ser realmente abordadas por comparações de médias de grupo e relações lineares, por maiores que sejam as amostras. Além disso, diferentes tipos de restrições, incluindo as logísticas, financeiras e éticas, podem tornar a pesquisa com amostras maiores inviável ou, pelo menos, questionável. O que tem ficado menos conhecido em muitos campos é que existem desenhos de estudos e métodos estatísticos para pesquisas envolvendo pequenas amostras ou mesmo indivíduos que nos permitem abordar questões de importância para a prática da educação. Este artigo discute um tipo de tais situações e fornece uma abordagem estatística coerente simples que fornece estimativas de ponto e intervalo de diferenças de interesse, independentemente do tipo de variável de resultado e que é útil em outros tipos de estudos envolvendo grandes amostras, pequenas amostras, e indivíduos solteiros.

Diseño a doble ciego y el uso de placebos psicológicos en investigación de resultados en psicoterapia: Es posible? Un estudio piloto de viabilidad / Double blind design and the use of psychological placebos in outcomes research in psychotherapy: Is it possible? A pilot feasibility study

El uso de placebos y diseños a doble ciego ha cumplido un papel crucial en la investigación clínica en Medicina. Su aplicación a la investigación de resultados en psicoterapia ha sido controversial. Muchos autores niegan la posibilidad de su aplicación debido a que el terapeuta debería conocer la condición del procedimiento aplicado. Se presenta detalladamente un estudio en el que se utilizó un procedimiento placebo correspondiente a EMDR (Eye Movement Desensitization and Reprocessing- Desensibilización y Reprocesamiento por Movimientos Oculares) con un Diseño Experimental de Caso Único. Los objetivos fueron los siguientes: poner a prueba la viabilidad de la implementación de un placebo de tipo psicológico en EMDR y la aplicación de un diseño a doble ciego en el estudio de resultados en EMDR. Se seleccionaron tres pacientes que sufrían de Trastorno por Estrés Postraumático. Se estableció una línea de base de la sintomatología presentada. Luego, los sujetos fueron asignados aleatoriamente a tres condiciones experimentales durante tres sesiones: (a) aplicación del protocolo EMDR sin ningún tipo de estimulación con auriculares en silencio (placebo 1), (b) aplicación del protocolo EMDR con estimulación bilateral auditiva simultánea (no alternada) (placebo 2) y (c) aplicación del protocolo estándar de EMDR con estimulación auditiva bilateral alternada (tratamiento activo). Tales procedimientos resultaron igualmente creíbles para el paciente y para el terapeuta. Esto permitió el desarrollo de un diseño de investigación a doble ciego para la investigación de resultados en EMDR. Finalmente, se discuten algunas posibles aplicaciones e implicancias de la introducción del uso de placebos psicológicos y diseño a doble ciego en la investigación en psicoterapia.

Double blind design and placebos have been of crucial importance in medical clinical research. Their use in outcomes research in the field of psychotherapy has been controversial, though. Their feasibility in such case has been denied by many authors based on the assumption that the psychotherapist would need to know the nature of the applied procedure. In view of this, the author has conducted a pilot feasibility study on three subjects within the context of his doctoral dissertation. Said dissertation aims at establishing the role of alternating bilateral auditory stimulation in the processing of traumatic memories as used in the EMDR (Eye Movement Desensitization and Reprocessing) technique. To such end, the EMDR basic principles and procedures are introduced -with particular attention to alternating bilateral auditory stimulation- and a pilot study using placebos during EMDR administration is presented in detail. The goals of this study are testing the feasibility of: (a) using a psychological placebo in EMDR therapy, and (b) applying a double blind design study in EMDR outcomes research. A single case experimental design was performed on three different patients suffering from PTSD (Posttraumatic Stress Disorder). A symptomatology baseline was established through out three weekly sessions using the DTS (Davidson Trauma Scale) and the OQ-45.2 (Outcomes Questionnaire 45.2). First, three CDs were recorded -one with no sound at all (CD-1, silence condition); another one with auditory stimulation consisting of a tic-tac sound recorded in monaural condition, and reproduced simultaneously over both earphones at a rate of one beat per second (CD-2, monaural condition) and a third one with alternating bilateral auditory stimulation consisting of the exact same sound recorded in stereophonic condition, and reproduced alternatively over the left and right earphones (CD-3, stereo condition)-. At a second stage, these three experimental conditions were assigned randomly to the three subjects, who were administered: (a) EMDR protocol without any type of stimulation, with no sound coming out of the earphones using CD-1 (placebo 1), (b) EMDR protocol with simultaneous bilateral auditory stimulation using CD-2 (placebo 2), and (c) EMDR protocol with alternating bilateral auditory stimulation using CD-3 (active treatment). In all cases, the experimental conditions were implemented during three full sessions in which the CDs were reproduced for the subjects through earphones, instead of speakers, to ensure that the psychotherapist was unaware of the actual conditions. Subsequently, the standard EMDR protocol (i.e., with alternating bilateral auditory stimulation) was administered to each subject until the end of the treatment, determined either by the symptoms being resolved or the maximum of ten sessions being completed. As a result of this pilot study, the author concludes that the feasibility of using double blind studies and placebos in EMDR psychotherapy has indeed been established. Since no apparent difficulties in the administration of the placebos were detected during the study, the use of psychological placebos seems viable. Such procedure is equally credible for the patient as well as for the psychotherapist, which renders possible the development of a double blind design in EMDR outcomes research. It should be noted, though, that the credibility of the placebo was not formally assessed, but rather was perceived through the author’s informal observation. Developing assessment criteria and formal tools to evaluate the credibility of placebo procedures is advisable if future investigations on the subject are to be carried. Even though this study was conducted under a Single Case Experimental Design, the placebo procedure employed could be easily adapted for its use in between group’s designs. Finally, some of the possible applications and consequences regarding the introduction of placebos and double blind design in psychotherapy research are discussed.

Effects of Indirect Moxibustion on Common Cold Symptoms in Elderly Subjects Lived in Nursing Home: Single-case Experimental Design / 全日本鍼灸学会雑誌

[Objective] The effect of indirect moxibustion on cold symptoms was examined in elderly subjects living in a nursing home.<BR>[Design] A single-case experimental design that enables clinical trials in small a number of subjects was used in this study.<BR>[Methods] Two elderly subjects living in a nursing house participated for 16 weeks. The experimental periods were allocated at random as treatment periods and control periods of eight weeks each. During the treatment periods, subjects were treated with indirect moxibustion GV 14 and BL 12 (both sides) 3 units each, 3 times a week. Rating scores with 4 and/or 5 grades were used to evaluate common cold symptoms.<BR>[Results] Concerning the presence of common cold, there were no significant differences between the treatment and control periods. Moreover, concerning common cold symptoms, there were no significant differences between treatment and control periods.<BR>[Conclusion] Effects of an indirect moxibustion on the common cold symptoms were not detected in this study. We speculated that the living environment of these subjects and the intensity of stimulation were factors. Furthermore, we suspected that a single-case experimental design was unsuitable for common cold research.