ABSTRACT
Data analysis workflows in many scientific domains have become increasingly complex and flexible. Here we assess the effect of this flexibility on the results of functional magnetic resonance imaging by asking 70 independent teams to analyse the same dataset, testing the same 9 ex-ante hypotheses1. The flexibility of analytical approaches is exemplified by the fact that no two teams chose identical workflows to analyse the data. This flexibility resulted in sizeable variation in the results of hypothesis tests, even for teams whose statistical maps were highly correlated at intermediate stages of the analysis pipeline. Variation in reported results was related to several aspects of analysis methodology. Notably, a meta-analytical approach that aggregated information across teams yielded a significant consensus in activated regions. Furthermore, prediction markets of researchers in the field revealed an overestimation of the likelihood of significant findings, even by researchers with direct knowledge of the dataset2-5. Our findings show that analytical flexibility can have substantial effects on scientific conclusions, and identify factors that may be related to variability in the analysis of functional magnetic resonance imaging. The results emphasize the importance of validating and sharing complex analysis workflows, and demonstrate the need for performing and reporting multiple analyses of the same data. Potential approaches that could be used to mitigate issues related to analytical variability are discussed.
Subject(s)
Data Analysis , Data Science/methods , Data Science/standards , Datasets as Topic , Functional Neuroimaging , Magnetic Resonance Imaging , Research Personnel/organization & administration , Brain/diagnostic imaging , Brain/physiology , Datasets as Topic/statistics & numerical data , Female , Humans , Logistic Models , Male , Meta-Analysis as Topic , Models, Neurological , Reproducibility of Results , Research Personnel/standards , SoftwareABSTRACT
Non-arbitrary sound-shape correspondences (SSC), such as the "bouba-kiki" effect, have been consistently observed across languages and together with other sound-symbolic phenomena challenge the classic linguistic dictum of the arbitrariness of the sign. Yet, it is unclear what makes a sound "round" or "spiky" to the human mind. Here we tested the hypothesis that visual experience is necessary for the emergence of SSC, supported by empirical evidence showing reduced SSC in visually impaired people. Results of two experiments comparing early blind and sighted individuals showed that SSC emerged strongly in both groups. Experiment 2, however, showed a partially different pattern of SSC in sighted and blind, that was mostly explained by a different effect of orthographic letter shape: The shape of written letters (spontaneously activated by spoken words) influenced SSC in the sighted, but not in the blind, who are exposed to an orthography (Braille) in which letters do not have spiky or round outlines. In sum, early blindness does not prevent the emergence of SSC, and differences between sighted and visually impaired people may be due the indirect influence (or lack thereof) of orthographic letter shape.
Subject(s)
Pattern Recognition, Visual/physiology , Psycholinguistics , Reading , Speech Perception/physiology , Touch Perception/physiology , Vision Disorders/physiopathology , Adult , Female , Humans , Male , Middle AgedABSTRACT
Across cultures and languages, people find similarities between the products of different senses in mysterious ways. By studying what is called cross-modal correspondences, cognitive psychologists discovered that lemons are fast rather than slow, boulders are sour, and red is heavier than yellow. Are these cross-modal correspondences established via sensory perception or can they be learned merely through language? We contribute to this debate by demonstrating that early blind people who lack the perceptual experience of color also think that red is heavier than yellow but to a lesser extent than sighted do.
