Garden of forking paths in ERP research - Effects of varying pre-processing and analysis steps in an N400 experiment.

This study tackles the Garden of Forking Paths, as a challenge for replicability and reproducibility of ERP studies. Here, we applied a multiverse analysis to a sample ERP N400 dataset, donated by an independent research team. We analyzed this dataset using 14 pipelines selected to showcase the full range of methodological variability found in the N400 literature using systematic review approach. The selected pipelines were compared in depth by looking into statistical test outcomes, descriptive statistics, effect size, data quality, and statistical power. In this way we provide a worked example of how analytic flexibility can impact results in research fields with high dimensionality such as ERP, when analyzed using standard null-hypothesis significance testing. Out of the methodological decisions that were varied, high-pass filter cut-off, artifact removal method, baseline duration, reference, measurement latency and locations, and amplitude measure (peak vs. mean) were all shown to affect at least some of the study outcome measures. Low-pass filtering was the only step which did not notably influence any of these measures. This study shows that even some of the seemingly minor procedural deviations can influence the conclusions of an ERP study. We demonstrate the power of multiverse analysis in both identifying the most reliable effects in a given study, and for providing insights into consequences of methodological decisions.

Contrastive Alveolar/Retroflex Phonemes in Singapore Mandarin Bilinguals: Comprehension Rates for Articulations in Different Accents, and Acoustic Analysis of Productions.

The standard Beijing variety of Mandarin has a clear alveolar-retroflex contrast for phonemes featuring voiceless sibilant frication (i.e., /s/, /ʂ/, /ʈs/, /ʈʂ/, /ʈsʰ/, /ʈʂʰ/). However, some studies show that varieties in the 'outer circle', such in Taiwan, have a reduced contrast for these speech sounds via a process known as 'deretroflexion'. The variety of Mandarin spoken in Singapore is also considered as 'outer circle', as it exhibits influences from Min Nan varieties. We investigated how bilinguals of Singapore Mandarin and English perceive and produce speech tokens in minimal pairs differing only in the alveolar/retroflex place of articulation. In all, 50 participants took part in two tasks. In Task 1, participants performed a lexical identification task for minimal pairs differing only the alveolar/retroflex place of articulation, as spoken by native speakers of two varieties: Beijing Mandarin and Singapore Mandarin. No difference in comprehension of the words was observed between the two varieties indicating that both varieties contain sufficient acoustic information for discrimination. In Task 2, participants read aloud from the list of minimal pairs while their voices were recorded. Acoustic analysis revealed that the phonemes do indeed differ acoustically in terms of center of gravity of the frication and in an alternative measure: long-term averaged spectra. The magnitude of this difference appears to be smaller than previously reported differences for the Beijing variety. These findings show that although some deretroflexion is evident in the speech of bilinguals of the Singaporean variety of Mandarin, it does not translate to ambiguity in the speech signal.

Is retroflexion a stable cue for distributional learning for speech sounds across languages? Learning for some bilingual adults, but not generalisable to a wider population in a well powered pre-registered study.

Bilinguals are widely reported to have certain kinds of cognitive advantages, including language learning advantages. One possible pathway is a language-specific transfer effect, whereby sensitivity to structural regularities in known languages can be brought to to-be-acquired languages that share particular features. Here we tested for transfer of a specific linguistic property, sensitivity to retroflexion as contrastive phonemic feature. We designed a task for bilinguals with homogeneous language exposure (i.e., bilingual in the same languages) and heterogeneous feature representation (i.e., differing levels of proficiency). Hindi and Mandarin Chinese both have retroflexion in phoneme contrasts (Hindi: stop consonants, Mandarin: sibilants). In a preregistered study, we conducted a statistical learning task for the Hindi dental-retroflex stop contrast with a group of early parallel English-Mandarin bilinguals, who varied in their Mandarin understanding levels. We based the target sample size on power analysis of a pilot study with a Bayesian stop-rule after minimum threshold. Contrary to the pilot study (N = 15), the main study (N = 50) did not find evidence for a learning effect, nor language-experience variance within the group. This finding suggests that statistical effects for the feature in question may be more fragile than commonly assumed, and may be evident in only a small subsample of the general population (as in our pilot). These stimuli have previously shown learning effects in children, so an additional possibility is that neural commitment to adults' languages prevents learning of the fine-grained stimulus contrast in question for this adult population.

Implicit Association Test (IAT) Studies Investigating Pitch-Shape Audiovisual Cross-modal Associations Across Language Groups.

Previous studies have shown that Chinese speakers and non-Chinese speakers exhibit different patterns of cross-modal congruence for the lexical tones of Mandarin Chinese, depending on which features of the pitch they attend to. But is this pattern of language-specific listening a conscious cultural strategy or an automatic processing effect? If automatic, does it also apply when the same pitch contours no longer sound like speech? Implicit Association Tests (IATs) provide an indirect measure of cross-modal association. In a series of IAT studies, conducted with participants with three kinds of language backgrounds (Chinese-dominant bilinguals, Chinese balanced bilinguals, and English speakers with no Chinese experience) we find language-specific congruence effects for Mandarin lexical tones but not for matched sine-wave stimuli. That is, for linguistic stimuli, non-Chinese speakers show advantages for pitch-height congruence (high-pointy, low-curvy); no congruence effects were found for Chinese speakers. For non-linguistic stimuli, all participant groups showed advantages for pitch-height congruence. The present findings suggest that non-lexical tone congruence (high-pointy, low-curvy) is a basic congruence pattern, and the acquisition of a language with lexical tone can alter this perception.

Japanese Sound-Symbolic Words for Representing the Hardness of an Object Are Judged Similarly by Japanese and English Speakers.

Contrary to the assumption of arbitrariness in modern linguistics, sound symbolism, which is the non-arbitrary relationship between sounds and meanings, exists. Sound symbolism, including the "Bouba-Kiki" effect, implies the universality of such relationships; individuals from different cultural and linguistic backgrounds can similarly relate sound-symbolic words to referents, although the extent of these similarities remains to be fully understood. Here, we examined if subjects from different countries could similarly infer the surface texture properties from words that sound-symbolically represent hardness in Japanese. We prepared Japanese sound-symbolic words of which novelty was manipulated by a genetic algorithm (GA). Japanese speakers in Japan and English speakers in both Singapore and the United States rated these words based on surface texture properties (hardness, warmness, and roughness), as well as familiarity. The results show that hardness-related words were rated as harder and rougher than softness-related words, regardless of novelty and countries. Multivariate analyses of the ratings classified the hardness-related words along the hardness-softness dimension at over 80% accuracy, regardless of country. Multiple regression analyses revealed that the number of speech sounds /g/ and /k/ predicted the ratings of the surface texture properties in non-Japanese countries, suggesting a systematic relationship between phonetic features of a word and perceptual quality represented by the word across culturally and linguistically diverse samples.

Early linguistic experience shapes bilingual adults' hearing for phonemes in both languages.

English and Mandarin Chinese differ in the voice onset times (VOTs) of /b/ and /p/. Hence the way bilinguals perceive these sounds may show 'tuning' to the language-specific acoustic structure of a bilingual's languages (a discrete model), or a shared representation across languages (a unitary model). We investigated whether an individual's early childhood exposure influences their model of phoneme perception across languages, in a large sample of early English-Mandarin bilingual adults in Singapore (N = 66). As preregistered, we mapped identification functions on a /b/-/p/ VOT continuum in each language. Bilingual balance was estimated using principal components analysis and entered into GLMMs of phoneme boundary and slope. VOT boundaries were earlier for English than Mandarin, and bilingual balance predicted the slope of the transition between categories across both languages: Those who heard more English from an earlier age showed steeper category boundaries than those who heard more Mandarin, suggesting early bilinguals may transfer their model for how phonemes differ from their earlier/stronger languages to later/weaker languages. We describe the transfer model of discrete phoneme representations and its implications for use of the phoneme identification task in diverse populations.

Good scientific practice in EEG and MEG research: Progress and perspectives.

Good scientific practice (GSP) refers to both explicit and implicit rules, recommendations, and guidelines that help scientists to produce work that is of the highest quality at any given time, and to efficiently share that work with the community for further scrutiny or utilization. For experimental research using magneto- and electroencephalography (MEEG), GSP includes specific standards and guidelines for technical competence, which are periodically updated and adapted to new findings. However, GSP also needs to be regularly revisited in a broader light. At the LiveMEEG 2020 conference, a reflection on GSP was fostered that included explicitly documented guidelines and technical advances, but also emphasized intangible GSP: a general awareness of personal, organizational, and societal realities and how they can influence MEEG research. This article provides an extensive report on most of the LiveMEEG contributions and new literature, with the additional aim to synthesize ongoing cultural changes in GSP. It first covers GSP with respect to cognitive biases and logical fallacies, pre-registration as a tool to avoid those and other early pitfalls, and a number of resources to enable collaborative and reproducible research as a general approach to minimize misconceptions. Second, it covers GSP with respect to data acquisition, analysis, reporting, and sharing, including new tools and frameworks to support collaborative work. Finally, GSP is considered in light of ethical implications of MEEG research and the resulting responsibility that scientists have to engage with societal challenges. Considering among other things the benefits of peer review and open access at all stages, the need to coordinate larger international projects, the complexity of MEEG subject matter, and today's prioritization of fairness, privacy, and the environment, we find that current GSP tends to favor collective and cooperative work, for both scientific and for societal reasons.

How to do Better N400 Studies: Reproducibility, Consistency and Adherence to Research Standards in the Existing Literature.

Given the complexity of ERP recording and processing pipeline, the resulting variability of methodological options, and the potential for these decisions to influence study outcomes, it is important to understand how ERP studies are conducted in practice and to what extent researchers are transparent about their data collection and analysis procedures. The review gives an overview of methodology reporting in a sample of 132 ERP papers, published between January 1980 - June 2018 in journals included in two large databases: Web of Science and PubMed. Because ERP methodology partly depends on the study design, we focused on a well-established component (the N400) in the most commonly assessed population (healthy neurotypical adults), in one of its most common modalities (visual images). The review provides insights into 73 properties of study design, data pre-processing, measurement, statistics, visualization of results, and references to supplemental information across studies within the same subfield. For each of the examined methodological decisions, the degree of consistency, clarity of reporting and deviations from the guidelines for best practice were examined. Overall, the results show that each study had a unique approach to ERP data recording, processing and analysis, and that at least some details were missing from all papers. In the review, we highlight the most common reporting omissions and deviations from established recommendations, as well as areas in which there was the least consistency. Additionally, we provide guidance for a priori selection of the N400 measurement window and electrode locations based on the results of previous studies.

Creating a Corpus of Multilingual Parent-Child Speech Remotely: Lessons Learned in a Large-Scale Onscreen Picturebook Sharing Task.

With lockdowns and social distancing measures in place, research teams looking to collect naturalistic parent-child speech interactions have to develop alternatives to in-lab recordings and observational studies with long-stretch recordings. We designed a novel micro-longitudinal study, the Talk Together Study, which allowed us to create a rich corpus of parent-child speech interactions in a fully online environment (N participants = 142, N recordings = 410). In this paper, we discuss the methods we used, and the lessons learned during adapting and running the study. These lessons learned cover nine domains of research design, monitoring and feedback: Recruitment strategies, Surveys and Questionnaires, Video-call scheduling, Speech elicitation tools, Videocall protocols, Participant remuneration strategies, Project monitoring, Participant retention, and Data Quality, and may be used as a primer for teams planning to conduct remote studies in the future.

Towards ARTEM-IS: Design guidelines for evidence-based EEG methodology reporting tools.

As the number of EEG papers increases, so too do the number of guidelines for how to report what has been done. However, current guidelines and checklists appear to have limited adoption, as systematic reviews have shown the journal article format is highly prone to errors, ambiguities and omissions of methodological details. This is a problem for transparency in the scientific record, along with reproducibility and metascience. Following lessons learned in the high complexity fields of aviation and surgery, we conclude that new tools are needed to overcome the limitations of written methodology descriptions, and that these tools should be developed through community consultation to ensure that they have the most utility for EEG stakeholders. As a first step in tool development, we present the ARTEM-IS Statement describing what action will be needed to create an Agreed Reporting Template for Electroencephalography Methodology - International Standard (ARTEM-IS), along with ARTEM-IS Design Guidelines for developing tools that use an evidence-based approach to error reduction. We first launched the statement at the LiveMEEG conference in 2020 along with a draft of an ARTEM-IS template for public consultation. Members of the EEG community are invited to join this collective effort to create evidence-based tools that will help make the process of reporting methodology intuitive to complete and foolproof by design.

Cross-Modal Perception of Noise-in-Music: Audiences Generate Spiky Shapes in Response to Auditory Roughness in a Novel Electroacoustic Concert Setting.

Noise has become integral to electroacoustic music aesthetics. In this paper, we define noise as sound that is high in auditory roughness, and examine its effect on cross-modal mapping between sound and visual shape in participants. In order to preserve the ecological validity of contemporary music aesthetics, we developed Rama, a novel interface, for presenting experimentally controlled blocks of electronically generated sounds that varied systematically in roughness, and actively collected data from audience interaction. These sounds were then embedded as musical drones within the overall sound design of a multimedia performance with live musicians, Audience members listened to these sounds, and collectively voted to create the shape of a visual graphic, presented as part of the audio-visual performance. The results of the concert setting were replicated in a controlled laboratory environment to corroborate the findings. Results show a consistent effect of auditory roughness on shape design, with rougher sounds corresponding to spikier shapes. We discuss the implications, as well as evaluate the audience interface.

Is a High Tone Pointy? Speakers of Different Languages Match Mandarin Chinese Tones to Visual Shapes Differently.

Studies investigating cross-modal correspondences between auditory pitch and visual shapes have shown children and adults consistently match high pitch to pointy shapes and low pitch to curvy shapes, yet no studies have investigated linguistic-uses of pitch. In the present study, we used a bouba/kiki style task to investigate the sound/shape mappings for Tones of Mandarin Chinese, for three groups of participants with different language backgrounds. We recorded the vowels [i] and [u] articulated in each of the four tones of Mandarin Chinese. In Study 1 a single auditory stimulus was presented with two images (one curvy, one spiky). In Study 2 a single image was presented with two auditory stimuli differing only in tone. Participants were asked to select the best match in an online 'Quiz.' Across both studies, we replicated the previously observed 'u-curvy, i-pointy' sound/shape cross-modal correspondence in all groups. However, Tones were mapped differently by people with different language backgrounds: speakers of Mandarin Chinese classified as Chinese-dominant systematically matched Tone 1 (high, steady) to the curvy shape and Tone 4 (falling) to the pointy shape, while English speakers with no knowledge of Chinese preferred to match Tone 1 (high, steady) to the pointy shape and Tone 3 (low, dipping) to the curvy shape. These effects were observed most clearly in Study 2 where tone-pairs were contrasted explicitly. These findings are in line with the dominant patterns of linguistic pitch perception for speakers of these languages (pitch-change, and pitch height, respectively). Chinese English balanced bilinguals showed a bivalent pattern, swapping between the Chinese pitch-change pattern and the English pitch-height pattern depending on the task. These findings show for that the supposedly universal pattern of mapping linguistic sounds to shape is modulated by the sensory properties of a speaker's language system, and that people with high functioning in more than one language can dynamically shift between patterns.

Glyph guessing for 'oo' and 'ee': spatial frequency information in sound symbolic matching for ancient and unfamiliar scripts.

In three experiments, we asked whether diverse scripts contain interpretable information about the speech sounds they represent. When presented with a pair of unfamiliar letters, adult readers correctly guess which is /i/ (the 'ee' sound in 'feet'), and which is /u/ (the 'oo' sound in 'shoe') at rates higher than expected by chance, as shown in a large sample of Singaporean university students (Experiment 1) and replicated in a larger sample of international Internet users (Experiment 2). To uncover what properties of the letters contribute to different scripts' 'guessability,' we analysed the visual spatial frequencies in each letter (Experiment 3). We predicted that the lower spectral frequencies in the formants of the vowel /u/ would pattern with lower spatial frequencies in the corresponding letters. Instead, we found that across all spatial frequencies, the letter with more black/white cycles (i.e. more ink) was more likely to be guessed as /u/, and the larger the difference between the glyphs in a pair, the higher the script's guessability. We propose that diverse groups of humans across historical time and geographical space tend to employ similar iconic strategies for representing speech in visual form, and provide norms for letter pairs from 56 diverse scripts.

When Does Maluma/Takete Fail? Two Key Failures and a Meta-Analysis Suggest That Phonology and Phonotactics Matter.

Eighty-seven years ago, Köhler reported that the majority of students picked the same answer in a quiz: Which novel word form ('maluma' or 'takete') went best with which abstract line drawing (one curved, one angular). Others have consistently shown the effect in a variety of contexts, with only one reported failure by Rogers and Ross. In the spirit of transparency, we report our own failure in the same journal. In our study, speakers of Syuba, from the Himalaya in Nepal, do not show a preference when matching word forms 'kiki' and 'bubu' to spiky versus curvy shapes. We conducted a meta-analysis of previous studies to investigate the relationship between pseudoword legality and task effects. Our combined analyses suggest a common source for both of the failures: 'wordiness' - We believe these tests fail when the test words do not behave according to the sound structure of the target language.

To call a cloud 'cirrus': sound symbolism in names for categories or items.

The aim of the present paper is to experimentally test whether sound symbolism has selective effects on labels with different ranges-of-reference within a simple noun-hierarchy. In two experiments, adult participants learned the make up of two categories of unfamiliar objects ('alien life forms'), and were passively exposed to either category-labels or item-labels, in a learning-by-guessing categorization task. Following category training, participants were tested on their visual discrimination of object pairs. For different groups of participants, the labels were either congruent or incongruent with the objects. In Experiment 1, when trained on items with individual labels, participants were worse (made more errors) at detecting visual object mismatches when trained labels were incongruent. In Experiment 2, when participants were trained on items in labelled categories, participants were faster at detecting a match if the trained labels were congruent, and faster at detecting a mismatch if the trained labels were incongruent. This pattern of results suggests that sound symbolism in category labels facilitates later similarity judgments when congruent, and discrimination when incongruent, whereas for item labels incongruence generates error in judgements of visual object differences. These findings reveal that sound symbolic congruence has a different outcome at different levels of labelling within a noun hierarchy. These effects emerged in the absence of the label itself, indicating subtle but pervasive effects on visual object processing.

Can a Word Sound Like a Shape Before You Have Seen It? Sound-Shape Mapping Prior to Conscious Awareness.

Nonarbitrary mappings between sound and shape (i.e., the bouba-kiki effect) have been shown across different cultures and early in development; however, the level of processing at which this effect arises remains unclear. Here we show that the mapping occurs prior to conscious awareness of the visual stimuli. Under continuous flash suppression, congruent stimuli (e.g., "kiki" inside an angular shape) broke through to conscious awareness faster than incongruent stimuli. This was true even when we trained people to pair unfamiliar letters with auditory word forms, a result showing that the effect was driven by the phonology, not the visual features, of the letters. Furthermore, visibility thresholds of the shapes decreased when they were preceded by a congruent auditory word form in a masking paradigm. Taken together, our results suggest that sound-shape mapping can occur automatically prior to conscious awareness of visual shapes, and that sensory congruence facilitates conscious awareness of a stimulus being present.

Infant VEPs reveal neural correlates of implicit naming: Lateralized differences between lexicalized versus name-unknown pictures.

Recent behavioural studies with toddlers have demonstrated that simply viewing a picture in silence triggers a cascade of linguistic processing which activates a representation of the picture's name (Mani and Plunkett, 2010, 2011). Electrophysiological studies have also shown that viewing a picture modulates the auditory evoked potentials (AEPs) triggered by later speech, from early in the second year of life (Duta et al., 2012; Friedrich and Friederici, 2005; Mani et al., 2011) further supporting the notion that picture viewing gives rise to a representation of the picture's name against which later speech can be matched. However, little is known about how and when the implicit name arises during picture viewing, or about the electrophysiological activity which supports this linguistic process. We report differences in the visual evoked potentials (VEPs) of fourteen-month-old infants who saw photographs of animals and objects, some of which were name-known (lexicalized), while waiting for an auditory label to be presented. During silent picture viewing, lateralized neural activity was selectively triggered by lexicalized items, as compared to nameless items. Lexicalized items generated a short-lasting negative-going deflection over frontal, left centro-temporal, and left occipital regions shortly after the picture appeared (126-225 ms). A positive deflection was also observed over the right hemisphere (particularly centro-temporal regions) in a later, longer-lasting window (421-720 ms). The lateralization of these differences in the VEP suggests the possible involvement of linguistic processes during picture viewing, and may reflect activity involved in the implicit activation of the picture's name.

Balloons and bavoons versus spikes and shikes: ERPs reveal shared neural processes for shape-sound-meaning congruence in words, and shape-sound congruence in pseudowords.

There is something about the sound of a pseudoword like takete that goes better with a spiky, than a curvy shape (Köhler, 1929:1947). Yet despite decades of research into sound symbolism, the role of this effect on real words in the lexicons of natural languages remains controversial. We report one behavioural and one ERP study investigating whether sound symbolism is active during normal language processing for real words in a speaker's native language, in the same way as for novel word forms. The results indicate that sound-symbolic congruence has a number of influences on natural language processing: Written forms presented in a congruent visual context generate more errors during lexical access, as well as a chain of differences in the ERP. These effects have a very early onset (40-80 ms, 100-160 ms, 280-320 ms) and are later overshadowed by familiar types of semantic processing, indicating that sound symbolism represents an early sensory-co-activation effect.

In the absence of animacy: superordinate category structure affects subordinate label verification.

Theoretical accounts as well as behavioral studies reporting animacy effects offer inconsistent and sometimes contradictory results. A possible explanation for these inconsistencies may be inadvertent biases in the stimuli selected for test - with category-specific effects driven by characteristics of test stimuli other than animacy per se. In this study, we pit animacy against feature structure (intra-item variability), in a picture-word matching task. For unimpaired adults, regardless of whether objects were from animate (mammals; insects) or inanimate (clothes; musical instruments) superordinate categories, participants were faster to match basic level labels with objects from categories with low intra-item variability (mammals; clothes) than from categories with high intra-item variability (insects; instruments). Thus, pitting animacy against variability allowed us to clarify that observable differences in processing speed between animals and instruments are systematically driven by the intra-item variability of the superordinate categories, and not by animacy itself.