Evaluating synthetic neuroimaging data augmentation for automatic brain tumour segmentation with a deep fully-convolutional network.

Gliomas observed in medical images require expert neuro-radiologist evaluation for treatment planning and monitoring, motivating development of intelligent systems capable of automating aspects of tumour evaluation. Deep learning models for automatic image segmentation rely on the amount and quality of training data. In this study we developed a neuroimaging synthesis technique to augment data for training fully-convolutional networks (U-nets) to perform automatic glioma segmentation. We used StyleGAN2-ada to simultaneously generate fluid-attenuated inversion recovery (FLAIR) magnetic resonance images and corresponding glioma segmentation masks. Synthetic data were successively added to real training data (n = 2751) in fourteen rounds of 1000 and used to train U-nets that were evaluated on held-out validation (n = 590) and test sets (n = 588). U-nets were trained with and without geometric augmentation (translation, zoom and shear), and Dice coefficients were computed to evaluate segmentation performance. We also monitored the number of training iterations before stopping, total training time, and time per iteration to evaluate computational costs associated with training each U-net. Synthetic data augmentation yielded marginal improvements in Dice coefficients (validation set +0.0409, test set +0.0355), whereas geometric augmentation improved generalization (standard deviation between training, validation and test set performances of 0.01 with, and 0.04 without geometric augmentation). Based on the modest performance gains for automatic glioma segmentation we find it hard to justify the computational expense of developing a synthetic image generation pipeline. Future work may seek to optimize the efficiency of synthetic data generation for augmentation of neuroimaging data.

Blind source separation of event-related potentials using a recurrent neural network.

Event-related potentials (ERPs) are a superposition of electric potential differences generated by neurophysiological activity associated with psychophysical events. Spatiotemporal dissociation of these signal sources can supplement conventional ERP analysis and improve source localization. However, results from established source separation methods applied to ERPs can be challenging to interpret. Hence, we have developed a recurrent neural network (RNN) method for blind source separation. The RNN transforms input step pulse signals representing events into corresponding ERP difference waveforms. Source waveforms are obtained from penultimate layer units and scalp maps are obtained from feed-forward output layer weights that project these source waveforms onto EEG electrode amplitudes. An interpretable, sparse source representation is achieved by incorporating L1 regularization of signals obtained from the penultimate layer of the network during training. This RNN method was applied to four ERP difference waveforms (MMN, N170, N400, P3) from the open-access ERP CORE database, and independent component analysis (ICA) was applied to the same data for comparison. The RNN decomposed these ERPs into eleven spatially and temporally separate sources that were less noisy, tended to be more ERP-specific, and were less similar to each other than ICA-derived sources. The RNN sources also had less ambiguity between source waveform amplitude, scalp potential polarity, and equivalent current dipole orientation than ICA sources. In conclusion, the proposed RNN blind source separation method can be effectively applied to grand-average ERP difference waves and holds promise for further development as a computational model of event-related neural signals.

Localized estimation of electromagnetic sources underlying event-related fields using recurrent neural networks.

Objective. To use a recurrent neural network (RNN) to reconstruct neural activity responsible for generating noninvasively measured electromagnetic signals.Approach. Output weights of an RNN were fixed as the lead field matrix from volumetric source space computed using the boundary element method with co-registered structural magnetic resonance images and magnetoencephalography (MEG). Initially, the network was trained to minimise mean-squared-error loss between its outputs and MEG signals, causing activations in the penultimate layer to converge towards putative neural source activations. Subsequently, L1 regularisation was applied to the final hidden layer, and the model was fine-tuned, causing it to favour more focused activations. Estimated source signals were then obtained from the outputs of the last hidden layer. We developed and validated this approach with simulations before applying it to real MEG data, comparing performance with beamformers, minimum-norm estimate, and mixed-norm estimate source reconstruction methods.Main results. The proposed RNN method had higher output signal-to-noise ratios and comparable correlation and error between estimated and simulated sources. Reconstructed MEG signals were also equal or superior to the other methods regarding their similarity to ground-truth. When applied to MEG data recorded during an auditory roving oddball experiment, source signals estimated with the RNN were generally biophysically plausible and consistent with expectations from the literature.Significance. This work builds on recent developments of RNNs for modelling event-related neural responses by incorporating biophysical constraints from the forward model, thus taking a significant step towards greater biological realism and introducing the possibility of exploring how input manipulations may influence localised neural activity.

Neural correlates of face perception modeled with a convolutional recurrent neural network.

Objective.Event-related potential (ERP) sensitivity to faces is predominantly characterized by an N170 peak that has greater amplitude and shorter latency when elicited by human faces than images of other objects. We aimed to develop a computational model of visual ERP generation to study this phenomenon which consisted of a three-dimensional convolutional neural network (CNN) connected to a recurrent neural network (RNN).Approach.The CNN provided image representation learning, complimenting sequence learning of the RNN for modeling visually-evoked potentials. We used open-access data from ERP Compendium of Open Resources and Experiments (40 subjects) to develop the model, generated synthetic images for simulating experiments with a generative adversarial network, then collected additional data (16 subjects) to validate predictions of these simulations. For modeling, visual stimuli presented during ERP experiments were represented as sequences of images (time x pixels). These were provided as inputs to the model. By filtering and pooling over spatial dimensions, the CNN transformed these inputs into sequences of vectors that were passed to the RNN. The ERP waveforms evoked by visual stimuli were provided to the RNN as labels for supervised learning. The whole model was trained end-to-end using data from the open-access dataset to reproduce ERP waveforms evoked by visual events.Main results.Cross-validation model outputs strongly correlated with open-access (r= 0.98) and validation study data (r= 0.78). Open-access and validation study data correlated similarly (r= 0.81). Some aspects of model behavior were consistent with neural recordings while others were not, suggesting promising albeit limited capacity for modeling the neurophysiology of face-sensitive ERP generation.Significance.The approach developed in this work is potentially of significant value for visual neuroscience research, where it may be adapted for multiple contexts to study computational relationships between visual stimuli and evoked neural activity.

A Guided Tutorial on Modelling Human Event-Related Potentials with Recurrent Neural Networks.

In cognitive neuroscience research, computational models of event-related potentials (ERP) can provide a means of developing explanatory hypotheses for the observed waveforms. However, researchers trained in cognitive neurosciences may face technical challenges in implementing these models. This paper provides a tutorial on developing recurrent neural network (RNN) models of ERP waveforms in order to facilitate broader use of computational models in ERP research. To exemplify the RNN model usage, the P3 component evoked by target and non-target visual events, measured at channel Pz, is examined. Input representations of experimental events and corresponding ERP labels are used to optimize the RNN in a supervised learning paradigm. Linking one input representation with multiple ERP waveform labels, then optimizing the RNN to minimize mean-squared-error loss, causes the RNN output to approximate the grand-average ERP waveform. Behavior of the RNN can then be evaluated as a model of the computational principles underlying ERP generation. Aside from fitting such a model, the current tutorial will also demonstrate how to classify hidden units of the RNN by their temporal responses and characterize them using principal component analysis. Statistical hypothesis testing can also be applied to these data. This paper focuses on presenting the modelling approach and subsequent analysis of model outputs in a how-to format, using publicly available data and shared code. While relatively less emphasis is placed on specific interpretations of P3 response generation, the results initiate some interesting discussion points.

Recurrent Neural Network Model of Human Event-related Potentials in Response to Intensity Oddball Stimulation.

The mismatch negativity (MMN) component of the human event-related potential (ERP) is frequently interpreted as a sensory prediction-error signal. However, there is ambiguity concerning the neurophysiology underlying hypothetical prediction and prediction-error signalling components, and whether these can be dissociated from overlapping obligatory components of the ERP that are sensitive to physical properties of sounds. In the present study, a hierarchical recurrent neural network (RNN) was fitted to ERP data from 38 subjects. After training the model to reproduce ERP waveforms evoked by 80 dB standard and 70 dB deviant stimuli, it was used to simulate a response to 90 dB deviant stimuli. Internal states of the RNN effectively combined to generate synthetic ERPs, where individual hidden units are loosely analogous to population-level sources. Model behaviour was characterised using principal component analysis of stimulus condition, layer, and individual unit responses. Hidden units were categorised according to their temporal response fields, and statistically significant differences among stimulus conditions were observed for amplitudes of units peaking in the 0-75 ms (P50), 75-125 ms (N1), and 250-400 ms (N3) latency ranges, surprisingly not including the measurement window of MMN. The model demonstrated opposite polarity changes in MMN amplitude produced by falling (70 dB) and rising (90 dB) intensity deviant stimuli, consistent with loudness dependence of sensory ERP components. This modelling study suggests that loudness dependence is a principal driver of intensity MMN, and future studies ought to clarify the distinction between loudness dependence, adaptation and prediction-error signalling.

Modelling mouse auditory response dynamics along a continuum of consciousness using a deep recurrent neural network.

Objective.Understanding neurophysiological changes that accompany transitions between anaesthetized and conscious states is a key objective of anesthesiology and consciousness science. This study aimed to characterize the dynamics of auditory-evoked potential morphology in mice along a continuum of consciousness.Approach.Epidural field potentials were recorded from above the primary auditory cortices of two groups of laboratory mice: urethane-anaesthetized (A,n= 14) and conscious (C,n= 17). Both groups received auditory stimulation in the form of a repeated pure-tone stimulus, before and after receiving 10 mg kg-1i.p. ketamine (AK and CK). Evoked responses were then ordered by ascending sample entropy into AK, A, CK, and C, considered to reflect physiological correlates of awareness. These data were used to train a recurrent neural network (RNN) with an input parameter encoding state. Model outputs were compared with grand-average event-related potential (ERP) waveforms. Subsequently, the state parameter was varied to simulate changes in the ERP that occur during transitions between states, and relationships with dominant peak amplitudes were quantified.Main results.The RNN synthesized output waveforms that were in close agreement with grand-average ERPs for each group (r2> 0.9,p< 0.0001). Varying the input state parameter generated model outputs reflecting changes in ERP morphology predicted to occur between states. Positive peak amplitudes within 25-50 ms, and negative peak amplitudes within 50-75 ms post-stimulus-onset, were found to display a sigmoidal characteristic during the transition from anaesthetized to conscious states. In contrast, negative peak amplitudes within 0-25 ms displayed greater linearity.Significance.This study demonstrates a method for modelling changes in ERP morphology that accompany transitions between states of consciousness using an RNN. In future studies, this approach may be applied to human data to support the clinical use of ERPs to predict transition to consciousness.

Identifying Obviously Artificial Medical Images Produced by a Generative Adversarial Network.

Synthetic medical images have an important role to play in developing data-driven medical image processing systems. Using a relatively small amount of patient data to train generative models that can produce an abundance of additional samples could bridge the gap towards big-data in niche medical domains. These generative models are evaluated in terms of the synthetic data they generate using the Visual Turing Test (VTT), Fréchet Inception Distance (FID), and other metrics. However, these are generally interpreted at the group level, and do not measure the artificiality of individual synthetic images. The present study attempts to address the challenge of automatically identifying artificial images that are obviously-artificial-looking, which may be necessary for filtering out poorly constructed synthetic images that might otherwise deteriorate the performance of assimilating systems. Synthetic computed tomography (CT) images from a progressively-grown generative adversarial network (PGGAN) were evaluated with a VTT and their image embeddings were analyzed for correlation with artificiality. Images categorized as obviously-artificial (≥0. 7 probability of being rated as fake) were classified using a battery of algorithms. The top-performing classifier, a support vector machine, exhibited accuracy of 75.5%, sensitivity of 0.743, and specificity of 0.769. This is an encouraging result that suggests a potential approach for validating synthetic medical image datasets. Clinical Relevance - Next-generation medical AI systems for image processing will utilize synthetic images produced by generative models. This paper presents an approach towards verifying artificial image legibility for quality-control before being deployed for these purposes.

An Electric Circuit Model of Central Auditory Processing that Replicates Low-level Features of the Mouse Mismatch Response.

Neurophysiology research using animals is often necessary to further our understanding of particular areas of medical interest. Human mismatch negativity (MMN) is one such area, where animal models are used to explore underlying mechanisms more invasively and with greater precision than typically possible with human subjects. Computational models can supplement these efforts by providing abstractions that lead to new insights and drive hypotheses. This study aims to establish whether a mouse mismatch response (MMR) analogous to human MMN can be modelled using electric circuit theory. Input to the auditory cortex was modelled as a step function multiplied by a frequency-dependent weighting designed to reflect spectral hearing sensitivity. Afferent sensory responses were modelled using a resistor-capacitor (RC) network, while bidirectional (bottom-up and top-down) responses were modelled using a resistor-inductor-capacitor (RLC) network. Synthetic EEG was combined with RC and RLC circuit currents in response to simulated sequences of auditory input, which comprised duration and frequency oddball paradigms. Two different states of awareness were considered: i) anaesthetized, including only the RC circuit, and ii) conscious, including both RC and RLC circuits. Event-related potential waveforms were obtained from ten simulated experiments for each oddball paradigm and state. These were qualitatively and quantitatively compared with data from a previous in-vivo study, and the model was deemed to successfully replicate low-level features of the mouse central auditory response. Clinical Relevance - Abnormal MMN is believed to reflect pathological changes associated with psychiatric disease. Maximizing the effectiveness of this biomarker will require a greater understanding of the specific cause(s) of these abnormalities. This study presents a computational model that can account for differences between responses to duration and frequency oddball paradigms, which is particularly significant for clinical MMN research.

Decoding violated sensory expectations from the auditory cortex of anaesthetised mice: Hierarchical recurrent neural network depicts separate 'danger' and 'safety' units.

The ability to respond appropriately to sensory information received from the external environment is among the most fundamental capabilities of central nervous systems. In the auditory domain, processes underlying this behaviour are studied by measuring auditory-evoked electrophysiology during sequences of sounds with predetermined regularities. Identifying neural correlates of ensuing auditory novelty responses is supported by research in experimental animals. In the present study, we reanalysed epidural field potential recordings from the auditory cortex of anaesthetised mice during frequency and intensity oddball stimulation. Multivariate pattern analysis (MVPA) and hierarchical recurrent neural network (RNN) modelling were adopted to explore these data with greater resolution than previously considered using conventional methods. Time-wise and generalised temporal decoding MVPA approaches revealed previously underestimated asymmetry between responses to sound-level transitions in the intensity oddball paradigm, in contrast with tone frequency changes. After training, the cross-validated RNN model architecture with four hidden layers produced output waveforms in response to simulated auditory inputs that were strongly correlated with grand-average auditory-evoked potential waveforms (r2 > .9). Units in hidden layers were classified based on their temporal response properties and characterised using principal component analysis and sample entropy. These demonstrated spontaneous alpha rhythms, sound onset and offset responses and putative 'safety' and 'danger' units activated by relatively inconspicuous and salient changes in auditory inputs, respectively. The hypothesised existence of corresponding biological neural sources is naturally derived from this model. If proven, this could have significant implications for prevailing theories of auditory processing.

Roving oddball paradigm elicits sensory gating, frequency sensitivity, and long-latency response in common marmosets.

Mismatch negativity (MMN) is a candidate biomarker for neuropsychiatric disease. Understanding the extent to which it reflects cognitive deviance-detection or purely sensory processes will assist practitioners in making informed clinical interpretations. This study compares the utility of deviance-detection and sensory-processing theories for describing MMN-like auditory responses of a common marmoset monkey during roving oddball stimulation. The following exploratory analyses were performed on an existing dataset: responses during the transition and repetition sequence of the roving oddball paradigm (standard -> deviant/S1 -> S2 -> S3) were compared; long-latency potentials evoked by deviant stimuli were examined using a double-epoch waveform subtraction; effects of increasing stimulus repetitions on standard and deviant responses were analyzed; and transitions between standard and deviant stimuli were divided into ascending and descending frequency changes to explore contributions of frequency-sensitivity. An enlarged auditory response to deviant stimuli was observed. This decreased exponentially with stimulus repetition, characteristic of sensory gating. A slow positive deflection was viewed over approximately 300-800 ms after the deviant stimulus, which is more difficult to ascribe to afferent sensory mechanisms. When split into ascending and descending frequency transitions, the resulting difference waveforms were disproportionally influenced by descending frequency deviant stimuli. This asymmetry is inconsistent with the general deviance-detection theory of MMN. These findings tentatively suggest that MMN-like responses from common marmosets are predominantly influenced by rapid sensory adaptation and frequency preference of the auditory cortex, while deviance-detection may play a role in long-latency activity.

Can intensity modulation of the auditory response explain intensity-decrement mismatch negativity?

Mismatch negativity (MMN) elicited by decrements in sound pressure level has been asserted as evidence for its dependence upon general deviance detection, while refuting the proposition that it is simply caused by modulating the intrinsic sensory response with different physical properties of sound. However, reports of intensity-decrement MMN are sparse compared with MMN to stimulus frequency or duration changes, and verifying the mechanisms that shape difference waveform morphology is essential for their responsible use as clinical biomarkers. In the present study, open-access EEG data from 40 healthy young adults recorded during an intensity-decrement oddball paradigm was analyzed to establish the effects of transitions between different level stimuli on the auditory evoked response. Standard stimuli were 80 dB and deviant stimuli were 70 dB. Event-related potentials were extracted from standards after standards (sS), deviants after standards (sD), and standards after deviants (dS). Mean amplitude across a recommended measurement window for MMN (125 to 225 ms) was calculated for each ERP waveform. This revealed statistically significant negative amplitude shift elicited by lower-intensity deviant stimuli, as expected, and an opposite direction, positive amplitude shift elicited by higher-intensity standard stimuli that followed lower-intensity deviants, relative to standard stimuli presented consecutively. These findings indicate that intensity-modulation of the auditory response influences cortical activity measured during the latency range of MMN. To what extent the hypothesized deviance detection mechanisms may also contribute is uncertain and remains to be elucidated.

More evidence for a long-latency mismatch response in urethane-anaesthetised mice.

Long-latency mismatch responses to oddball stimuli have recently been observed from anaesthetised rodents. This electrophysiological activity is viewed through 200 to 700 ms post-stimulus; a window that is typically obstructed from analysis by the response to subsequent stimuli in the auditory paradigm. A novel difference waveform computation using two adjoining evoked responses has enabled visualisation of this activity over a longer window than previously available. In the present study, this technique was retroactively applied to data from 13 urethane-anaesthetised mice. Oddball paradigm waveforms were compared with those of a many-standards control sequence, confirming that oddball stimuli evoked long-latency potentials that did not arise from standard or control stimuli. Statistical tests were performed to identify regions of significant difference. Oddball-induced mismatch responses were found to display significantly greater long-latency potentials than identical stimuli presented in an equal-probability context. As such, it may be concluded that long-latency potentials were evoked by the oddball condition. How this feature of the anaesthetised rodent mismatch response relates to human mismatch negativity is unclear, although it may be tentatively linked to the human P3a component, which emerges downstream from mismatch negativity under certain conditions. These results demonstrate that the time dynamics of mismatch responses from anaesthetised rodents are more extensive than previously considered.

Classical and controlled auditory mismatch responses to multiple physical deviances in anaesthetised and conscious mice.

Human mismatch negativity (MMN) is modelled in rodents and other non-human species to examine its underlying neurological mechanisms, primarily described in terms of deviance-detection and adaptation. Using the mouse model, we aim to elucidate subtle dependencies between the mismatch response (MMR) and different physical properties of sound. Epidural field potentials were recorded from urethane-anaesthetised and conscious mice during oddball and many-standards control paradigms with stimuli varying in duration, frequency, intensity and inter-stimulus interval. Resulting auditory evoked potentials, classical MMR (oddball - standard), and controlled MMR (oddball - control) waveforms were analysed. Stimulus duration correlated with stimulus-off response peak latency, whereas frequency, intensity and inter-stimulus interval correlated with stimulus-on N1 and P1 (conscious only) peak amplitudes. These relationships were instrumental in shaping classical MMR morphology in both anaesthetised and conscious animals, suggesting these waveforms reflect modification of normal auditory processing by different physical properties of sound. Controlled MMR waveforms appeared to exhibit habituation to auditory stimulation over time, which was equally observed in response to oddball and standard stimuli. These findings are inconsistent with the mechanisms thought to underlie human MMN, which currently do not address differences due to specific physical features of sound. Thus, no evidence was found to objectively support the deviance-detection or adaptation hypotheses of MMN in relation to anaesthetised or conscious mice. These findings highlight the potential risk of mischaracterising difference waveform components that are principally influenced by physical sensitivities and habituation of the auditory system.

Double-epoch subtraction reveals long-latency mismatch response in urethane-anaesthetized mice.

BACKGROUND: Anaesthetized rodents are examined for their capacity to model human mismatch negativity (MMN). In the present study, oddball and deviant-alone control paradigms, with stimuli varying in frequency (ascending and descending) and intensity (louder and quieter), were presented to anaesthetized mice to determine whether they elicit a translational mismatch response (MMR). NEW METHOD: Resulting waveforms displayed long-latency (>200 ms post-stimulus) components, only made fully visible from oddball paradigm data by applying a double-epoch subtraction. In this approach, an extended epoch containing two consecutive standard evoked responses was subtracted from the response to an oddball followed by a standard (i.e. oddball:standard - standard:standard). RESULTS: The trailing standard responses effectively cancelled each other out, revealing biphasic long-latency components. These MMR waveforms correlated strongly with deviant-alone paradigm evoked potentials >200 ms post-stimulus, potentially indicative of shared underlying mechanisms. Interestingly, these components were absent from the quieter oddball MMR. COMPARISON WITH EXISTING METHOD(S): Classical mismatch negativity computation is incapable of fully characterizing the long-latency biphasic response observed from this study, due to the inbuilt constraint of a single stimulus epoch. These results also suggest that the deviant-alone paradigm may be considered akin to a positive control for sensory-memory disruption, widely thought to be at the root of MMN generation in humans. CONCLUSIONS: Long-latency auditory evoked potential components are observed from anaesthetized mice in response to frequency and increasing intensity oddball stimuli. These display some congruencies with human MMN.

Event-related potential arithmetic to analyze offset potentials from conscious mice.

BACKGROUND: This paper presents a method for isolating time-dependent event-related potential (ERP) components which are superimposed on the gross ERP waveform. The experimental data that inspired this approach was recorded from the auditory cortex of conscious laboratory mice in response to presentation of ten different duration pure-tone auditory stimuli. NEW METHOD: The grand-average ERP for each individual stimulus displayed a relatively low amplitude deflection following stimulus offset. In order to isolate this component for analysis, a series of simple arithmetic operations were performed, involving averaging of multiple stimuli ERPs and subtracting this from each individual ERP. RESULTS: Offset potentials were isolated and quantified. Peak latency was determined by auditory stimulus duration; peak amplitude did not reach the threshold for statistical significance, over the range of durations tested. COMPARISON WITH EXISTING METHOD(S): To the best of my knowledge there are no alternative methods for isolating offset potentials from the gross ERP waveform at present. This novel approach may introduce less subjective bias to analyses than manually selecting measurement windows and performing custom baseline corrections. CONCLUSIONS: A similar method may be applied to other human or non-human datasets to identify and characterize time-dependent sensory-cognitive processes obscured by gross waveform morphology.