Spatiotemporal bias of the human gaze toward hierarchical visual features during natural scene viewing.

The human gaze is directed at various locations from moment to moment in acquiring information necessary to recognize the external environment at the fine resolution of foveal vision. Previous studies showed that the human gaze is attracted to particular locations in the visual field at a particular time, but it remains unclear what visual features produce such spatiotemporal bias. In this study, we used a deep convolutional neural network model to extract hierarchical visual features from natural scene images and evaluated how much the human gaze is attracted to the visual features in space and time. Eye movement measurement and visual feature analysis using the deep convolutional neural network model showed that the gaze was more strongly attracted to spatial locations containing higher-order visual features than to locations containing lower-order visual features or to locations predicted by conventional saliency. Analysis of the time course of gaze attraction revealed that the bias to higher-order visual features was prominent within a short period after the beginning of observation of the natural scene images. These results demonstrate that higher-order visual features are a strong gaze attractor in both space and time, suggesting that the human visual system uses foveal vision resources to extract information from higher-order visual features with higher spatiotemporal priority.

Bodily ownership of an independent supernumerary limb: an exploratory study.

Can our brain perceive a sense of ownership towards an independent supernumerary limb; one that can be moved independently of any other limb and provides its own independent movement feedback? Following the rubber-hand illusion experiment, a plethora of studies have shown that the human representation of "self" is very plastic. But previous studies have almost exclusively investigated ownership towards "substitute" artificial limbs, which are controlled by the movements of a real limb and/or limbs from which non-visual sensory feedback is provided on an existing limb. Here, to investigate whether the human brain can own an independent artificial limb, we first developed a novel independent robotic "sixth finger." We allowed participants to train using the finger and examined whether it induced changes in the body representation using behavioral as well as cognitive measures. Our results suggest that unlike a substitute artificial limb (like in the rubber hand experiment), it is more difficult for humans to perceive a sense of ownership towards an independent limb. However, ownership does seem possible, as we observed clear tendencies of changes in the body representation that correlated with the cognitive reports of the sense of ownership. Our results provide the first evidence to show that an independent supernumerary limb can be embodied by humans.

Information spreading by a combination of MEG source estimation and multivariate pattern classification.

To understand information representation in human brain activity, it is important to investigate its fine spatial patterns at high temporal resolution. One possible approach is to use source estimation of magnetoencephalography (MEG) signals. Previous studies have mainly quantified accuracy of this technique according to positional deviations and dispersion of estimated sources, but it remains unclear how accurately MEG source estimation restores information content represented by spatial patterns of brain activity. In this study, using simulated MEG signals representing artificial experimental conditions, we performed MEG source estimation and multivariate pattern analysis to examine whether MEG source estimation can restore information content represented by patterns of cortical current in source brain areas. Classification analysis revealed that the corresponding artificial experimental conditions were predicted accurately from patterns of cortical current estimated in the source brain areas. However, accurate predictions were also possible from brain areas whose original sources were not defined. Searchlight decoding further revealed that this unexpected prediction was possible across wide brain areas beyond the original source locations, indicating that information contained in the original sources can spread through MEG source estimation. This phenomenon of "information spreading" may easily lead to false-positive interpretations when MEG source estimation and classification analysis are combined to identify brain areas that represent target information. Real MEG data analyses also showed that presented stimuli were able to be predicted in the higher visual cortex at the same latency as in the primary visual cortex, also suggesting that information spreading took place. These results indicate that careful inspection is necessary to avoid false-positive interpretations when MEG source estimation and multivariate pattern analysis are combined.

Multivariate Analysis of Magnetic Resonance Imaging Signals of the Human Brain.

Magnetic resonance imaging (MRI) of the human brain plays an important role in the field of medical imaging as well as basic neuroscience. It measures proton spin relaxation, the time constant of which depends on tissue type, and allows us to visualize anatomical structures in the brain. It can also measure functional signals that depend on the local ratio of oxyhemoglobin to deoxyhemoglobin in the blood, which is believed to reflect the degree of neural activity in the corresponding area. MRI thus provides anatomical and functional information about the human brain with high spatial resolution. Conventionally, MRI signals are measured and analyzed for each individual voxel. However, these signals are essentially multivariate because they are measured from multiple voxels simultaneously, and the pattern of activity might carry more useful information than each individual voxel does. This paper reviews recent trends in multivariate analysis of MRI signals in the human brain, and discusses applications of this technique in the fields of medical imaging and neuroscience.

Inter-subject neural code converter for visual image representation.

Brain activity patterns differ from person to person, even for an identical stimulus. In functional brain mapping studies, it is important to align brain activity patterns between subjects for group statistical analyses. While anatomical templates are widely used for inter-subject alignment in functional magnetic resonance imaging (fMRI) studies, they are not sufficient to identify the mapping between voxel-level functional responses representing specific mental contents. Recent work has suggested that statistical learning methods could be used to transform individual brain activity patterns into a common space while preserving representational contents. Here, we propose a flexible method for functional alignment, "neural code converter," which converts one subject's brain activity pattern into another's representing the same content. The neural code converter was designed to learn statistical relationships between fMRI activity patterns of paired subjects obtained while they saw an identical series of stimuli. It predicts the signal intensity of individual voxels of one subject from a pattern of multiple voxels of the other subject. To test this method, we used fMRI activity patterns measured while subjects observed visual images consisting of random and structured patches. We show that fMRI activity patterns for visual images not used for training the converter could be predicted from those of another subject where brain activity was recorded for the same stimuli. This confirms that visual images can be accurately reconstructed from the predicted activity patterns alone. Furthermore, we show that a classifier trained only on predicted fMRI activity patterns could accurately classify measured fMRI activity patterns. These results demonstrate that the neural code converter can translate neural codes between subjects while preserving contents related to visual images. While this method is useful for functional alignment and decoding, it may also provide a basis for brain-to-brain communication using the converted pattern for designing brain stimulation.

Modular encoding and decoding models derived from bayesian canonical correlation analysis.

Neural encoding and decoding provide perspectives for understanding neural representations of sensory inputs. Recent functional magnetic resonance imaging (fMRI) studies have succeeded in building prediction models for encoding and decoding numerous stimuli by representing a complex stimulus as a combination of simple elements. While arbitrary visual images were reconstructed using a modular model that combined the outputs of decoder modules for multiscale local image bases (elements), the shapes of the image bases were heuristically determined. In this work, we propose a method to establish mappings between the stimulus and the brain by automatically extracting modules from measured data. We develop a model based on Bayesian canonical correlation analysis, in which each module is modeled by a latent variable that relates a set of pixels in a visual image to a set of voxels in an fMRI activity pattern. The estimated mapping from a latent variable to pixels can be regarded as an image basis. We show that the model estimates a modular representation with spatially localized multiscale image bases. Further, using the estimated mappings, we derive encoding and decoding models that produce accurate predictions for brain activity and stimulus images. Our approach thus provides a novel means of revealing neural representations of stimuli by automatically extracting modules, which can be used to generate effective prediction models for encoding and decoding.

Spike suppression in a local cortical circuit induced by transcranial magnetic stimulation.

Transcranial magnetic stimulation (TMS) noninvasively interferes with human cortical function, and is widely used as an effective technique for probing causal links between neural activity and cognitive function. However, the physiological mechanisms underlying TMS-induced effects on neural activity remain unclear. We examined the mechanism by which TMS disrupts neural activity in a local circuit in early visual cortex using a computational model consisting of conductance-based spiking neurons with excitatory and inhibitory synaptic connections. We found that single-pulse TMS suppressed spiking activity in a local circuit model, disrupting the population response. Spike suppression was observed when TMS was applied to the local circuit within a limited time window after the local circuit received sensory afferent input, as observed in experiments investigating suppression of visual perception with TMS targeting early visual cortex. Quantitative analyses revealed that the magnitude of suppression was significantly larger for synaptically-connected neurons than for isolated individual neurons, suggesting that intracortical inhibitory synaptic coupling also plays an important role in TMS-induced suppression. A conventional local circuit model of early visual cortex explained only the early period of visual suppression observed in experiments. However, models either involving strong recurrent excitatory synaptic connections or sustained excitatory input were able to reproduce the late period of visual suppression. These results suggest that TMS targeting early visual cortex disrupts functionally distinct neural signals, possibly corresponding to feedforward and recurrent information processing, by imposing inhibitory effects through intracortical inhibitory synaptic connections.

Influence of coherence between multiple cortical columns on alpha rhythm: a computational modeling study.

In electroencephalographic (EEG) and magnetoencephalographic (MEG) signals, stimulus-induced amplitude increase and decrease in the alpha rhythm, known as event-related synchronization and desynchronization (ERS/ERD), emerge after a task onset. ERS/ERD is assumed to reflect neural processes relevant to cognitive tasks. Previous studies suggest that several sources of alpha rhythm, each of which can serve as an alpha rhythm generator, exist in the cortex. Since EEG/MEG signals represent spatially summed neural activities, ERS/ERD of the alpha rhythm may reflect the consequence of the interactions between multiple alpha rhythm generators. Two candidates modulate the magnitude of ERS/ERD: (1) coherence between the activities of the alpha rhythm generators and (2) mean amplitude of the activities of the alpha rhythm generators. In this study, we use a computational model of multiple alpha rhythm generators to determine the factor that dominantly causes ERS/ERD. Each alpha rhythm generator is modeled based on local column circuits in the primary visual cortex and made to interact with the neighboring generators through excitatory connections. We observe that the model consistently reproduces spontaneous alpha rhythms, event-related potentials, phase-locked alpha rhythms, and ERS/ERD in a specific range of connectivity coefficients. Independent analyses of the coherence and amplitude of multiple alpha rhythm generators reveal that the ERS/ERD in the simulated data is dominantly caused by stimulus-induced changes in the coherence between multiple alpha rhythm generators. Nonlinear phenomena such as phase-resetting and entrainment of the alpha rhythm are related to the neural mechanism underlying ERS/ERD.

Visual image reconstruction from human brain activity using a combination of multiscale local image decoders.

Perceptual experience consists of an enormous number of possible states. Previous fMRI studies have predicted a perceptual state by classifying brain activity into prespecified categories. Constraint-free visual image reconstruction is more challenging, as it is impractical to specify brain activity for all possible images. In this study, we reconstructed visual images by combining local image bases of multiple scales, whose contrasts were independently decoded from fMRI activity by automatically selecting relevant voxels and exploiting their correlated patterns. Binary-contrast, 10 x 10-patch images (2(100) possible states) were accurately reconstructed without any image prior on a single trial or volume basis by measuring brain activity only for several hundred random images. Reconstruction was also used to identify the presented image among millions of candidates. The results suggest that our approach provides an effective means to read out complex perceptual states from brain activity while discovering information representation in multivoxel patterns.

A network model of perceptual suppression induced by transcranial magnetic stimulation.

We modeled the inhibitory effects of transcranial magnetic stimulation (TMS) on a neural population. TMS is a noninvasive technique, with high temporal resolution, that can stimulate the brain via a brief magnetic pulse from a coil placed on the scalp. Because of these advantages, TMS is extensively used as a powerful tool in experimental studies of motor, perception, and other functions in humans. However, the mechanisms by which TMS interferes with neural activities, especially in terms of theoretical aspects, are totally unknown. In this study, we focused on the temporal properties of TMS-induced perceptual suppression, and we computationally analyzed the response of a simple network model of a sensory feature detector system to a TMS-like perturbation. The perturbation caused the mean activity to transiently increase and then decrease for a long period, accompanied by a loss in the degree of activity localization. When the afferent input consisted of a dual phase, with a strong transient component and a weak sustained component, there was a critical latency period of the perturbation during which the network activity was completely suppressed and converged to the resting state. The range of the suppressive period increased with decreasing afferent input intensity and reached more than 10 times the time constant of the neuron. These results agree well with typical experimental data for occipital TMS and support the conclusion that dynamical interaction in a neural population plays an important role in TMS-induced perceptual suppression.

Unconscious adaptation: a new illusion of depth induced by stimulus features without depth.

Here, we show a new illusion of depth induced by psychophysical adaptation to dynamic random-dot stereograms (RDS) that are interocularly anticorrelated (i.e., in which the images for the two eyes have reversed contrast polarity with each other). After prolonged viewing of anticorrelated RDS, the presentation of uncorrelated RDS (i.e., in which two images are mutually independent random-dot patterns) produces the sensation of depth, although both anticorrelated and uncorrelated RDSs are perceptually rivalrous with no consistent depth by themselves. Contrary to other aftereffects demonstrated in a number of visual dimensions, including motion, orientation, and disparity, this illusion results from unconscious adaptation; observers are not aware of what they are being adapted to during the process of adaptation. We further demonstrate that this illusion can be predicted from the simulated responses of disparity-selective neurons based on a local filtering model. Model simulations indicate that the inspection of anticorrelated RDS causes the adaptation of all disparity detectors except one sensitive to its disparity; therefore, those selectively unadapted detectors show relatively strong activation in response to the subsequent presentation of uncorrelated RDS and produce depth perception.