Integrated information theory (IIT) 4.0: Formulating the properties of phenomenal existence in physical terms.

This paper presents Integrated Information Theory (IIT) 4.0. IIT aims to account for the properties of experience in physical (operational) terms. It identifies the essential properties of experience (axioms), infers the necessary and sufficient properties that its substrate must satisfy (postulates), and expresses them in mathematical terms. In principle, the postulates can be applied to any system of units in a state to determine whether it is conscious, to what degree, and in what way. IIT offers a parsimonious explanation of empirical evidence, makes testable predictions concerning both the presence and the quality of experience, and permits inferences and extrapolations. IIT 4.0 incorporates several developments of the past ten years, including a more accurate formulation of the axioms as postulates and mathematical expressions, the introduction of a unique measure of intrinsic information that is consistent with the postulates, and an explicit assessment of causal relations. By fully unfolding a system's irreducible cause-effect power, the distinctions and relations specified by a substrate can account for the quality of experience.

IIT is ideally positioned to explain perceptual phenomena.

The target article's critique of the integrated information theory (IIT) of consciousness is misguided on several fronts, which I hope are addressed in other comments, but here I focus on the connection (or supposed lack thereof) between IIT and rigorous phenomenology, and IIT's connection to the psychophysics of perception.

Of maps and grids.

Neuroscience has made remarkable advances in accounting for how the brain performs its various functions. Consciousness, too, is usually approached in functional terms: the goal is to understand how the brain represents information, accesses that information, and acts on it. While useful for prediction, this functional, information-processing approach leaves out the subjective structure of experience: it does not account for how experience feels. Here, we consider a simple model of how a "grid-like" network meant to resemble posterior cortical areas can represent spatial information and act on it to perform a simple "fixation" function. Using standard neuroscience tools, we show how the model represents topographically the retinal position of a stimulus and triggers eye muscles to fixate or follow it. Encoding, decoding, and tuning functions of model units illustrate the working of the model in a way that fully explains what the model does. However, these functional properties have nothing to say about the fact that a human fixating a stimulus would also "see" it-experience it at a location in space. Using the tools of Integrated Information Theory, we then show how the subjective properties of experienced space-its extendedness-can be accounted for in objective, neuroscientific terms by the "cause-effect structure" specified by the grid-like cortical area. By contrast, a "map-like" network without lateral connections, meant to resemble a pretectal circuit, is functionally equivalent to the grid-like system with respect to representation, action, and fixation but cannot account for the phenomenal properties of space.

What is visible across the visual field?

It is sometimes claimed that because the resolution and sensitivity of visual perception are better in the fovea than in the periphery, peripheral vision cannot support the same kinds of colour and sharpness percepts as foveal vision. The fact that a scene nevertheless seems colourful and sharp throughout the visual field then poses a puzzle. In this study, I use a detailed model of human spatial vision to estimate the visibility of certain properties of natural scenes, including aspects of colourfulness, sharpness, and blurriness, across the visual field. The model is constructed to reproduce basic aspects of human contrast and colour sensitivity over a range of retinal eccentricities. I apply the model to colourful, complex natural scene images, and estimate the degree to which colour and edge information are present in the model's representation of the scenes. I find that, aside from the intrinsic drift in the spatial scale of the representation, there are not large qualitative differences between foveal and peripheral representations of 'colourfulness' and 'sharpness'.

Consciousness depends on integration between parietal cortex, striatum, and thalamus.

The neural substrates of consciousness remain elusive. Competing theories that attempt to explain consciousness disagree on the contribution of frontal versus posterior cortex and omit subcortical influences. This lack of understanding impedes the ability to monitor consciousness, which can lead to adverse clinical consequences. To test substrates and measures of consciousness, we recorded simultaneously from frontal cortex, parietal cortex, and subcortical structures, the striatum and thalamus, in awake, sleeping, and anesthetized macaques. We manipulated consciousness on a finer scale using thalamic stimulation, rousing macaques from continuously administered anesthesia. Our results show that, unlike measures targeting complexity, a measure additionally capturing neural integration (Φ∗) robustly correlated with changes in consciousness. Machine learning approaches show parietal cortex, striatum, and thalamus contributed more than frontal cortex to decoding differences in consciousness. These findings highlight the importance of integration between parietal and subcortical structures and challenge a key role for frontal cortex in consciousness.

Conscious Perception as Integrated Information Patterns in Human Electrocorticography.

A significant problem in neuroscience concerns the distinction between neural processing that is correlated with conscious percepts from processing that is not. Here, we tested if a hierarchical structure of causal interactions between neuronal populations correlates with conscious perception. We derived the hierarchical causal structure as a pattern of integrated information, inspired by the integrated information theory (IIT) of consciousness. We computed integrated information patterns from intracranial electrocorticography (ECoG) from six human neurosurgical patients with electrodes implanted over lateral and ventral cortices. During recording, subjects viewed continuous flash suppression (CFS) and backward masking (BM) stimuli intended to dissociate conscious percept from stimulus, and unmasked suprathreshold stimuli. Object-sensitive areas revealed correspondence between conscious percepts and integrated information patterns. We quantified this correspondence using unsupervised classification methods that revealed clustering of visual experiences with integrated information, but not with broader information measures including mutual information and entropy. Our findings point to a significant role of locally integrated information for understanding the neural substrate of conscious object perception.

Plasticity in the Structure of Visual Space.

Visual space embodies all visual experiences, yet what determines the topographical structure of visual space remains unclear. Here we test a novel theoretical framework that proposes intrinsic lateral connections in the visual cortex as the mechanism underlying the structure of visual space. The framework suggests that the strength of lateral connections between neurons in the visual cortex shapes the experience of spatial relatedness between locations in the visual field. As such, an increase in lateral connection strength shall lead to an increase in perceived relatedness and a contraction in perceived distance. To test this framework through human psychophysics experiments, we used a Hebbian training protocol in which two-point stimuli were flashed in synchrony at separate locations in the visual field, to strengthen the lateral connections between two separate groups of neurons in the visual cortex. After training, participants experienced a contraction in perceived distance. Intriguingly, the perceptual contraction occurred not only between the two training locations that were linked directly by the changed connections, but also between the outward untrained locations that were linked indirectly through the changed connections. Moreover, the effect of training greatly decreased if the two training locations were too close together or too far apart and went beyond the extent of lateral connections. These findings suggest that a local change in the strength of lateral connections is sufficient to alter the topographical structure of visual space.

Are we underestimating the richness of visual experience?

It has been argued that the bandwidth of perceptual experience is low-that the richness of experience is illusory and that the amount of visual information observers can perceive and remember is extremely limited. However, the evidence suggests that this postulated poverty of experiential content is illusory and that visual phenomenology is immensely rich. To properly estimate perceptual content, experimentalists must move beyond the limitations of binary alternative-forced choice procedures and analyze reports of experience more broadly. This will open our eyes to the true richness of experience and to its neuronal substrates.

On the Differentiation of Foveal and Peripheral Early Visual Evoked Potentials.

The C1 is one of the earliest visual evoked potentials observed following the onset of a patterned stimulus. The polarity of its peak is dependent on whether stimuli are presented in the upper or lower regions of the peripheral visual field, but has been argued to be negative for stimuli presented to the fovea. However, there has yet to be a systematic investigation into the extent to which the peripheral C1 (pC1) and foveal C1 (fC1) can be differentiated on the basis of response characteristics to different stimuli. The current study employed checkerboard patterns (Exp 1) and sinusoidal gratings of different spatial frequency (Exp 2) presented to the fovea or within one of the four quadrants of the peripheral visual field. The checkerboard stimuli yielded a sizable difference in peak component latency, with the fC1 peaking ~32 ms after the pC1. Further, the pC1 showed a band-pass response magnitude profile that peaked at 4 cycles per degree (cpd), whereas the fC1 was high-pass for spatial frequency, with a cut-off around 4 cpd. Finally, the scalp topographies of the pC1 and fC1 in both experiments differed greatly, with the fC1 being more posterior than the pC1. The results reported here call into question recent attempts to characterize general C1 processes without regard to whether stimuli are placed in the fovea or in the periphery.

Optimal Combination of the Binocular Cues to 3D Motion.

PURPOSE: Perception necessarily entails combining separate sensory estimates into a single coherent whole. The perception of three-dimensional (3D) motion, for instance, can rely on two binocular cues: one related to the change in binocular disparity over time (CD) and the other related to interocular velocity differences (IOVD). Although previous work has shown that neither cue is strictly necessary for the perception of 3D motion, observers are able to judge 3D motion in displays in which one or the other cue has been eliminated, it is unclear whether or how the two cues are combined in situations in which both are present. METHODS: We tested the visual performance of a sample of 81 individuals (Mage = 20.34, 49 females) in four main conditions that measured, respectively, static stereoacuity, CD, IOVD, and combined CD+IOVD sensitivity. RESULTS: We show that the sensitivity to the two binocular cues to 3D motion varies substantially across observers (CD: Md' = 1.01, SDd' = 1.1; IOVD: Md' = 1.16, SDd' = 1.03). Furthermore, sensitivity to the two cues was independent across observers (r[48] = 0.12, P = 0.42). Importantly, however, observed CD+IOVD performance was well-predicted based on the assumption that each observer combines the two cues in a statistically optimal fashion (r[79] = 0.75, P < 0.001). CONCLUSIONS: Our findings provide an explanation for the previously puzzling variability found in 3D perception across observers and laboratories, with some results suggesting that motion-in-depth percepts are largely determined by changes in binocular disparity, whereas others indicate that interocular velocity differences are key. Our results underline the existence of two complementary binocular mechanisms underlying 3D motion perception, with observers relying on these two mechanisms to different extents depending on their individual sensitivity.

Similar Sensitivity to Ladder Contours in Macular Degeneration Patients and Controls.

PURPOSE: To determine whether people with central field loss (CFL) from macular degeneration have improved ability to recognize a particularly difficult spatial configuration embedded in noise, the peripherally-viewed 'ladder contour'. The visibility of these configuration has been linked to general contour integration ability and crowding limitations in peripheral vision. METHODS: We used a trial-based yes-no task. CFL patients and normally-sighted controls performed the task, looking for ladder contours embedded in a field of randomly oriented Gabor patches, at a range of stimulus presentation times (varying stimulus difficulty). Viewing eccentricity in CFL patients was set by their preferred retinal loci (PRLs) and matched artificially in the control group. The contours were presented so as to be tangent to the CFL region, given a patient's PRL location. RESULTS: CFL and normally-sighted groups performed similarly on the task. The only significant determinant of performance was the viewing eccentricity. CONCLUSIONS: CFL patients do not seem to develop any improved ability to recognize ladder contours with their parafoveal retina, which suggests that there is no underlying improvement in contour integration or reduction in crowding limitations in the region of the PRL despite extended daily use.

Binocular rivalry with peripheral prisms used for hemianopia rehabilitation.

PURPOSE: To determine the relative binocular signal strength of moving images that are peripherally viewed through a monocular field expansion prism as opposed to moving images viewed directly. We hypothesised that prism blur might make prism images predominate less than images viewed directly with the other eye. METHODS: We employed the binocular rivalry paradigm to measure the relative binocular effectiveness of directly viewed vs prism images. Four normally-sighted subjects tracked the rivalrous visibility of opponent-coloured targets seen dichoptically in the same part of the retinal visual field, using monocular field expansion prisms to produce the dichoptic display. We analysed the effects of external signal strength (whether or not motion was present in either image), retinal position or eccentricity of the targets, and controlled for target saturation. RESULTS: We found that prism images predominate less than directly viewed images. When both eyes were presented with pattern in the dichoptic display, direct-to-prism predominance was 51%:31%. When only the direct view was presented with pattern, direct-to-prism predominance was 74%:12%; when only the prism view was presented with pattern, direct-to-prism predominance was 25%:58%. Dominance durations followed established binocular rivalry rules. CONCLUSIONS: The prism image in a monocular, peripheral field expansion prism is perceptually weaker than the corresponding direct image in the other eye. However, the prism image is still seen a significant proportion of the time, especially when no moving pattern is present in the direct view. We conclude that the rivalry ratio of the prism device is sufficiently effective for clinical applications.

Perceived contrast in complex images.

To understand how different spatial frequencies contribute to the overall perceived contrast of complex, broadband photographic images, we adapted the classification image paradigm. Using natural images as stimuli, we randomly varied relative contrast amplitude at different spatial frequencies and had human subjects determine which images had higher contrast. Then, we determined how the random variations corresponded with the human judgments. We found that the overall contrast of an image is disproportionately determined by how much contrast is between 1 and 6 c/°, around the peak of the contrast sensitivity function (CSF). We then employed the basic components of contrast psychophysics modeling to show that the CSF alone is not enough to account for our results and that an increase in gain control strength toward low spatial frequencies is necessary. One important consequence of this is that contrast constancy, the apparent independence of suprathreshold perceived contrast and spatial frequency, will not hold during viewing of natural images. We also found that images with darker low-luminance regions tended to be judged as having higher overall contrast, which we interpret as the consequence of darker local backgrounds resulting in higher band-limited contrast response in the visual system.

Using pattern classification to measure adaptation to the orientation of high order aberrations.

BACKGROUND: The image formed by the eye's optics is blurred by the ocular aberrations, specific to each eye. Recent studies demonstrated that the eye is adapted to the level of blur produced by the high order aberrations (HOA). We examined whether visual coding is also adapted to the orientation of the natural HOA of the eye. METHODS AND FINDINGS: Judgments of perceived blur were measured in 5 subjects in a psychophysical procedure inspired by the "Classification Images" technique. Subjects were presented 500 pairs of images, artificially blurred with HOA from 100 real eyes (i.e. different orientations), with total blur level adjusted to match the subject's natural blur. Subjects selected the image that appeared best focused in each random pair, in a 6-choice ranked response. Images were presented through Adaptive Optics correction of the subject's aberrations. The images selected as best focused were identified as positive, the other as negative responses. The highest classified positive responses correlated more with the subject's Point Spread Function, PSF, (r = 0.47 on average) than the negative (r = 0.34) and the difference was significant for all subjects (p<0.02). Using the orientation of the best fitting ellipse of angularly averaged integrated PSF intensities (weighted by the subject's responses) we found that in 4 subjects the positive PSF response was close to the subject's natural PSF orientation (within 21 degrees on average) whereas the negative PSF response was almost perpendicularly oriented to the natural PSF (at 76 degrees on average). CONCLUSIONS: The Classification-Images inspired method is very powerful in identifying the internally coded blur of subjects. The consistent bias of the Positive PSFs towards the natural PSF in most subjects indicates that the internal code of blur appears rather specific to each subject's high order aberrations and reveals that the calibration mechanisms for normalizing blur also operate using orientation cues.

Adaptation to blurred and sharpened video.

The visual system can distinguish different levels of blur and different levels of excess sharpness. Adaptation alters this capacity so that the adapted blur (or sharp) level looks more like a normal, properly focused image. Here, we describe the more general pattern of aftereffects of blur and sharp adaptation by measuring matching functions, using video clips from a DVD movie as stimuli. Results show that blur and sharp adaptation are selective: The sharpening aftereffects of blur adaptation are strongest for blurry videos while the blurring aftereffects of sharp adaptation are strongest for sharp videos. Despite the spatiotemporal variability of our adaptor and test stimuli, we found adaptation effects similar in magnitude to previous studies using invariant static images. A recent model of blur adaptation can be simplified to explain the form of our data, leading us to conclude that what we see as blur/sharp adaptation is a consequence of narrowband contrast adaptation.

Contrast sensitivity for oriented patterns in 1/f noise: contrast response and the horizontal effect.

When observers detect an oriented, broadband contrast increment on a background of 1/f spatial noise, thresholds will be lowest for obliquely orientated stimuli and highest for horizontally oriented stimuli-an anisotropy termed the "horizontal effect." Here, we assessed what spatial frequencies within the broadband increment were relied on by observers in performing the original task and which spatial frequencies contribute to the anisotropic performance. We found that against a background of 1/f noise, contrast thresholds are lowest for content around 8 cycles per degree, and that at this spatial frequency a horizontal effect is seen which closely resembles the anisotropy observed in broadband masking. The magnitude of the horizontal effect decreased at lower and higher spatial frequencies. To allow for a fit to a standard "gain control" model of psychophysical contrast discrimination, threshold-versus-contrast (TvC) functions were measured for the 8-cpd noise broadband content against either an identical pattern (i.e., pedestal) or a broadband 1/f noise pattern, whose contrast was varied. Results and model application indicate that the threshold pattern for oriented noise around 8 cpd, and for oriented broadband content, is best explained as the result of an anisotropic contrast gain control process.

The horizontal effect in suppression: Anisotropic overlay and surround suppression at high and low speeds.

When a pattern of broad spatial content is viewed by an observer, the multiple spatial components in the pattern stimulate detecting-mechanisms that suppress each other. This suppression is anisotropic, being relatively greater at horizontal, and least at obliques (the "horizontal effect"). Here, suppression of a grating by a naturalistic (1/f) broadband mask is shown to be larger when the broadband masks are temporally similar to the target's temporal properties, and generally anisotropic, with the anisotropy present across all spatio-temporal parings tested. We also show that both suppression from within the region of the test pattern (overlay suppression) and from outside of this region (surround suppression) show the horizontal-effect anisotropy. We conclude that these suppression effects stem from locally-tuned and anisotropically-weighted gain-control pools.

An anisotropy of orientation-tuned suppression that matches the anisotropy of typical natural scenes.

Broadband oriented-noise masks were used to assess the orientation properties of spatial-context suppression in 'general' viewing conditions (i.e., a fixated, large field of 'naturalistic' noise). Suppression was orientation-tuned with a Gaussian shape and bandwidth of 40 degrees that was consistent across test orientation (0 degrees, 45 degrees, 90 degrees, and 135 degrees). Strength of suppression was highly anisotropic following a "horizontal effect" pattern (strongest suppression at horizontal and least suppression at oblique test orientations). Next, the time course of anisotropic masking was investigated by varying stimulus onset asynchrony (SOA). A standard "oblique effect" anisotropy is observed at long SOAs but becomes a "horizontal effect" when a noise mask is present within approximately 50 ms of the test onset. The orientation-tuned masking appears to result from an anisotropic gain-control mechanism that pools the weighted responses to the broadband mask, resulting in a changeover from oblique effect to horizontal effect. In addition, the relative magnitude of suppression at the orientations tested corresponds to the relative magnitudes of the content of typical natural scenes at the same orientations. We suggest that this anisotropic suppression may serve to equalize the visual system's response across orientation when viewing typical natural scenes, 'discounting' the anisotropy of typical natural scene content.

Illusory bands in orientation and spatial frequency: a cortical analog to Mach bands.

Illusory bands at a luminance transition in space (ie an edge) are well known. Here we demonstrate illusory bands of enhanced orientations or spatial frequencies at transitions between higher-contrast and lower-contrast image content along the orientation and spatial-frequency dimensions--the dimensions of cortical spatial coding. We conclude that this illusion is a consequence of cortical-level suppression of units of similar orientations and spatial frequencies and serves to aid texture segmentation while providing efficient neural coding.