Computational components of visual predictive coding circuitry.

If a full visual percept can be said to be a 'hypothesis', so too can a neural 'prediction' - although the latter addresses one particular component of image content (such as 3-dimensional organisation, the interplay between lighting and surface colour, the future trajectory of moving objects, and so on). And, because processing is hierarchical, predictions generated at one level are conveyed in a backward direction to a lower level, seeking to predict, in fact, the neural activity at that prior stage of processing, and learning from errors signalled in the opposite direction. This is the essence of 'predictive coding', at once an algorithm for information processing and a theoretical basis for the nature of operations performed by the cerebral cortex. Neural models for the implementation of predictive coding invoke specific functional classes of neuron for generating, transmitting and receiving predictions, and for producing reciprocal error signals. Also a third general class, 'precision' neurons, tasked with regulating the magnitude of error signals contingent upon the confidence placed upon the prediction, i.e., the reliability and behavioural utility of the sensory data that it predicts. So, what is the ultimate source of a 'prediction'? The answer is multifactorial: knowledge of the current environmental context and the immediate past, allied to memory and lifetime experience of the way of the world, doubtless fine-tuned by evolutionary history too. There are, in consequence, numerous potential avenues for experimenters seeking to manipulate subjects' expectation, and examine the neural signals elicited by surprising, and less surprising visual stimuli. This review focuses upon the predictive physiology of mouse and monkey visual cortex, summarising and commenting on evidence to date, and placing it in the context of the broader field. It is concluded that predictive coding has a firm grounding in basic neuroscience and that, unsurprisingly, there remains much to learn.

Transduction catalysis: Doxorubicin amplifies rAAV-mediated gene expression in the cortex of higher-order vertebrates.

Rapid and efficient gene transduction via recombinant adeno-associated viruses (rAAVs) is highly desirable across many basic and clinical research domains. Here, we report that vector co-infusion with doxorubicin, a clinical anti-cancer drug, markedly enhanced rAAV-mediated transgene expression in the cerebral cortex across mammalian species (cat, mouse, and macaque), acting throughout the time period examined and detectable at just three days after transfection. This enhancement showed serotype generality, being common to all rAAV serotypes tested (2, 8, 9, and PHP.eB) and was observed both locally and at remote locations consistent with doxorubicin undergoing retrograde axonal transport. All these effects were observed at doses matching human blood plasma levels in clinical therapy and lacked detectable cytotoxicity as assessed by cell morphology, activity, apoptosis, and behavioral testing. Altogether, this study identifies an effective means to improve the capability and scope of in vivo rAAV applications, amplifying cell transduction at doxorubicin concentrations paralleling medical practice.

Hierarchical Representation for Chromatic Processing across Macaque V1, V2, and V4.

The perception of color is an internal label for the inferred spectral reflectance of visible surfaces. To study how spectral representation is transformed through modular subsystems of successive cortical areas, we undertook simultaneous optical imaging of intrinsic signals in macaque V1, V2, and V4, supplemented by higher-resolution electrophysiology and two-photon imaging in awake macaques. We find a progressive evolution in the scale and precision of chromotopic maps, expressed by a uniform blob-like architecture of hue responses within each area. Two-photon imaging reveals enhanced hue-specific cell clustering in V2 compared with V1. A phenomenon of endspectral (red and blue) responses that is clear in V1, recedes in V2, and is virtually absent in V4. The increase in mid- and extra-spectral hue representations through V2 and V4 reflects the nature of hierarchical processing as higher areas read out locations in chromatic space from progressive integration of signals relayed by V1.

Immediate Impact of Acute Elevation of Intraocular Pressure on Cortical Visual Motion Processing.

Purpose: To physiologically examine the impairment of cortical sensitivity to visual motion during acute elevation of intraocular pressure (IOP). Methods: Motion processing in the cat brain is well characterized, its X and Y cell visual pathways being functionally analogous to parvocellular and magnocellular pathways in primates. Using this model, we performed ocular anterior chamber perfusion to reversibly elevate IOP over a range from 30 to 90 mm Hg while monitoring cortical activity with intrinsic signal optical imaging. Drifting random-dot fields and gratings were used to characterize cortical population responses to motion direction and orientation in early visual areas 17 and 18. Results: We found that acute IOP elevations at 50 mm Hg and above, which is often observed in acute glaucoma, suppressed cortical motion direction responses. This suppression was more profound in area 17 than in area 18, and more profound in central than peripheral visual field (eccentricities 0°-4° vs. 4°-8°) within area 17. In addition, orientation responses were more suppressed than motion direction responses for the same IOP modulation. Conclusions: In contrast to human chronic glaucoma that may cause greater dysfunction in large-cell magnocellular than in small-cell parvocellular visual pathways, our direct measurement of cortical processing networks implies that the small X-cell pathway shows greater vulnerability to acute IOP elevation than the large Y-cell pathway in visual motion processing. The results demonstrate that fine discrimination mechanisms for motion in the central visual field are particularly impacted by acute IOP attacks, suggesting a neural basis for immediate visual deficits in the fine motion perception of acute glaucoma patients.

Impact of acute intraocular pressure elevation on the visual acuity of non-human primates.

BACKGROUND: Glaucoma is the leading cause of irreversible blindness worldwide and elevated intraocular pressure (IOP) is an established risk factor. Visual acuity, the capacity for fine analysis of spatial frequency (SF) information, is relatively preserved in central vision until the later stages of chronic glaucoma. However, for acute glaucoma that is associated with sharp IOP elevation, how visual acuity is affected by acute IOP elevation remains unclear. METHODS: Using intrinsic-signal optical imaging of large areas of visual cortices V1 and V2 in seven rhesus macaques, visual acuity was directly examined during acute IOP elevation at 70 mmHg, a pressure often observed in acute angle-closure glaucoma. Acute IOP elevation was achieved by reversible monocular anterior chamber perfusions, and visual acuity was quantified by cortical population responses to various SFs ranging from 0.5-6 cycles/°. FINDINGS: Acute IOP elevation particularly depressed the ability of the visual cortex to register fine details (at high SFs referring to visual acuity), an effect that was progressively more severe toward the central visual field. These results completely contrast with long-term impairments present in chronic glaucoma. INTERPRETATION: Our results show that impairment of fine visual discrimination within the central visual field is the principal consequence of sharp IOP elevation, implicating relatively greater dysfunction in parvocellular pathways. This study provides direct cortical neural evidence for the immediate visual acuity impairment in acute glaucoma patients. FUND: National Natural Science Foundation of China, Chinese Academy of Sciences, Shanghai Committee of Science and Technology, and Shanghai Municipal Health Commission.

The functional logic of corticostriatal connections.

Unidirectional connections from the cortex to the matrix of the corpus striatum initiate the cortico-basal ganglia (BG)-thalamocortical loop, thought to be important in momentary action selection and in longer-term fine tuning of behavioural repertoire; a discrete set of striatal compartments, striosomes, has the complementary role of registering or anticipating reward that shapes corticostriatal plasticity. Re-entrant signals traversing the cortico-BG loop impact predominantly frontal cortices, conveyed through topographically ordered output channels; by contrast, striatal input signals originate from a far broader span of cortex, and are far more divergent in their termination. The term 'disclosed loop' is introduced to describe this organisation: a closed circuit that is open to outside influence at the initial stage of cortical input. The closed circuit component of corticostriatal afferents is newly dubbed 'operative', as it is proposed to establish the bid for action selection on the part of an incipient cortical action plan; the broader set of converging corticostriatal afferents is described as contextual. A corollary of this proposal is that every unit of the striatal volume, including the long, C-shaped tail of the caudate nucleus, should receive a mandatory component of operative input, and hence include at least one area of BG-recipient cortex amongst the sources of its corticostriatal afferents. Individual operative afferents contact twin classes of GABAergic striatal projection neuron (SPN), distinguished by their neurochemical character, and onward circuitry. This is the basis of the classic direct and indirect pathway model of the cortico-BG loop. Each pathway utilises a serial chain of inhibition, with two such links, or three, providing positive and negative feedback, respectively. Operative co-activation of direct and indirect SPNs is, therefore, pictured to simultaneously promote action, and to restrain it. The balance of this rival activity is determined by the contextual inputs, which summarise the external and internal sensory environment, and the state of ongoing behavioural priorities. Notably, the distributed sources of contextual convergence upon a striatal locus mirror the transcortical network harnessed by the origin of the operative input to that locus, thereby capturing a similar set of contingencies relevant to determining action. The disclosed loop formulation of corticostriatal and subsequent BG loop circuitry, as advanced here, refines the operating rationale of the classic model and allows the integration of more recent anatomical and physiological data, some of which can appear at variance with the classic model. Equally, it provides a lucid functional context for continuing cellular studies of SPN biophysics and mechanisms of synaptic plasticity.

Neural Elements for Predictive Coding.

Predictive coding theories of sensory brain function interpret the hierarchical construction of the cerebral cortex as a Bayesian, generative model capable of predicting the sensory data consistent with any given percept. Predictions are fed backward in the hierarchy and reciprocated by prediction error in the forward direction, acting to modify the representation of the outside world at increasing levels of abstraction, and so to optimize the nature of perception over a series of iterations. This accounts for many 'illusory' instances of perception where what is seen (heard, etc.) is unduly influenced by what is expected, based on past experience. This simple conception, the hierarchical exchange of prediction and prediction error, confronts a rich cortical microcircuitry that is yet to be fully documented. This article presents the view that, in the current state of theory and practice, it is profitable to begin a two-way exchange: that predictive coding theory can support an understanding of cortical microcircuit function, and prompt particular aspects of future investigation, whilst existing knowledge of microcircuitry can, in return, influence theoretical development. As an example, a neural inference arising from the earliest formulations of predictive coding is that the source populations of forward and backward pathways should be completely separate, given their functional distinction; this aspect of circuitry - that neurons with extrinsically bifurcating axons do not project in both directions - has only recently been confirmed. Here, the computational architecture prescribed by a generalized (free-energy) formulation of predictive coding is combined with the classic 'canonical microcircuit' and the laminar architecture of hierarchical extrinsic connectivity to produce a template schematic, that is further examined in the light of (a) updates in the microcircuitry of primate visual cortex, and (b) rapid technical advances made possible by transgenic neural engineering in the mouse. The exercise highlights a number of recurring themes, amongst them the consideration of interneuron diversity as a spur to theoretical development and the potential for specifying a pyramidal neuron's function by its individual 'connectome,' combining its extrinsic projection (forward, backward or subcortical) with evaluation of its intrinsic network (e.g., unidirectional versus bidirectional connections with other pyramidal neurons).

Cerebral hierarchies: predictive processing, precision and the pulvinar.

This paper considers neuronal architectures from a computational perspective and asks what aspects of neuroanatomy and neurophysiology can be disclosed by the nature of neuronal computations? In particular, we extend current formulations of the brain as an organ of inference--based upon hierarchical predictive coding--and consider how these inferences are orchestrated. In other words, what would the brain require to dynamically coordinate and contextualize its message passing to optimize its computational goals? The answer that emerges rests on the delicate (modulatory) gain control of neuronal populations that select and coordinate (prediction error) signals that ascend cortical hierarchies. This is important because it speaks to a hierarchical anatomy of extrinsic (between region) connections that form two distinct classes, namely a class of driving (first-order) connections that are concerned with encoding the content of neuronal representations and a class of modulatory (second-order) connections that establish context-in the form of the salience or precision ascribed to content. We explore the implications of this distinction from a formal perspective (using simulations of feature-ground segregation) and consider the neurobiological substrates of the ensuing precision-engineered dynamics, with a special focus on the pulvinar and attention.

Figure-ground modulation in awake primate thalamus.

Figure-ground discrimination refers to the perception of an object, the figure, against a nondescript background. Neural mechanisms of figure-ground detection have been associated with feedback interactions between higher centers and primary visual cortex and have been held to index the effect of global analysis on local feature encoding. Here, in recordings from visual thalamus of alert primates, we demonstrate a robust enhancement of neuronal firing when the figure, as opposed to the ground, component of a motion-defined figure-ground stimulus is located over the receptive field. In this paradigm, visual stimulation of the receptive field and its near environs is identical across both conditions, suggesting the response enhancement reflects higher integrative mechanisms. It thus appears that cortical activity generating the higher-order percept of the figure is simultaneously reentered into the lowest level that is anatomically possible (the thalamus), so that the signature of the evolving representation of the figure is imprinted on the input driving it in an iterative process.

Reflections on agranular architecture: predictive coding in the motor cortex.

The agranular architecture of motor cortex lacks a functional interpretation. Here, we consider a 'predictive coding' account of this unique feature based on asymmetries in hierarchical cortical connections. In sensory cortex, layer 4 (the granular layer) is the target of ascending pathways. We theorise that the operation of predictive coding in the motor system (a process termed 'active inference') provides a principled rationale for the apparent recession of the ascending pathway in motor cortex. The extension of this theory to interlaminar circuitry also accounts for a sub-class of 'mirror neuron' in motor cortex--whose activity is suppressed when observing an action--explaining how predictive coding can gate hierarchical processing to switch between perception and action.

Predictions not commands: active inference in the motor system.

The descending projections from motor cortex share many features with top-down or backward connections in visual cortex; for example, corticospinal projections originate in infragranular layers, are highly divergent and (along with descending cortico-cortical projections) target cells expressing NMDA receptors. This is somewhat paradoxical because backward modulatory characteristics would not be expected of driving motor command signals. We resolve this apparent paradox using a functional characterisation of the motor system based on Helmholtz's ideas about perception; namely, that perception is inference on the causes of visual sensations. We explain behaviour in terms of inference on the causes of proprioceptive sensations. This explanation appeals to active inference, in which higher cortical levels send descending proprioceptive predictions, rather than motor commands. This process mirrors perceptual inference in sensory cortex, where descending connections convey predictions, while ascending connections convey prediction errors. The anatomical substrate of this recurrent message passing is a hierarchical system consisting of functionally asymmetric driving (ascending) and modulatory (descending) connections: an arrangement that we show is almost exactly recapitulated in the motor system, in terms of its laminar, topographic and physiological characteristics. This perspective casts classical motor reflexes as minimising prediction errors and may provide a principled explanation for why motor cortex is agranular.

Differential feedback modulation of center and surround mechanisms in parvocellular cells in the visual thalamus.

Many cells in both the central visual system and other sensory systems exhibit a center surround organization in their receptive field, where the response to a centrally placed stimulus is modified when a surrounding area is also stimulated. This can follow from laterally directed connections in the local circuit at the level of the cell in question but could also involve more complex interactions. In the lateral geniculate nucleus (LGN), the cells relaying the retinal input display a concentric, center surround organization that in part follows from the similar organization characterizing the retinal cells providing their input. However, local thalamic inhibitory interneurons also play a role, and as we examine here, feedback from the visual cortex too. Here, we show in the primate (macaque) that spatially organized cortical feedback provides a clear and differential influence serving to enhance both responses to stimulation within the center of the receptive field and the ability of the nonclassical surround mechanism to attenuate this. In short, both center and surround mechanisms are influenced by the feedback. This dynamically sharpens the spatial focus of the receptive field and introduces nonlinearities from the cortical mechanism into the LGN.

Interhemispheric integration in visual search.

The search task of Luck, Hillyard, Mangun and Gazzaniga (1989) was optimised to test for the presence of a bilateral field advantage in the visual search capabilities of normal subjects. The modified design used geometrically regular arrays of 2, 4 or 8 items restricted to hemifields delineated by the vertical or horizontal meridian; the target, if present, appeared at one of two fixed positions per quadrant at an eccentricity of 11 deg. Group and individual performance data were analysed in terms of the slope of response time against display-size functions ('RT slope'). Averaging performance across all conditions save display mode (bilateral vs. unilateral) revealed a significant bilateral advantage in the form of a 21% increase in apparent item scanning speed for target detection; in the absence of a target, bilateral displays gave a 5% increase in speed that was not significant. Factor analysis by ANOVA confirmed this main effect of display mode, and also revealed several higher order interactions with display geometry, indicating that the bilateral advantage was masked at certain target positions by a crowding-like effect. In a numerical model of search efficiency (i.e. RT slope), bilateral advantage was parameterised by an interhemispheric 'transfer factor' (T) that governs the strength of the ipsilateral representation of distractors, and modifies the level of intrahemispheric competition with the target. The factor T was found to be higher in superior field than inferior field; this result held for the modelled data of each individual subject, as well as the group, representing a uniform tendency for the bilateral advantage to be more prominent in inferior field. In fact statistical analysis and modelling of search efficiency showed that the geometrical display factors (target polar and quadrantic location, and associated crowding effects) were all remarkably consistent across subjects. Greater variability was inferred within a fixed, decisional component of response time, with individual subjects capable of opposite hemifield biases. The results are interpretable by a guided search model of spatial attention - a first, parallel stage guiding selection by a second, serial stage - with the proviso that the first stage is relatively insular within each hemisphere. The bilateral advantage in search efficiency can then be attributed to a relative gain in target weight within the initial parallel stage, owing to a reduction in distractor competition mediated specifically by intrahemispheric circuitry. In the absence of a target there is no effective guidance, and hence no basis for a bilateral advantage to enhance search efficiency; the equivalence of scanning speed for the two display modes (bilateral and unilateral) implies a unitary second-stage process mediated via efficient interhemispheric integration.

Spatially distributed encoding of covert attentional shifts in human thalamus.

Spatial attention modulates signal processing within visual nuclei of the thalamus--but do other nuclei govern the locus of attention in top-down mode? We examined functional MRI (fMRI) data from three subjects performing a task requiring covert attention to 1 of 16 positions in a circular array. Target position was cued after stimulus offset, requiring subjects to perform target detection from iconic visual memory. We found positionally specific responses at multiple thalamic sites, with individual voxels activating at more than one direction of attentional shift. Voxel clusters at anatomically equivalent sites across subjects revealed a broad range of directional tuning at each site, with little sign of contralateral bias. By reference to a thalamic atlas, we identified the nuclear correspondence of the four most reliably activated sites across subjects: mediodorsal/central-intralaminar (oculomotor thalamus), caudal intralaminar/parafascicular, suprageniculate/limitans, and medial pulvinar/lateral posterior. Hence, the cortical network generating a top-down control signal for relocating attention acts in concert with a spatially selective thalamic apparatus-the set of active nuclei mirroring the thalamic territory of cortical "eye-field" areas, thus supporting theories which propose the visuomotor origins of covert attentional selection.

Feature binding in the feedback layers of area V2.

The visual features of an object are processed by multiple, functionally specialized areas of cerebral cortex. When several objects are seen simultaneously, what mechanism preserves the association of features that belong to a single item? We address this question-known as the "binding problem"-by examining combinatorial feature selectivity of neurons in area V2. In recording from anesthetized macaques, we estimate that dual selectivity for chromatic and spatiotemporal attributes is 50% more common (27% vs. 18% sampling frequency) in superficial and deep layer neurons receiving feedback connections from higher areas, compared with layer 4-3 neurons relaying ascending signals. The operation of feedback pathways is thought to mediate attentional modulation of activity that may achieve binding through acting to select one single object for higher representation and filtering out competing objects. We propose that dual-selective neurons perform a "bridging" function, mediating the transfer of feedback-induced bias between feature dimensions. The bias can be propagated through V2 output neurons (of unitary selectivity) to higher levels of specialized processing and so promote selection of the target object's representation among multiple feature maps. The bridging function would thus act to unify the outcome of parallel, object-selective processes taking place along segregated visual pathways.

Motion processing: where the medium is the message.

The primate retina serves up three channels for visual entertainment, of which just one is used for the primary analysis of motion. A prominent, unique class of neuron has a dominant role in transmission from cortical area V1.

The importance of being agranular: a comparative account of visual and motor cortex.

The agranular cortex is an important landmark-anatomically, as the architectural flag of mammalian motor cortex, and historically, as a spur to the development of theories of localization of function. But why, exactly, do agranularity and motor function go together? To address this question, it should be noted that not only does motor cortex lack granular layer four, it also has a relatively thinner layer three. Therefore, it is the two layers which principally constitute the ascending pathways through the sensory (granular) cortex that have regressed in motor cortex: simply stated, motor cortex does not engage in serial reprocessing of incoming sensory data. But why should a granular architecture not be demanded by the downstream relay of motor instructions through the motor cortex? The scant anatomical evidence available regarding laminar patterns suggests that the pathways from frontal and premotor areas to the primary motor cortex actually bear a greater resemblance to the descending, or feedback connections of sensory cortex that avoid the granular layer. The action of feedback connections is generally described as "modulatory" at a cellular level, or "selective" in terms of systems analysis. By contrast, ascending connections may be labelled "driving" or "instructive". Where the motor cortex uses driving inputs, they are most readily identified as sensory signals instructing the visual location of targets and the kinaesthetic state of the body. Visual signals may activate motor concepts, e.g. "mirror neurons", and the motor plan must select the appropriate muscles and forces to put the plan into action, if the decision to move is taken. This, perhaps, is why "driving" motor signals might be inappropriate-the optimal selection and its execution are conditional upon both kinaesthetic and motivational factors. The argument, summarized above, is constructed in honour of Korbinian Brodmann's centenary, and follows two of the fundamental principles of his school of thought: that uniformities in cortical structure, and development imply global conservation of some aspects of function, whereas regional variations in architecture can be used to chart the "organs" of the cortex, and perhaps to understand their functional differences.

The brain circuitry of attention.

From an operational perspective, attention is a matter of organizing multiple brain centres to act in concert on the task at hand. Taking focal visual attention as an example, recent anatomical findings suggest that the pulvinar might act as a remote hub for coordinating spatial activity within multiple cortical visual maps. The pulvinar can, in turn, be influenced by signals originating in the frontal and parietal eye fields, using common visuomotor neural circuitry, with the superior colliculus acting as an important link. By identifying a complex, real neural architecture ('RNA') model for attention, it is possible to integrate several different modes of operation - such as parallel or serial, bottom-up or top-down, preattentive or attentive - that characterize conflicting cognitive models of attention in visual search paradigms.

The functional organization of area V2, I: specialization across stripes and layers.

We used qualitative tests to assess the sensitivity of 1043 V2 neurons (predominantly multiunits) in anesthetised macaque monkeys to direction, length, orientation, and color of moving bar stimuli. Spectral sensitivity was additionally tested by noting ON or OFF responses to flashed stimuli of varied size and color. The location of 649 units was identified with respect to cycles of cytochrome oxidase stripes (thick-inter-thin-inter) and cortical layer. We used an initial 8-way stripe classification (4 stripes, and 4 "marginal" zones at interstripes boundaries), and a 9-way layer classification (5 standard layers (2-6), and 4 "marginal" strata at layer boundaries). These classes were collapsed differently for particular analyses of functional distribution; the main stripe-by-layer analysis was performed on 18 compartments (3 stripes x 6 layers). We found direction sensitivity only within thick stripes, orientation sensitivity mainly in thick stripes and interstripes, and spectral sensitivity mainly in thin stripes. Positive length summation was relatively more frequent in thick stripes and interstripes, and negative length/size summation in thin stripes. All these "majority" characteristics of stripes were most prominent in layers 3A and 3B. By contrast, "minority" characteristics (e.g. spectral sensitivity in thick stripes; positive size summation in thin stripes) tended to be most frequent in the outer layers, that is, layers 2 and 6. In consequence, going by the four functions tested, the distinctions between stripes were maximal in layer 3, moderate in layer 2, and minimal in layer 6. Pooling all layers, there was some indication of asymmetry in the stripe cycle, in that thin stripe characteristics (spectral sensitivity, orientation insensitivity, and negative size summation) were also evident in the marginal zone and interstripe immediately lateral to a thin stripe, but less so medially. Within thin stripes, spectral and orientation selectivities were negatively correlated; this was still more accentuated amongst the minority spectrally tuned cells of thick stripes, but absent from interstripes, where these two properties were randomly assorted. Directional and spectral sensitivities were each coupled to negative size summation, but not to each other. We conclude that these functional characteristics of stripes are consistent with segregated, specialized pathways ascending through their middle layers, whilst the outer layers, 1, 2, and 6, utilize feedback from higher areas to adopt a more integrative role.