A unifying framework for functional organization in early and higher ventral visual cortex.

A key feature of cortical systems is functional organization: the arrangement of functionally distinct neurons in characteristic spatial patterns. However, the principles underlying the emergence of functional organization in the cortex are poorly understood. Here, we develop the topographic deep artificial neural network (TDANN), the first model to predict several aspects of the functional organization of multiple cortical areas in the primate visual system. We analyze the factors driving the TDANN's success and find that it balances two objectives: learning a task-general sensory representation and maximizing the spatial smoothness of responses according to a metric that scales with cortical surface area. In turn, the representations learned by the TDANN are more brain-like than in spatially unconstrained models. Finally, we provide evidence that the TDANN's functional organization balances performance with between-area connection length. Our results offer a unified principle for understanding the functional organization of the primate ventral visual system.

Characterizing Spatiotemporal Population Receptive Fields in Human Visual Cortex with fMRI.

The use of fMRI and computational modeling has advanced understanding of spatial characteristics of population receptive fields (pRFs) in human visual cortex. However, we know relatively little about the spatiotemporal characteristics of pRFs because neurons' temporal properties are one to two orders of magnitude faster than fMRI BOLD responses. Here, we developed an image-computable framework to estimate spatiotemporal pRFs from fMRI data. First, we developed a simulation software that predicts fMRI responses to a time-varying visual input given a spatiotemporal pRF model and solves the model parameters. The simulator revealed that ground-truth spatiotemporal parameters can be accurately recovered at the millisecond resolution from synthesized fMRI responses. Then, using fMRI and a novel stimulus paradigm, we mapped spatiotemporal pRFs in individual voxels across human visual cortex in 10 participants (both females and males). We find that a compressive spatiotemporal (CST) pRF model better explains fMRI responses than a conventional spatial pRF model across visual areas spanning the dorsal, lateral, and ventral streams. Further, we find three organizational principles of spatiotemporal pRFs: (1) from early to later areas within a visual stream, spatial and temporal windows of pRFs progressively increase in size and show greater compressive nonlinearities, (2) later visual areas show diverging spatial and temporal windows across streams, and (3) within early visual areas (V1-V3), both spatial and temporal windows systematically increase with eccentricity. Together, this computational framework and empirical results open exciting new possibilities for modeling and measuring fine-grained spatiotemporal dynamics of neural responses using fMRI.

Rethinking cortical recycling in ventral temporal cortex.

High-level visual areas in ventral temporal cortex (VTC) support recognition of important categories, such as faces and words. Word-selective regions are left lateralized and emerge at the onset of reading instruction. Face-selective regions are right lateralized and have been documented in infancy. Prevailing theories suggest that face-selective regions become right lateralized due to competition with word-selective regions in the left hemisphere. However, recent longitudinal studies examining face- and word-selective responses in childhood do not provide support for this theory. Instead, there is evidence that word representations recycle cortex previously involved in processing other stimuli, such as limbs. These findings call for more longitudinal investigations of cortical recycling and a new era of work that links visual experience and behavior with neural responses.

Both mOTS-words and pOTS-words prefer emoji stimuli over text stimuli during a reading task.

The visual word form area in the occipitotemporal sulcus (OTS), here referred to as OTS-words, responds more strongly to text than other visual stimuli and is crucial for reading. We hypothesized, that this text preference may be driven by a preference for reading tasks, as in most prior fMRI studies only the text stimuli were readable. Hence, we performed three fMRI experiments (N=15) and systematically varied the participant's task and the stimulus, investigating mOTS-words and pOTS-words subregions. In experiment 1, we contrasted text stimuli with non-readable visual stimuli (faces, limbs, houses, objects). Experiment 2 utilized an fMRI adaptation paradigm, presenting compound words in text or emoji formats. In experiment 3, participants performed a reading or a color task on compound words in text or emoji format. Using experiment 1 data, we identified mOTS-words and pOTS-words by contrasting texts with non-readable stimuli. In experiment 2, pOTS-words, but not mOTS-words, showed fMRI adaptation for compound words in both text and emoji formats. In experiment 3, surprisingly, both subregions showed higher responses to compound words in emoji than text format. Moreover, mOTS-words showed higher responses during the reading than the color task and a task-stimulus interaction. Multivariate analyses revealed that distributed responses in pOTS-words encode the visual stimulus, while responses in mOTS-words encode both stimulus and task. Together, our findings suggest that the function of the OTS-words subregions goes beyond the specific visual processing of text and that these regions are flexibly recruited whenever semantic meaning needs to be assigned to visual input.

High-resolution myelin-water fraction and quantitative relaxation mapping using 3D ViSTa-MR fingerprinting.

PURPOSE: This study aims to develop a high-resolution whole-brain multi-parametric quantitative MRI approach for simultaneous mapping of myelin-water fraction (MWF), T1, T2, and proton-density (PD), all within a clinically feasible scan time. METHODS: We developed 3D visualization of short transverse relaxation time component (ViSTa)-MRF, which combined ViSTa technique with MR fingerprinting (MRF), to achieve high-fidelity whole-brain MWF and T1/T2/PD mapping on a clinical 3T scanner. To achieve fast acquisition and memory-efficient reconstruction, the ViSTa-MRF sequence leverages an optimized 3D tiny-golden-angle-shuffling spiral-projection acquisition and joint spatial-temporal subspace reconstruction with optimized preconditioning algorithm. With the proposed ViSTa-MRF approach, high-fidelity direct MWF mapping was achieved without a need for multicompartment fitting that could introduce bias and/or noise from additional assumptions or priors. RESULTS: The in vivo results demonstrate the effectiveness of the proposed acquisition and reconstruction framework to provide fast multi-parametric mapping with high SNR and good quality. The in vivo results of 1 mm- and 0.66 mm-isotropic resolution datasets indicate that the MWF values measured by the proposed method are consistent with standard ViSTa results that are 30× slower with lower SNR. Furthermore, we applied the proposed method to enable 5-min whole-brain 1 mm-iso assessment of MWF and T1/T2/PD mappings for infant brain development and for post-mortem brain samples. CONCLUSIONS: In this work, we have developed a 3D ViSTa-MRF technique that enables the acquisition of whole-brain MWF, quantitative T1, T2, and PD maps at 1 and 0.66 mm isotropic resolution in 5 and 15 min, respectively. This advancement allows for quantitative investigations of myelination changes in the brain.

Longitudinal development of category representations in ventral temporal cortex predicts word and face recognition.

Regions in ventral temporal cortex that are involved in visual recognition of categories like words and faces undergo differential development during childhood. However, categories are also represented in distributed responses across high-level visual cortex. How distributed category representations develop and if this development relates to behavioral changes in recognition remains largely unknown. Here, we used functional magnetic resonance imaging to longitudinally measure the development of distributed responses across ventral temporal cortex to 10 categories in school-age children over several years. Our results reveal both strengthening and weakening of category representations with age, which was mainly driven by changes across category-selective voxels. Representations became particularly more distinct for words in the left hemisphere and for faces bilaterally. Critically, distinctiveness for words and faces across category-selective voxels in left and right lateral ventral temporal cortex, respectively, predicted individual children's word and face recognition performance. These results suggest that the development of distributed representations in ventral temporal cortex has behavioral ramifications and advance our understanding of prolonged cortical development during childhood.

Human white matter myelinates faster in utero than ex utero.

The formation of myelin, the fatty sheath that insulates nerve fibers, is critical for healthy brain function. A fundamental open question is what impact being born has on myelin growth. To address this, we evaluated a large (n = 300) cross-sectional sample of newborns from the Developing Human Connectome Project (dHCP). First, we developed software for the automated identification of 20 white matter bundles in individual newborns that is well suited for large samples. Next, we fit linear models that quantify how T1w/T2w (a myelin-sensitive imaging contrast) changes over time at each point along the bundles. We found faster growth of T1w/T2w along the lengths of all bundles before birth than right after birth. Further, in a separate longitudinal sample of preterm infants (N = 34), we found lower T1w/T2w than in full-term peers measured at the same age. By applying the linear models fit on the cross-section sample to the longitudinal sample of preterm infants, we find that their delay in T1w/T2w growth is well explained by the amount of time they spent developing in utero and ex utero. These results suggest that white matter myelinates faster in utero than ex utero. The reduced rate of myelin growth after birth, in turn, explains lower myelin content in individuals born preterm and could account for long-term cognitive, neurological, and developmental consequences of preterm birth. We hypothesize that closely matching the environment of infants born preterm to what they would have experienced in the womb may reduce delays in myelin growth and hence improve developmental outcomes.

Rethinking simultaneous suppression in visual cortex via compressive spatiotemporal population receptive fields.

When multiple visual stimuli are presented simultaneously in the receptive field, the neural response is suppressed compared to presenting the same stimuli sequentially. The prevailing hypothesis suggests that this suppression is due to competition among multiple stimuli for limited resources within receptive fields, governed by task demands. However, it is unknown how stimulus-driven computations may give rise to simultaneous suppression. Using fMRI, we find simultaneous suppression in single voxels, which varies with both stimulus size and timing, and progressively increases up the visual hierarchy. Using population receptive field (pRF) models, we find that compressive spatiotemporal summation rather than compressive spatial summation predicts simultaneous suppression, and that increased simultaneous suppression is linked to larger pRF sizes and stronger compressive nonlinearities. These results necessitate a rethinking of simultaneous suppression as the outcome of stimulus-driven compressive spatiotemporal computations within pRFs, and open new opportunities to study visual processing capacity across space and time.

A Unifying Principle for the Functional Organization of Visual Cortex.

A key feature of many cortical systems is functional organization: the arrangement of neurons with specific functional properties in characteristic spatial patterns across the cortical surface. However, the principles underlying the emergence and utility of functional organization are poorly understood. Here we develop the Topographic Deep Artificial Neural Network (TDANN), the first unified model to accurately predict the functional organization of multiple cortical areas in the primate visual system. We analyze the key factors responsible for the TDANN's success and find that it strikes a balance between two specific objectives: achieving a task-general sensory representation that is self-supervised, and maximizing the smoothness of responses across the cortical sheet according to a metric that scales relative to cortical surface area. In turn, the representations learned by the TDANN are lower dimensional and more brain-like than those in models that lack a spatial smoothness constraint. Finally, we provide evidence that the TDANN's functional organization balances performance with inter-area connection length, and use the resulting models for a proof-of-principle optimization of cortical prosthetic design. Our results thus offer a unified principle for understanding functional organization and a novel view of the functional role of the visual system in particular.

Characterizing spatiotemporal population receptive fields in human visual cortex with fMRI.

The use of fMRI and computational modeling has advanced understanding of spatial characteristics of population receptive fields (pRFs) in human visual cortex. However, we know relatively little about the spatiotemporal characteristics of pRFs because neurons' temporal properties are one to two orders of magnitude faster than fMRI BOLD responses. Here, we developed an image-computable framework to estimate spatiotemporal pRFs from fMRI data. First, we developed a simulation software that predicts fMRI responses to a time varying visual input given a spatiotemporal pRF model and solves the model parameters. The simulator revealed that ground-truth spatiotemporal parameters can be accurately recovered at the millisecond resolution from synthesized fMRI responses. Then, using fMRI and a novel stimulus paradigm, we mapped spatiotemporal pRFs in individual voxels across human visual cortex in 10 participants. We find that a compressive spatiotemporal (CST) pRF model better explains fMRI responses than a conventional spatial pRF model across visual areas spanning the dorsal, lateral, and ventral streams. Further, we find three organizational principles of spatiotemporal pRFs: (i) from early to later areas within a visual stream, spatial and temporal integration windows of pRFs progressively increase in size and show greater compressive nonlinearities, (ii) later visual areas show diverging spatial and temporal integration windows across streams, and (iii) within early visual areas (V1-V3), both spatial and temporal integration windows systematically increase with eccentricity. Together, this computational framework and empirical results open exciting new possibilities for modeling and measuring fine-grained spatiotemporal dynamics of neural responses in the human brain using fMRI.

Comparing retinotopic maps of children and adults reveals a late-stage change in how V1 samples the visual field.

Adult visual performance differs with angular location -it is better for stimuli along the horizontal than vertical, and lower than upper vertical meridian of the visual field. These perceptual asymmetries are paralleled by asymmetries in cortical surface area in primary visual cortex (V1). Children, unlike adults, have similar visual performance at the lower and upper vertical meridian. Do children have similar V1 surface area representing the upper and lower vertical meridian? Using MRI, we measure the surface area of retinotopic maps (V1-V3) in children and adults. Many features of the maps are similar between groups, including greater V1 surface area for the horizontal than vertical meridian. However, unlike adults, children have a similar amount of V1 surface area representing the lower and upper vertical meridian. These data reveal a late-stage change in V1 organization that may relate to the emergence of the visual performance asymmetry along the vertical meridian by adulthood.

White matter connections of high-level visual areas predict cytoarchitecture better than category-selectivity in childhood, but not adulthood.

Ventral temporal cortex (VTC) consists of high-level visual regions that are arranged in consistent anatomical locations across individuals. This consistency has led to several hypotheses about the factors that constrain the functional organization of VTC. A prevailing theory is that white matter connections influence the organization of VTC, however, the nature of this constraint is unclear. Here, we test 2 hypotheses: (1) white matter tracts are specific for each category or (2) white matter tracts are specific to cytoarchitectonic areas of VTC. To test these hypotheses, we used diffusion magnetic resonance imaging to identify white matter tracts and functional magnetic resonance imaging to identify category-selective regions in VTC in children and adults. We find that in childhood, white matter connections are linked to cytoarchitecture rather than category-selectivity. In adulthood, however, white matter connections are linked to both cytoarchitecture and category-selectivity. These results suggest a rethinking of the view that category-selective regions in VTC have category-specific white matter connections early in development. Instead, these findings suggest that the neural hardware underlying the processing of categorical stimuli may be more domain-general than previously thought, particularly in childhood.

High-resolution myelin-water fraction and quantitative relaxation mapping using 3D ViSTa-MR fingerprinting.

Purpose: This study aims to develop a high-resolution whole-brain multi-parametric quantitative MRI approach for simultaneous mapping of myelin-water fraction (MWF), T1, T2, and proton-density (PD), all within a clinically feasible scan time. Methods: We developed 3D ViSTa-MRF, which combined Visualization of Short Transverse relaxation time component (ViSTa) technique with MR Fingerprinting (MRF), to achieve high-fidelity whole-brain MWF and T1/T2/PD mapping on a clinical 3T scanner. To achieve fast acquisition and memory-efficient reconstruction, the ViSTa-MRF sequence leverages an optimized 3D tiny-golden-angle-shuffling spiral-projection acquisition and joint spatial-temporal subspace reconstruction with optimized preconditioning algorithm. With the proposed ViSTa-MRF approach, high-fidelity direct MWF mapping was achieved without a need for multi-compartment fitting that could introduce bias and/or noise from additional assumptions or priors. Results: The in-vivo results demonstrate the effectiveness of the proposed acquisition and reconstruction framework to provide fast multi-parametric mapping with high SNR and good quality. The in-vivo results of 1mm- and 0.66mm-iso datasets indicate that the MWF values measured by the proposed method are consistent with standard ViSTa results that are 30x slower with lower SNR. Furthermore, we applied the proposed method to enable 5-minute whole-brain 1mm-iso assessment of MWF and T1/T2/PD mappings for infant brain development and for post-mortem brain samples. Conclusions: In this work, we have developed a 3D ViSTa-MRF technique that enables the acquisition of whole-brain MWF, quantitative T1, T2, and PD maps at 1mm and 0.66mm isotropic resolution in 5 and 15 minutes, respectively. This advancement allows for quantitative investigations of myelination changes in the brain.

White matter myelination during early infancy is linked to spatial gradients and myelin content at birth.

Development of myelin, a fatty sheath that insulates nerve fibers, is critical for brain function. Myelination during infancy has been studied with histology, but postmortem data cannot evaluate the longitudinal trajectory of white matter development. Here, we obtained longitudinal diffusion MRI and quantitative MRI measures of longitudinal relaxation rate (R1) of white matter in 0, 3 and 6 months-old human infants, and developed an automated method to identify white matter bundles and quantify their properties in each infant's brain. We find that R1 increases from newborns to 6-months-olds in all bundles. R1 development is nonuniform: there is faster development in white matter that is less mature in newborns, and development rate increases along inferior-to-superior as well as anterior-to-posterior spatial gradients. As R1 is linearly related to myelin fraction in white matter bundles, these findings open new avenues to elucidate typical and atypical white matter myelination in early infancy.

Attention enhances category representations across the brain with strengthened residual correlations to ventral temporal cortex.

How does attention enhance neural representations of goal-relevant stimuli while suppressing representations of ignored stimuli across regions of the brain? While prior studies have shown that attention enhances visual responses, we lack a cohesive understanding of how selective attention modulates visual representations across the brain. Here, we used functional magnetic resonance imaging (fMRI) while participants performed a selective attention task on superimposed stimuli from multiple categories and used a data-driven approach to test how attention affects both decodability of category information and residual correlations (after regressing out stimulus-driven variance) with category-selective regions of ventral temporal cortex (VTC). Our data reveal three main findings. First, when two objects are simultaneously viewed, the category of the attended object can be decoded more readily than the category of the ignored object, with the greatest attentional enhancements observed in occipital and temporal lobes. Second, after accounting for the response to the stimulus, the correlation in the residual brain activity between a cortical region and a category-selective region of VTC was elevated when that region's preferred category was attended vs. ignored, and more so in the right occipital, parietal, and frontal cortices. Third, we found that the stronger the residual correlations between a given region of cortex and VTC, the better visual category information could be decoded from that region. These findings suggest that heightened residual correlations by selective attention may reflect the sharing of information between sensory regions and higher-order cortical regions to provide attentional enhancement of goal-relevant information.

Establishing the functional relevancy of white matter connections in the visual system and beyond.

For over a century, researchers have examined the functional relevancy of white matter bundles. Consequently, many large-scale bundles spanning several centimeters have been associated in their entirety with specific brain functions, such as language or attention. However, these coarse structural-functional relationships are at odds with modern understanding of the fine-grained functional organization of human cortex, such as the mosaic of category-selective regions in ventral temporal cortex. Here, we review a multimodal approach that combines fMRI to define functional regions of interest within individual's brains with dMRI tractography to identify the white matter bundles of the same individual. Combining these data allows to determine which subsets of streamlines within a white matter bundle connect to specific functional regions in each individual. That is, this approach identifies the functionally defined white matter sub-bundles of the brain. We argue that this approach not only enhances the accuracy of interpreting the functional relevancy of white matter bundles, but also enables segmentation of these large-scale bundles into meaningful functional units, which can then be linked to behavior with enhanced precision. Importantly, this approach has the potential for making new discoveries of the fine-grained functional relevancy of white matter connections in the visual system and the brain more broadly, akin to the flurry of research that has identified functional regions in cortex.

Infants' cortex undergoes microstructural growth coupled with myelination during development.

Development of cortical tissue during infancy is critical for the emergence of typical brain functions in cortex. However, how cortical microstructure develops during infancy remains unknown. We measured the longitudinal development of cortex from birth  to six months of age  using multimodal quantitative imaging of cortical microstructure. Here we show that infants' cortex undergoes profound microstructural tissue growth during the first six months of human life. Comparison of postnatal to prenatal transcriptomic gene expression data demonstrates that myelination and synaptic processes are dominant contributors to this postnatal microstructural tissue growth. Using visual cortex as a model system, we find hierarchical microstructural growth: higher-level visual areas have less mature tissue at birth than earlier visual areas but grow at faster rates. This overturns the prominent view that visual areas that are most mature at birth develop fastest. Together, in vivo, longitudinal, and quantitative measurements, which we validated with ex vivo transcriptomic data, shed light on the rate, sequence, and biological mechanisms of developing cortical systems during early infancy. Importantly, our findings propose a hypothesis that cortical myelination is a key factor in cortical development during early infancy, which has important implications for diagnosis of neurodevelopmental disorders and delays in infants.

Holistic face recognition is an emergent phenomenon of spatial processing in face-selective regions.

Spatial processing by receptive fields is a core property of the visual system. However, it is unknown how spatial processing in high-level regions contributes to recognition behavior. As face inversion is thought to disrupt typical holistic processing of information in faces, we mapped population receptive fields (pRFs) with upright and inverted faces in the human visual system. Here we show that in face-selective regions, but not primary visual cortex, pRFs and overall visual field coverage are smaller and shifted downward in response to face inversion. From these measurements, we successfully predict the relative behavioral detriment of face inversion at different positions in the visual field. This correspondence between neural measurements and behavior demonstrates how spatial processing in face-selective regions may enable holistic perception. These results not only show that spatial processing in high-level visual regions is dynamically used towards recognition, but also suggest a powerful approach for bridging neural computations by receptive fields to behavior.

Cortical recycling in high-level visual cortex during childhood development.

Human ventral temporal cortex contains category-selective regions that respond preferentially to ecologically relevant categories such as faces, bodies, places and words and that are causally involved in the perception of these categories. How do these regions develop during childhood? We used functional magnetic resonance imaging to measure longitudinal development of category selectivity in school-age children over 1 to 5 years. We discovered that, from young childhood to the teens, face- and word-selective regions in ventral temporal cortex expand and become more category selective, but limb-selective regions shrink and lose their preference for limbs. Critically, as a child develops, increases in face and word selectivity are directly linked to decreases in limb selectivity, revealing that during childhood, limb selectivity in ventral temporal cortex is repurposed into word and face selectivity. These data provide evidence for cortical recycling during childhood development. This has important implications for understanding typical as well as atypical brain development and necessitates a rethinking of how cortical function develops during childhood.

Differential spatial computations in ventral and lateral face-selective regions are scaffolded by structural connections.

Face-processing occurs across ventral and lateral visual streams, which are involved in static and dynamic face perception, respectively. However, the nature of spatial computations across streams is unknown. Using functional MRI and population receptive field (pRF) mapping, we measured pRFs in face-selective regions. Results reveal that spatial computations by pRFs in ventral face-selective regions are concentrated around the center of gaze (fovea), but spatial computations in lateral face-selective regions extend peripherally. Diffusion MRI reveals that these differences are mirrored by a preponderance of white matter connections between ventral face-selective regions and foveal early visual cortex (EVC), while connections with lateral regions are distributed more uniformly across EVC eccentricities. These findings suggest a rethinking of spatial computations in face-selective regions, showing that they vary across ventral and lateral streams, and further propose that spatial computations in high-level regions are scaffolded by the fine-grain pattern of white matter connections from EVC.