Visual word processing engages a hierarchical, distributed, and bilateral cortical network.

Although the Visual Word Form Area (VWFA) in left temporal cortex is considered the pre-eminent region in visual word processing, other regions are also implicated. We examined the entire text-selective circuit, using functional MRI. Ten regions of interest (ROIs) per hemisphere were defined, which, based on clustering, grouped into early vision, high-level vision, and language clusters. We analyzed the responses of the ROIs and clusters to words, inverted words, and consonant strings using univariate, multivariate, and functional connectivity measures. Bilateral modulation by stimulus condition was evident, with a stronger effect in left hemisphere regions. Last, using graph theory, we observed that the VWFA was equivalently connected with early visual and language clusters in both hemispheres, reflecting its role as a mediator in the circuit. Although the individual ROIs and clusters bilaterally were flexibly altered by the nature of the input, stability held at the level of global circuit connectivity, reflecting the complex hierarchical distributed system serving visual text perception.

From decomposition to distributed theories of morphological processing in reading.

The morphological structure of complex words impacts how they are processed during visual word recognition. This impact varies over the course of reading acquisition and for different languages and writing systems. Many theories of morphological processing rely on a decomposition mechanism, in which words are decomposed into explicit representations of their constituent morphemes. In distributed accounts, in contrast, morphological sensitivity arises from the tuning of finer-grained representations to useful statistical regularities in the form-to-meaning mapping, without the need for explicit morpheme representations. In this theoretically guided review, we summarize research into the mechanisms of morphological processing, and discuss findings within the context of decomposition and distributed accounts. Although many findings fit within a decomposition model of morphological processing, we suggest that the full range of results is more naturally explained by a distributed approach, and discuss additional benefits of adopting this perspective.

A neural network model of the effect of prior experience with regularities on subsequent category learning.

Categories are often structured by the similarities of instances within the category defined across dimensions or features. Researchers typically assume that there is a direct, linear relationship between the physical input dimensions across which category exemplars are defined and the psychological representation of these dimensions. However, this assumption is not always warranted. Through a set of simulations, we demonstrate that the psychological representations of input dimensions developed through long-term prior experience can place very strong constraints on category learning. We compare the model's behavior to auditory, visual, and cross-modal human category learning and make conclusions regarding the nature of the psychological representations of the dimensions in those studies. These simulations support the conclusion that the nature of psychological representations of input dimensions is a critical aspect to understanding the mechanisms underlying category learning.

A connectivity-constrained computational account of topographic organization in primate high-level visual cortex.

Inferotemporal (IT) cortex in humans and other primates is topographically organized, containing multiple hierarchically organized areas selective for particular domains, such as faces and scenes. This organization is commonly viewed in terms of evolved domain-specific visual mechanisms. Here, we develop an alternative, domain-general and developmental account of IT cortical organization. The account is instantiated in interactive topographic networks (ITNs), a class of computational models in which a hierarchy of model IT areas, subject to biologically plausible connectivity-based constraints, learns high-level visual representations optimized for multiple domains. We find that minimizing a wiring cost on spatially organized feedforward and lateral connections, alongside realistic constraints on the sign of neuronal connectivity within model IT, results in a hierarchical, topographic organization. This organization replicates a number of key properties of primate IT cortex, including the presence of domain-selective spatial clusters preferentially involved in the representation of faces, objects, and scenes; columnar responses across separate excitatory and inhibitory units; and generic spatial organization whereby the response correlation of pairs of units falls off with their distance. We thus argue that topographic domain selectivity is an emergent property of a visual system optimized to maximize behavioral performance under generic connectivity-based constraints.

Deep learning of shared perceptual representations for familiar and unfamiliar faces: Reply to commentaries.

We recently argued that human unfamiliar face identity perception reflects substantial perceptual expertise, and that the advantage for familiar over unfamiliar face identity matching reflects a learned mapping between generic high-level perceptual features and a unique identity representation of each individual (Blauch, Behrmann and Plaut, 2020). Here we respond to two commentaries by Young and Burton (2020) and Yovel and Abudarham (2020), clarifying and elaborating our stance on various theoretical issues, and discussing topics for future research in human face recognition and the learning of perceptual representations.

Computational insights into human perceptual expertise for familiar and unfamiliar face recognition.

Humans are generally thought to be experts at face recognition, and yet identity perception for unfamiliar faces is surprisingly poor compared to that for familiar faces. Prior theoretical work has argued that unfamiliar face identity perception suffers because the majority of identity-invariant visual variability is idiosyncratic to each identity, and thus, each face identity must be learned essentially from scratch. Using a high-performing deep convolutional neural network, we evaluate this claim by examining the effects of visual experience in untrained, object-expert and face-expert networks. We found that only face training led to substantial generalization in an identity verification task of novel unfamiliar identities. Moreover, generalization increased with the number of previously learned identities, highlighting the generality of identity-invariant information in face images. To better understand how familiarity builds upon generic face representations, we simulated familiarization with face identities by fine-tuning the network on images of the previously unfamiliar identities. Familiarization produced a sharp boost in verification, but only approached ceiling performance in the networks that were highly trained on faces. Moreover, in these face-expert networks, the sharp familiarity benefit was seen only at the identity-based output probability layer, and did not depend on changes to perceptual representations; rather, familiarity effects required learning only at the level of identity readout from a fixed expert representation. Our results thus reconcile the existence of a large familiar face advantage with claims that both familiar and unfamiliar face identity processing depend on shared expert perceptual representations.

The Face of Image Reconstruction: Progress, Pitfalls, Prospects.

Recent research has demonstrated that neural and behavioral data acquired in response to viewing face images can be used to reconstruct the images themselves. However, the theoretical implications, promises, and challenges of this direction of research remain unclear. We evaluate the potential of this research for elucidating the visual representations underlying face recognition. Specifically, we outline complementary and converging accounts of the visual content, the representational structure, and the neural dynamics of face processing. We illustrate how this research addresses fundamental questions in the study of normal and impaired face recognition, and how image reconstruction provides a powerful framework for uncovering face representations, for unifying multiple types of empirical data, and for facilitating both theoretical and methodological progress.

Unique N170 signatures to words and faces in deaf ASL signers reflect experience-specific adaptations during early visual processing.

Previous studies with deaf adults reported reduced N170 waveform asymmetry to visual words, a finding attributed to reduced phonological mapping in left-hemisphere temporal regions compared to hearing adults. An open question remains whether this pattern indeed results from reduced phonological processing or from general neurobiological adaptations in visual processing of deaf individuals. Deaf ASL signers and hearing nonsigners performed a same-different discrimination task with visually presented words, faces, or cars, while scalp EEG time-locked to the onset of the first item in each pair was recorded. For word recognition, the typical left-lateralized N170 in hearing participants and reduced left-sided asymmetry in deaf participants were replicated. The groups did not differ on word discrimination but better orthographic skill was associated with larger N170 in the right hemisphere only for deaf participants. Face recognition was characterized by unique N170 signatures for both groups, and deaf individuals exhibited superior face discrimination performance. Laterality or discrimination performance effects did not generalize to the N170 responses to cars, confirming that deaf signers are not inherently less lateralized in their electrophysiological responses to words and critically, giving support to the phonological mapping hypothesis. P1 was attenuated for deaf participants compared to the hearing, but in both groups, P1 selectively discriminated between highly learned familiar objects - words and faces versus less familiar objects - cars. The distinct electrophysiological signatures to words and faces reflected experience-driven adaptations to words and faces that do not generalize to object recognition.

Protracted Developmental Trajectory of Shape Processing along the Two Visual Pathways.

Studies of the emergence of shape representations in childhood have focused primarily on the ventral visual pathway. Importantly, however, there is increasing evidence that, in adults, the dorsal pathway also represents shape-based information. These dorsal representations follow a gradient with more posterior regions being more shape-sensitive than anterior regions and with representational similarity in some posterior regions that is equivalent to that observed in some ventral regions. To explore the emergence and nature of dorsal shape representations in development, we acquired both fMRI BOLD signals and behavioral data in children (aged 8-10 years) using a parametric image scrambling paradigm. Children exhibited adult-like large-scale organization of shape processing along both ventral and dorsal pathways. Also, as in adults, the activation profiles of children's posterior dorsal and ventral regions were correlated with recognition performance, reflecting a possible contribution of these signals to perception. There were age-related changes, however, with children being more affected by the distortion of shape information than adults, both behaviorally and neurally. These findings reveal that shape-processing mechanisms along both dorsal and ventral pathways are subject to a protracted developmental trajectory.

The large-scale organization of shape processing in the ventral and dorsal pathways.

Although shape perception is considered a function of the ventral visual pathway, evidence suggests that the dorsal pathway also derives shape-based representations. In two psychophysics and neuroimaging experiments, we characterized the response properties, topographical organization and perceptual relevance of these representations. In both pathways, shape sensitivity increased from early visual cortex to extrastriate cortex but then decreased in anterior regions. Moreover, the lateral aspect of the ventral pathway and posterior regions of the dorsal pathway were sensitive to the availability of fundamental shape properties, even for unrecognizable images. This apparent representational similarity between the posterior-dorsal and lateral-ventral regions was corroborated by a multivariate analysis. Finally, as with ventral pathway, the activation profile of posterior dorsal regions was correlated with recognition performance, suggesting a possible contribution to perception. These findings challenge a strict functional dichotomy between the pathways and suggest a more distributed model of shape processing.

Word and face processing engage overlapping distributed networks: Evidence from RSVP and EEG investigations.

Words and faces have vastly different visual properties, but increasing evidence suggests that word and face processing engage overlapping distributed networks. For instance, fMRI studies have shown overlapping activity for face and word processing in the fusiform gyrus despite well-characterized lateralization of these objects to the left and right hemispheres, respectively. To investigate whether face and word perception influences perception of the other stimulus class and elucidate the mechanisms underlying such interactions, we presented images using rapid serial visual presentations. Across 3 experiments, participants discriminated 2 face, word, and glasses targets (T1 and T2) embedded in a stream of images. As expected, T2 discrimination was impaired when it followed T1 by 200 to 300 ms relative to longer intertarget lags, the so-called attentional blink. Interestingly, T2 discrimination accuracy was significantly reduced at short intertarget lags when a face was followed by a word (face-word) compared with glasses-word and word-word combinations, indicating that face processing interfered with word perception. The reverse effect was not observed; that is, word-face performance was no different than the other object combinations. EEG results indicated the left N170 to T1 was correlated with the word decrement for face-word trials, but not for other object combinations. Taken together, the results suggest face processing interferes with word processing, providing evidence for overlapping neural mechanisms of these 2 object types. Furthermore, asymmetrical face-word interference points to greater overlap of face and word representations in the left than the right hemisphere. (PsycINFO Database Record

Statistical learning of parts and wholes: A neural network approach.

Statistical learning is often considered to be a means of discovering the units of perception, such as words and objects, and representing them as explicit "chunks." However, entities are not undifferentiated wholes but often contain parts that contribute systematically to their meanings. Studies of incidental auditory or visual statistical learning suggest that, as participants learn about wholes they become insensitive to parts embedded within them, but this seems difficult to reconcile with a broad range of findings in which parts and wholes work together to contribute to behavior. Bayesian approaches provide a principled description of how parts and wholes can contribute simultaneously to performance, but are generally not intended to model the computations that actually give rise to this performance. In the current work, we develop an account based on learning in artificial neural networks in which the representation of parts and wholes is a matter of degree, and the extent to which they cooperate or compete arises naturally through incidental learning. We show that the approach accounts for a wide range of findings concerning the relationship between parts and wholes in auditory and visual statistical learning, including some findings previously thought to be problematic for neural network approaches. (PsycINFO Database Record

Hemispheric Organization in Disorders of Development.

A recent theoretical account posits that, during the acquisition of word recognition in childhood, the pressure to couple visual and language representations in the left hemisphere (LH) results in competition with the LH representation of faces, which consequently become largely, albeit not exclusively, lateralized to the right hemisphere (RH). We explore predictions from this hypothesis using a hemifield behavioral paradigm with words and faces as stimuli, with concurrent ERP measurement, in a group of adults with developmental dyslexia (DD) or with congenital prosopagnosia (CP), and matched control participants. Behaviorally, the DD group exhibited clear deficits in both word and face processing relative to controls, while the CP group showed a specific deficit in face processing only. This pattern was mirrored in the ERP data too. The DD group evinced neither the normal ERP pattern of RH dominance for faces nor the LH dominance for words. In contrast, the CP group showed the typical ERP superiority for words in the LH but did not show the typical RH superiority for faces. These findings are consistent with the hypothesis that the typical hemispheric organization for words can develop in the absence of typical hemispheric organization for faces but not vice versa, supporting the account of interactive perceptual development.

Spatiotemporal dynamics of similarity-based neural representations of facial identity.

Humans' remarkable ability to quickly and accurately discriminate among thousands of highly similar complex objects demands rapid and precise neural computations. To elucidate the process by which this is achieved, we used magnetoencephalography to measure spatiotemporal patterns of neural activity with high temporal resolution during visual discrimination among a large and carefully controlled set of faces. We also compared these neural data to lower level "image-based" and higher level "identity-based" model-based representations of our stimuli and to behavioral similarity judgments of our stimuli. Between ∼50 and 400 ms after stimulus onset, face-selective sources in right lateral occipital cortex and right fusiform gyrus and sources in a control region (left V1) yielded successful classification of facial identity. In all regions, early responses were more similar to the image-based representation than to the identity-based representation. In the face-selective regions only, responses were more similar to the identity-based representation at several time points after 200 ms. Behavioral responses were more similar to the identity-based representation than to the image-based representation, and their structure was predicted by responses in the face-selective regions. These results provide a temporally precise description of the transformation from low- to high-level representations of facial identity in human face-selective cortex and demonstrate that face-selective cortical regions represent multiple distinct types of information about face identity at different times over the first 500 ms after stimulus onset. These results have important implications for understanding the rapid emergence of fine-grained, high-level representations of object identity, a computation essential to human visual expertise.

Modeling the N400 ERP component as transient semantic over-activation within a neural network model of word comprehension.

The study of the N400 event-related brain potential has provided fundamental insights into the nature of real-time comprehension processes, and its amplitude is modulated by a wide variety of stimulus and context factors. It is generally thought to reflect the difficulty of semantic access, but formulating a precise characterization of this process has proved difficult. Laszlo and colleagues (Laszlo & Plaut, 2012; Laszlo & Armstrong, 2014) used physiologically constrained neural networks to model the N400 as transient over-activation within semantic representations, arising as a consequence of the distribution of excitation and inhibition within and between cortical areas. The current work extends this approach to successfully model effects on both N400 amplitudes and behavior of word frequency, semantic richness, repetition, semantic and associative priming, and orthographic neighborhood size. The account is argued to be preferable to one based on "implicit semantic prediction error" (Rabovsky & McRae, 2014) for a number of reasons, the most fundamental of which is that the current model actually produces N400-like waveforms in its real-time activation dynamics.

'What' Is Happening in the Dorsal Visual Pathway.

The cortical visual system is almost universally thought to be segregated into two anatomically and functionally distinct pathways: a ventral occipitotemporal pathway that subserves object perception, and a dorsal occipitoparietal pathway that subserves object localization and visually guided action. Accumulating evidence from both human and non-human primate studies, however, challenges this binary distinction and suggests that regions in the dorsal pathway contain object representations that are independent of those in ventral cortex and that play a functional role in object perception. We review here the evidence implicating dorsal object representations, and we propose an account of the anatomical organization, functional contributions, and origins of these representations in the service of perception.

Feature-based face representations and image reconstruction from behavioral and neural data.

The reconstruction of images from neural data can provide a unique window into the content of human perceptual representations. Although recent efforts have established the viability of this enterprise using functional magnetic resonance imaging (MRI) patterns, these efforts have relied on a variety of prespecified image features. Here, we take on the twofold task of deriving features directly from empirical data and of using these features for facial image reconstruction. First, we use a method akin to reverse correlation to derive visual features from functional MRI patterns elicited by a large set of homogeneous face exemplars. Then, we combine these features to reconstruct novel face images from the corresponding neural patterns. This approach allows us to estimate collections of features associated with different cortical areas as well as to successfully match image reconstructions to corresponding face exemplars. Furthermore, we establish the robustness and the utility of this approach by reconstructing images from patterns of behavioral data. From a theoretical perspective, the current results provide key insights into the nature of high-level visual representations, and from a practical perspective, these findings make possible a broad range of image-reconstruction applications via a straightforward methodological approach.

Relative meaning frequencies for 578 homonyms in two Spanish dialects: A cross-linguistic extension of the English eDom norms.

Relative meaning frequency is a critical factor to consider in studies of semantic ambiguity. In this work, we examined how this measure may change across the European and Rioplatense dialects of Spanish, as well as how the overall distributional properties differ between Spanish and English, using a computer-assisted norming approach based on dictionary definitions (Armstrong, Tokowicz, & Plaut, 2012). The results showed that the two dialects differ considerably in terms of the relative meaning frequencies of their constituent homonyms, and that the overall distributions of relative frequencies vary considerably across languages, as well. These results highlight the need for localized norms to design powerful studies of semantic ambiguity and suggest that dialectal differences may be responsible for some discrepant effects related to homonymy. In quantifying the reliability of the norms, we also established that as few as seven ratings are needed to converge on a highly stable set of ratings. This approach is therefore a very practical means of acquiring essential data in studies of semantic ambiguity, relative to past approaches, such as those based on the classification of free associates. The norms also present new possibilities for studying semantic ambiguity effects within and between populations who speak one or more languages. The norms and associated software are available for download at http://edom.cnbc.cmu.edu/ or http://www.bcbl.eu/databases/edom/ .