Determinantal point process attention over grid cell code supports out of distribution generalization.

Deep neural networks have made tremendous gains in emulating human-like intelligence, and have been used increasingly as ways of understanding how the brain may solve the complex computational problems on which this relies. However, these still fall short of, and therefore fail to provide insight into how the brain supports strong forms of generalization of which humans are capable. One such case is out-of-distribution (OOD) generalization - successful performance on test examples that lie outside the distribution of the training set. Here, we identify properties of processing in the brain that may contribute to this ability. We describe a two-part algorithm that draws on specific features of neural computation to achieve OOD generalization, and provide a proof of concept by evaluating performance on two challenging cognitive tasks. First we draw on the fact that the mammalian brain represents metric spaces using grid cell code (e.g., in the entorhinal cortex): abstract representations of relational structure, organized in recurring motifs that cover the representational space. Second, we propose an attentional mechanism that operates over the grid cell code using determinantal point process (DPP), that we call DPP attention (DPP-A) - a transformation that ensures maximum sparseness in the coverage of that space. We show that a loss function that combines standard task-optimized error with DPP-A can exploit the recurring motifs in the grid cell code, and can be integrated with common architectures to achieve strong OOD generalization performance on analogy and arithmetic tasks. This provides both an interpretation of how the grid cell code in the mammalian brain may contribute to generalization performance, and at the same time a potential means for improving such capabilities in artificial neural networks.

The relational bottleneck as an inductive bias for efficient abstraction.

A central challenge for cognitive science is to explain how abstract concepts are acquired from limited experience. This has often been framed in terms of a dichotomy between connectionist and symbolic cognitive models. Here, we highlight a recently emerging line of work that suggests a novel reconciliation of these approaches, by exploiting an inductive bias that we term the relational bottleneck. In that approach, neural networks are constrained via their architecture to focus on relations between perceptual inputs, rather than the attributes of individual inputs. We review a family of models that employ this approach to induce abstractions in a data-efficient manner, emphasizing their potential as candidate models for the acquisition of abstract concepts in the human mind and brain.

Evidence for Spinozan "Unbelieving" in the Right Inferior Prefrontal Cortex.

Humans can think about possible states of the world without believing in them, an important capacity for high-level cognition. Here, we use fMRI and a novel "shell game" task to test two competing theories about the nature of belief and its neural basis. According to the Cartesian theory, information is first understood, then assessed for veracity, and ultimately encoded as either believed or not believed. According to the Spinozan theory, comprehension entails belief by default, such that understanding without believing requires an additional process of "unbelieving." Participants (n = 70) were experimentally induced to have beliefs, desires, or mere thoughts about hidden states of the shell game (e.g., believing that the dog is hidden in the upper right corner). That is, participants were induced to have specific "propositional attitudes" toward specific "propositions" in a controlled way. Consistent with the Spinozan theory, we found that thinking about a proposition without believing it is associated with increased activation of the right inferior frontal gyrus. This was true whether the hidden state was desired by the participant (because of reward) or merely thought about. These findings are consistent with a version of the Spinozan theory whereby unbelieving is an inhibitory control process. We consider potential implications of these results for the phenomena of delusional belief and wishful thinking.

Two Ways to Build a Thought: Distinct Forms of Compositional Semantic Representation across Brain Regions.

To understand a simple sentence such as "the woman chased the dog", the human mind must dynamically organize the relevant concepts to represent who did what to whom. This structured recombination of concepts (woman, dog, chased) enables the representation of novel events, and is thus a central feature of intelligence. Here, we use functional magnetic resonance (fMRI) and encoding models to delineate the contributions of three brain regions to the representation of relational combinations. We identify a region of anterior-medial prefrontal cortex (amPFC) that shares representations of noun-verb conjunctions across sentences: for example, a combination of "woman" and "chased" to encode woman-as-chaser, distinct from woman-as-chasee. This PFC region differs from the left-mid superior temporal cortex (lmSTC) and hippocampus, two regions previously implicated in representing relations. lmSTC represents broad role combinations that are shared across verbs (e.g., woman-as-agent), rather than narrow roles, limited to specific actions (woman-as-chaser). By contrast, a hippocampal sub-region represents events sharing narrow conjunctions as dissimilar. The success of the hippocampal conjunctive encoding model is anti-correlated with generalization performance in amPFC on a trial-by-trial basis, consistent with a pattern separation mechanism. Thus, these three regions appear to play distinct, but complementary, roles in encoding compositional event structure.

Concepts and Compositionality: In Search of the Brain's Language of Thought.

Imagine Genghis Khan, Aretha Franklin, and the Cleveland Cavaliers performing an opera on Maui. This silly sentence makes a serious point: As humans, we can flexibly generate and comprehend an unbounded number of complex ideas. Little is known, however, about how our brains accomplish this. Here we assemble clues from disparate areas of cognitive neuroscience, integrating recent research on language, memory, episodic simulation, and computational models of high-level cognition. Our review is framed by Fodor's classic language of thought hypothesis, according to which our minds employ an amodal, language-like system for combining and recombining simple concepts to form more complex thoughts. Here, we highlight emerging work on combinatorial processes in the brain and consider this work's relation to the language of thought. We review evidence for distinct, but complementary, contributions of map-like representations in subregions of the default mode network and sentence-like representations of conceptual relations in regions of the temporal and prefrontal cortex.

An architecture for encoding sentence meaning in left mid-superior temporal cortex.

Human brains flexibly combine the meanings of words to compose structured thoughts. For example, by combining the meanings of "bite," "dog," and "man," we can think about a dog biting a man, or a man biting a dog. Here, in two functional magnetic resonance imaging (fMRI) experiments using multivoxel pattern analysis (MVPA), we identify a region of left mid-superior temporal cortex (lmSTC) that flexibly encodes "who did what to whom" in visually presented sentences. We find that lmSTC represents the current values of abstract semantic variables ("Who did it?" and "To whom was it done?") in distinct subregions. Experiment 1 first identifies a broad region of lmSTC whose activity patterns (i) facilitate decoding of structure-dependent sentence meaning ("Who did what to whom?") and (ii) predict affect-related amygdala responses that depend on this information (e.g., "the baby kicked the grandfather" vs. "the grandfather kicked the baby"). Experiment 2 then identifies distinct, but neighboring, subregions of lmSTC whose activity patterns carry information about the identity of the current "agent" ("Who did it?") and the current "patient" ("To whom was it done?"). These neighboring subregions lie along the upper bank of the superior temporal sulcus and the lateral bank of the superior temporal gyrus, respectively. At a high level, these regions may function like topographically defined data registers, encoding the fluctuating values of abstract semantic variables. This functional architecture, which in key respects resembles that of a classical computer, may play a critical role in enabling humans to flexibly generate complex thoughts.

Chromaticity of color perception and object color knowledge.

Sensorimotor theories of semantic memory require overlap between conceptual and perceptual representations. One source of evidence for such overlap comes from neuroimaging reports of co-activation during memory retrieval and perception; for example, regions involved in color perception (i.e., regions that respond more to colored than grayscale stimuli) are activated by retrieval of object color. One unanswered question from these studies is whether distinctions that are observed during perception are likewise observed during memory retrieval. That is, are regions defined by a chromaticity effect in perception similarly modulated by the chromaticity of remembered objects (e.g., lemons more than coal)? Subjects performed color perception and color retrieval tasks while undergoing fMRI. We observed increased activation during both perception and memory retrieval of chromatic compared to achromatic stimuli in overlapping areas of the left lingual gyrus, but not in dorsal or anterior regions activated during color perception. These results support sensorimotor theories but suggest important distinctions within the conceptual system.