The neural and cognitive basis of expository text comprehension.

As science and technology rapidly progress, it becomes increasingly important to understand how individuals comprehend expository technical texts that explain these advances. This study examined differences in individual readers' technical comprehension performance and differences among texts, using functional brain imaging to measure regional brain activity while students read passages on technical topics and then took a comprehension test. Better comprehension of the technical passages was related to higher activation in regions of the left inferior frontal gyrus, left superior parietal lobe, bilateral dorsolateral prefrontal cortex, and bilateral hippocampus. These areas are associated with the construction of a mental model of the passage and with the integration of new and prior knowledge in memory. Poorer comprehension of the passages was related to greater activation of the ventromedial prefrontal cortex and the precuneus, areas involved in autobiographical and episodic memory retrieval. More comprehensible passages elicited more brain activation associated with establishing links among different types of information in the text and activation associated with establishing conceptual coherence within the text representation. These findings converge with previous behavioral research in their implications for teaching technical learners to become better comprehenders and for improving the structure of instructional texts, to facilitate scientific and technological comprehension.

Similarities and differences in the neural representations of abstract concepts across English and Mandarin.

Recent research suggests there is a neural organization for representing abstract concepts that is common across English speakers. To investigate the possible role of language on the representation of abstract concepts, multivariate pattern analytic (MVPA) techniques were applied to fMRI data to compare the neural representations of 28 individual abstract concepts between native English and Mandarin speakers. Factor analyses of the activation patterns of the 28 abstract concepts from both languages characterized this commonality in terms of a set of four underlying neurosemantic dimensions, indicating the degree to which a concept is verbally represented, internal to the person, contains social content, and is rule-based. These common semantic dimensions (factors) underlying the 28 concepts provided a sufficient basis for reliably identifying the individual abstract concepts from their neural signature in the other language with a mean rank accuracy of 0.65 (p < .001). Although the neural dimensions used for representing abstract concepts are common across languages, differences in the meaning of some individual concepts can be accommodated in terms of differential salience of particular dimensions. These semantic dimensions constitute a set of neurocognitive resources for abstract concept representations within a larger set of regions responsible for general semantic processing.

The neuroscience of advanced scientific concepts.

Cognitive neuroscience methods can identify the fMRI-measured neural representation of familiar individual concepts, such as apple, and decompose them into meaningful neural and semantic components. This approach was applied here to determine the neural representations and underlying dimensions of representation of far more abstract physics concepts related to matter and energy, such as fermion and dark matter, in the brains of 10 Carnegie Mellon physics faculty members who thought about the main properties of each of the concepts. One novel dimension coded the measurability vs. immeasurability of a concept. Another novel dimension of representation evoked particularly by post-classical concepts was associated with four types of cognitive processes, each linked to particular brain regions: (1) Reasoning about intangibles, taking into account their separation from direct experience and observability; (2) Assessing consilience with other, firmer knowledge; (3) Causal reasoning about relations that are not apparent or observable; and (4) Knowledge management of a large knowledge organization consisting of a multi-level structure of other concepts. Two other underlying dimensions, previously found in physics students, periodicity, and mathematical formulation, were also present in this faculty sample. The data were analyzed using factor analysis of stably responding voxels, a Gaussian-naïve Bayes machine-learning classification of the activation patterns associated with each concept, and a regression model that predicted activation patterns associated with each concept based on independent ratings of the dimensions of the concepts. The findings indicate that the human brain systematically organizes novel scientific concepts in terms of new dimensions of neural representation.

Neural Representations of Procedural Knowledge.

Although declarative concepts (e.g., apple) have been shown to be identifiable from their functional MRI (fMRI) signatures, the correspondence has yet to be established for executing a complex procedure such as tying a knot. In this study, 7 participants were trained to tie seven knots. Their neural representations of these seven procedures were assessed with fMRI as they imagined tying each knot. A subset of the trained participants physically tied each knot in a later fMRI session. Findings demonstrated that procedural knowledge of tying a particular knot can be reliably identified from its fMRI signature, and such procedural signatures were found here in frontal, parietal, motor, and cerebellar regions. In addition, a classifier trained on mental tying signatures was able to reliably identify when participants were planning to tie knots before they physically tied them, which suggests that the mental-tying and physical-tying procedural signatures are similar. These findings indicate that fMRI activation patterns can illuminate the representation and organization of procedural knowledge.

Neural Representations of Abstract Concepts: Identifying Underlying Neurosemantic Dimensions.

The abstractness of concepts is sometimes defined indirectly as lacking concreteness, this view provides little insight into their cognitive or neural basis. Multivariate pattern analytic techniques applied to functional magnetic resonance imaging data were used to characterize the neural representations of 28 individual abstract concepts. A classifier trained on the concepts' neural signatures reliably decoded their neural representations in an independent subset of data for each participant. There was considerable commonality of the neural representations across participants as indicated by the accurate classification of each participant's concepts based on the neural signatures obtained in other participants. Group-level factor analysis revealed 3 semantic dimensions underlying the 28 concepts, suggesting a brain-based ontology for this set of abstract concepts. The 3 dimensions corresponded to 1) the degree a concept was Verbally Represented; 2) whether a concept was External (or Internal) to the individual, and 3) whether the concept contained Social Content. Further exploration of the Verbal Representation dimension suggests that the degree a concept is verbally represented can be construed as a point on a continuum between language faculties and perceptual faculties. A predictive model, based on independent behavioral ratings of the 28 concepts along the 3 factor dimensions, provided converging evidence for the interpretations.

Converging measures of neural change at the microstructural, informational, and cortical network levels in the hippocampus during the learning of the structure of organic compounds.

The critical role of the hippocampus in human learning has been illuminated by neuroimaging studies that increasingly improve the detail with which hippocampal function is understood. However, the hippocampal information developed with different types of imaging technologies is seldom integrated within a single investigation of the neural changes that occur during learning. Here, we show three different ways in which a small hippocampal region changes as the structures and names of a set of organic compounds are being learned, reflecting changes at the microstructural, informational, and cortical network levels. The microstructural changes are sensed using measures of water diffusivity. The informational changes are assessed using machine learning of the neural representations of organic compounds as they are encoded in the fMRI-measured activation levels of a set of hippocampal voxels. The changes in cortical networks are measured in terms of the functional connectivity between hippocampus and parietal regions. The co-location of these three hippocampal changes reflects that structure's involvement in learning at all three levels of explanation, consistent with the multiple ways in which learning brings about neural change.

Brain reading and behavioral methods provide complementary perspectives on the representation of concepts.

The advent of brain reading techniques has enabled new approaches to the study of concept representation, based on the analysis of multivoxel activation patterns evoked by the contemplation of individual concepts such as animal concepts. The present fMRI study characterized the representation of 30 animal concepts. Dimensionality reduction of the multivoxel activation patterns underlying the individual animal concepts indicated that the semantic building blocks of the brain's representations of the animals corresponded to intrinsic animal properties (e.g. fierceness, intelligence, size). These findings were compared to behavioral studies of concept representation, which have typically collected pairwise similarity ratings between two concepts (e.g. Henley, 1969). Behavioral similarity judgments, by contrast, indicated that the animals were organized into taxonomically defined groups (e.g. canine, feline, equine). The difference in the results between the brain reading and behavioral approaches might derive from differences in cognitive processing during judging similarities versus contemplating one animal at a time. Brain reading approaches may have an advantage in describing thoughts about an individual concept, owing to the ability to decode brain activation patterns elicited by the brief consideration of a single concept (e.g. word reading) without a complex cognitive or behavioral task (e.g. similarity judgments). On the other hand, some behavioral tasks may tend to evoke a concept from numerous perspectives, yielding a representation of the breadth and sophistication of the concept knowledge. These results suggest that neural and behavioral measures offer complementary perspectives that together characterize the content and structure of concept representations.

Commonalities and differences in the neural representations of English, Portuguese, and Mandarin sentences: When knowledge of the brain-language mappings for two languages is better than one.

This study extended cross-language semantic decoding (based on a concept's fMRI signature) to the decoding of sentences across three different languages (English, Portuguese and Mandarin). A classifier was trained on either the mapping between words and activation patterns in one language or the mappings in two languages (using an equivalent amount of training data), and then tested on its ability to decode the semantic content of a third language. The model trained on two languages was reliably more accurate than a classifier trained on one language for all three pairs of languages. This two-language advantage was selective to abstract concept domains such as social interactions and mental activity. Representational Similarity Analyses (RSA) of the inter-sentence neural similarities resulted in similar clustering of sentences in all the three languages, indicating a shared neural concept space among languages. These findings identify semantic domains that are common across these three languages versus those that are more language or culture-specific.

A brain-based account of "basic-level" concepts.

This study provides a brain-based account of how object concepts at an intermediate (basic) level of specificity are represented, offering an enriched view of what it means for a concept to be a basic-level concept, a research topic pioneered by Rosch and others (Rosch et al., 1976). Applying machine learning techniques to fMRI data, it was possible to determine the semantic content encoded in the neural representations of object concepts at basic and subordinate levels of abstraction. The representation of basic-level concepts (e.g. bird) was spatially broad, encompassing sensorimotor brain areas that encode concrete object properties, and also language and heteromodal integrative areas that encode abstract semantic content. The representation of subordinate-level concepts (robin) was less widely distributed, concentrated in perceptual areas that underlie concrete content. Furthermore, basic-level concepts were representative of their subordinates in that they were neurally similar to their typical but not atypical subordinates (bird was neurally similar to robin but not woodpecker). The findings provide a brain-based account of the advantages that basic-level concepts enjoy in everyday life over subordinate-level concepts: the basic level is a broad topographical representation that encompasses both concrete and abstract semantic content, reflecting the multifaceted yet intuitive meaning of basic-level concepts.

Predicting the brain activation pattern associated with the propositional content of a sentence: Modeling neural representations of events and states.

Even though much has recently been learned about the neural representation of individual concepts and categories, neuroimaging research is only beginning to reveal how more complex thoughts, such as event and state descriptions, are neurally represented. We present a predictive computational theory of the neural representations of individual events and states as they are described in 240 sentences. Regression models were trained to determine the mapping between 42 neurally plausible semantic features (NPSFs) and thematic roles of the concepts of a proposition and the fMRI activation patterns of various cortical regions that process different types of information. Given a semantic characterization of the content of a sentence that is new to the model, the model can reliably predict the resulting neural signature, or, given an observed neural signature of a new sentence, the model can predict its semantic content. The models were also reliably generalizable across participants. This computational model provides an account of the brain representation of a complex yet fundamental unit of thought, namely, the conceptual content of a proposition. In addition to characterizing a sentence representation at the level of the semantic and thematic features of its component concepts, factor analysis was used to develop a higher level characterization of a sentence, specifying the general type of event representation that the sentence evokes (e.g., a social interaction versus a change of physical state) and the voxel locations most strongly associated with each of the factors. Hum Brain Mapp 38:4865-4881, 2017. © 2017 Wiley Periodicals, Inc.

Neural representations of the concepts in simple sentences: Concept activation prediction and context effects.

Although it has been possible to identify individual concepts from a concept's brain activation pattern, there have been significant obstacles to identifying a proposition from its fMRI signature. Here we demonstrate the ability to decode individual prototype sentences from readers' brain activation patterns, by using theory-driven regions of interest and semantic properties. It is possible to predict the fMRI brain activation patterns evoked by propositions and words which are entirely new to the model with reliably above-chance rank accuracy. The two core components implemented in the model that reflect the theory were the choice of intermediate semantic features and the brain regions associated with the neurosemantic dimensions. This approach also predicts the neural representation of object nouns across participants, studies, and sentence contexts. Moreover, we find that the neural representation of an agent-verb-object proto-sentence is more accurately characterized by the neural signatures of its components as they occur in a similar context than by the neural signatures of these components as they occur in isolation.

Machine learning of neural representations of suicide and emotion concepts identifies suicidal youth.

The clinical assessment of suicidal risk would be significantly complemented by a biologically-based measure that assesses alterations in the neural representations of concepts related to death and life in people who engage in suicidal ideation. This study used machine-learning algorithms (Gaussian Naïve Bayes) to identify such individuals (17 suicidal ideators vs 17 controls) with high (91%) accuracy, based on their altered fMRI neural signatures of death and life-related concepts. The most discriminating concepts were death, cruelty, trouble, carefree, good, and praise. A similar classification accurately (94%) discriminated 9 suicidal ideators who had made a suicide attempt from 8 who had not. Moreover, a major facet of the concept alterations was the evoked emotion, whose neural signature served as an alternative basis for accurate (85%) group classification. The study establishes a biological, neurocognitive basis for altered concept representations in participants with suicidal ideation, which enables highly accurate group membership classification.

Commonality of neural representations of sentences across languages: Predicting brain activation during Portuguese sentence comprehension using an English-based model of brain function.

The aim of the study was to test the cross-language generative capability of a model that predicts neural activation patterns evoked by sentence reading, based on a semantic characterization of the sentence. In a previous study on English monolingual speakers (Wang et al., submitted), a computational model performed a mapping from a set of 42 concept-level semantic features (Neurally Plausible Semantic Features, NPSFs) as well as 6 thematic role markers to neural activation patterns (assessed with fMRI), to predict activation levels in a network of brain locations. The model used two types of information gained from the English-based fMRI data to predict the activation for individual sentences in Portuguese. First, it used the mapping weights from NPSFs to voxel activation levels derived from the model for English reading. Second, the brain locations for which the activation levels were predicted were derived from a factor analysis of the brain activation patterns during English reading. These meta-language locations were defined by the clusters of voxels with high loadings on each of the four main dimensions (factors), namely people, places, actions and feelings, underlying the neural representations of the stimulus sentences. This cross-language model succeeded in predicting the brain activation patterns associated with the reading of 60 individual Portuguese sentences that were entirely new to the model, attaining accuracies reliably above chance level. The prediction accuracy was not affected by whether the Portuguese speaker was monolingual or Portuguese-English bilingual. The model's confusion errors indicated an accurate capture of the events or states described in the sentence at a conceptual level. Overall, the cross-language predictive capability of the model demonstrates the neural commonality between speakers of different languages in the representations of everyday events and states, and provides an initial characterization of the common meta-language neural basis.

Identifying thematic roles from neural representations measured by functional magnetic resonance imaging.

The generativity and complexity of human thought stem in large part from the ability to represent relations among concepts and form propositions. The current study reveals how a given object such as rabbit is neurally encoded differently and identifiably depending on whether it is an agent ("the rabbit punches the monkey") or a patient ("the monkey punches the rabbit"). Machine-learning classifiers were trained on functional magnetic resonance imaging (fMRI) data evoked by a set of short videos that conveyed agent-verb-patient propositions. When tested on a held-out video, the classifiers were able to reliably identify the thematic role of an object from its associated fMRI activation pattern. Moreover, when trained on one subset of the study participants, classifiers reliably identified the thematic roles in the data of a left-out participant (mean accuracy = .66), indicating that the neural representations of thematic roles were common across individuals.

Neural Representations of Physics Concepts.

We used functional MRI (fMRI) to assess neural representations of physics concepts (momentum, energy, etc.) in juniors, seniors, and graduate students majoring in physics or engineering. Our goal was to identify the underlying neural dimensions of these representations. Using factor analysis to reduce the number of dimensions of activation, we obtained four physics-related factors that were mapped to sets of voxels. The four factors were interpretable as causal motion visualization, periodicity, algebraic form, and energy flow. The individual concepts were identifiable from their fMRI signatures with a mean rank accuracy of .75 using a machine-learning (multivoxel) classifier. Furthermore, there was commonality in participants' neural representation of physics; a classifier trained on data from all but one participant identified the concepts in the left-out participant (mean accuracy = .71 across all nine participant samples). The findings indicate that abstract scientific concepts acquired in an educational setting evoke activation patterns that are identifiable and common, indicating that science education builds abstract knowledge using inherent, repurposed brain systems.

Modality-independent representations of small quantities based on brain activation patterns.

Machine learning or MVPA (Multi Voxel Pattern Analysis) studies have shown that the neural representation of quantities of objects can be decoded from fMRI patterns, in cases where the quantities were visually displayed. Here we apply these techniques to investigate whether neural representations of quantities depicted in one modality (say, visual) can be decoded from brain activation patterns evoked by quantities depicted in the other modality (say, auditory). The main finding demonstrated, for the first time, that quantities of dots were decodable by a classifier that was trained on the neural patterns evoked by quantities of auditory tones, and vice-versa. The representations that were common across modalities were mainly right-lateralized in frontal and parietal regions. A second finding was that the neural patterns in parietal cortex that represent quantities were common across participants. These findings demonstrate a common neuronal foundation for the representation of quantities across sensory modalities and participants and provide insight into the role of parietal cortex in the representation of quantity information.