Similarity Grouping as Feature-Based Selection.

Across the natural world as well as the artificial worlds of maps, diagrams, and data visualizations, feature similarity (e.g., color and shape) links spatially separate areas into sets. Despite a century of study, it is yet unclear what mechanism underlies this gestalt similarity grouping. One recent proposal is that similarity grouping-for example, seeing a red, vertical, or square group-is just global selection of those features. Although parsimonious, this account makes the counterintuitive prediction that similarity grouping is strictly serial: A green group cannot be constructed at the same time as a red group. We tested this prediction with a novel measure-a grouping illusion within number-estimation tasks that should work only if participants simultaneously construct groups-and found the strongest evidence yet in favor of serial feature-based attention ( Ns = 14, 12, and 12 for Experiment 1, Experiment 2, and Experiment 3, respectively).

Representing composed meanings through temporal binding.

A key feature of human thought and language is compositionality, the ability to bind pre-existing concepts or word meanings together in order to express new ideas. Here we ask how newly composed complex concepts are mentally represented and matched to the outside world, by testing whether it is harder to verify if a picture matches the meaning of a phrase, like big pink tree, than the meaning of a single word, like tree. Five sentence-picture verification experiments provide evidence that, in fact, the meaning of a phrase can often be checked just as fast as the meaning of one single word (and sometimes faster), indicating that the phrase's constituent concepts can be represented and checked in parallel. However, verification times were increased when matched phrases had more complex modification structures, indicating that it is costly to represent structural relations between constituent concepts. This pattern of data can be well-explained if concepts are composed together using two different mechanisms, binding by synchrony and binding by asynchrony, which have been suggested as solutions to the "binding problem" faced in both vision science and higher-level cognition. Our results suggest that they can also explain aspects of compositional language processing.

Building phrases in language production: an MEG study of simple composition.

Although research on language production has developed detailed maps of the brain basis of single word production in both time and space, little is known about the spatiotemporal dynamics of the processes that combine individual words into larger representations during production. Studying composition in production is challenging due to difficulties both in controlling produced utterances and in measuring the associated brain responses. Here, we circumvent both problems using a minimal composition paradigm combined with the high temporal resolution of magnetoencephalography (MEG). With MEG, we measured the planning stages of simple adjective-noun phrases ('red tree'), matched list controls ('red, blue'), and individual nouns ('tree') and adjectives ('red'), with results indicating combinatorial processing in the ventro-medial prefrontal cortex (vmPFC) and left anterior temporal lobe (LATL), two regions previously implicated for the comprehension of similar phrases. These effects began relatively quickly (∼180 ms) after the presentation of a production prompt, suggesting that combination commences with initial lexical access. Further, while in comprehension, vmPFC effects have followed LATL effects, in this production paradigm vmPFC effects occurred mostly in parallel with LATL effects, suggesting that a late process in comprehension is an early process in production. Thus, our results provide a novel neural bridge between psycholinguistic models of comprehension and production that posit functionally similar combinatorial mechanisms operating in reversed order.

Flexible composition: MEG evidence for the deployment of basic combinatorial linguistic mechanisms in response to task demands.

The present study investigates whether a minimal manipulation in task demands can induce core linguistic combinatorial mechanisms to extend beyond the bounds of normal grammatical phrases. Using magnetoencephalography, we measured neural activity evoked by the processing of adjective-noun phrases in canonical (red cup) and reversed order (cup red). During a task not requiring composition (verification against a color blob and shape outline), we observed significant combinatorial activity during canonical phrases only - as indexed by minimum norm source activity localized to the left anterior temporal lobe at 200-250 ms(cf. [1], [2]). When combinatorial task demands were introduced (by simply combining the blob and outline into a single colored shape) we observed significant combinatorial activity during reversed sequences as well. These results demonstrate the first direct evidence that basic linguistic combinatorial mechanisms can be deployed outside of normal grammatical expressions in response to task demands, independent of changes in lexical or attentional factors.

Prediction is no panacea: the key to language is in the unexpected.

For action systems, the critical task is to predict what will happen next. In language, however, the critical task is not to predict the next auditory event but to extract meaning. Reducing language to an action system, and putting prediction at center, mistakenly marginalizes our core capacity to communicate the novel and unpredictable.

Stepwise acquisition of vocal combinatorial capacity in songbirds and human infants.

Human language, as well as birdsong, relies on the ability to arrange vocal elements in new sequences. However, little is known about the ontogenetic origin of this capacity. Here we track the development of vocal combinatorial capacity in three species of vocal learners, combining an experimental approach in zebra finches (Taeniopygia guttata) with an analysis of natural development of vocal transitions in Bengalese finches (Lonchura striata domestica) and pre-lingual human infants. We find a common, stepwise pattern of acquiring vocal transitions across species. In our first study, juvenile zebra finches were trained to perform one song and then the training target was altered, prompting the birds to swap syllable order, or insert a new syllable into a string. All birds solved these permutation tasks in a series of steps, gradually approximating the target sequence by acquiring new pairwise syllable transitions, sometimes too slowly to accomplish the task fully. Similarly, in the more complex songs of Bengalese finches, branching points and bidirectional transitions in song syntax were acquired in a stepwise fashion, starting from a more restrictive set of vocal transitions. The babbling of pre-lingual human infants showed a similar pattern: instead of a single developmental shift from reduplicated to variegated babbling (that is, from repetitive to diverse sequences), we observed multiple shifts, where each new syllable type slowly acquired a diversity of pairwise transitions, asynchronously over development. Collectively, these results point to a common generative process that is conserved across species, suggesting that the long-noted gap between perceptual versus motor combinatorial capabilities in human infants may arise partly from the challenges in constructing new pairwise vocal transitions.

Combination Across Domains: An MEG Investigation into the Relationship between Mathematical, Pictorial, and Linguistic Processing.

Debates surrounding the evolution of language often hinge upon its relationship to cognition more generally and many investigations have attempted to demark the boundary between the two. Though results from these studies suggest that language may recruit domain-general mechanisms during certain types of complex processing, the domain-generality of basic combinatorial mechanisms that lie at the core of linguistic processing is still unknown. Our previous work (Bemis and Pylkkänen, 2011, 2012) used magnetoencephalography to isolate neural activity associated with the simple composition of an adjective and a noun ("red boat") and found increased activity during this processing localized to the left anterior temporal lobe (lATL), ventro-medial prefrontal cortex (vmPFC), and left angular gyrus (lAG). The present study explores the domain-generality of these effects and their associated combinatorial mechanisms through two parallel non-linguistic combinatorial tasks designed to be as minimal and natural as the linguistic paradigm. In the first task, we used pictures of colored shapes to elicit combinatorial conceptual processing similar to that evoked by the linguistic expressions and find increased activity again localized to the vmPFC during combinatorial processing. This result suggests that a domain-general semantic combinatorial mechanism operates during basic linguistic composition, and that activity generated by its processing localizes to the vmPFC. In the second task, we recorded neural activity as subjects performed simple addition between two small numerals. Consistent with a wide array of recent results, we find no effects related to basic addition that coincide with our linguistic effects and instead find increased activity localized to the intraparietal sulcus. This result suggests that the scope of the previously identified linguistic effects is restricted to compositional operations and does not extend generally to all tasks that are merely similar in form.

Simple composition: a magnetoencephalography investigation into the comprehension of minimal linguistic phrases.

The expressive power of language lies in its ability to construct an infinite array of ideas out of a finite set of pieces. Surprisingly, few neurolinguistic investigations probe the basic processes that constitute the foundation of this ability, choosing instead to focus on relatively complex combinatorial operations. Contrastingly, in the present work, we investigate the neural circuits underlying simple linguistic composition, such as required by the minimal phrase "red boat." Using magnetoencephalography, we examined activity in humans generated at the visual presentation of target nouns, such as "boat," and varied the combinatorial operations induced by its surrounding context. Nouns in minimal compositional contexts ("red boat") were compared with those appearing in matched non-compositional contexts, such as after an unpronounceable consonant string ("xkq boat") or within a list ("cup, boat"). Source analysis did not implicate traditional language areas (inferior frontal gyrus, posterior temporal regions) in such basic composition. Instead, we found increased combinatorial-related activity in the left anterior temporal lobe (LATL) and ventromedial prefrontal cortex (vmPFC). These regions have been linked previously to syntactic (LATL) and semantic (vmPFC) combinatorial processing in more complex linguistic contexts. Thus, we suggest that these regions play a role in basic syntactic and semantic composition, respectively. Importantly, the temporal ordering of the effects, in which LATL activity (∼225 ms) precedes vmPFC activity (∼400 ms), is consistent with many processing models that posit syntactic composition before semantic composition during the construction of linguistic representations.