Averting the gaze disengages the environment and facilitates remembering.

When people are asked moderately difficult questions, they often avert their gazes. We report five experiments in which we documented this phenomenon. They demonstrate that (1) the frequency of gaze aversion is related to the difficulty of cognitive processing, (2) this behavior cannot be due solely to demand characteristics or embarrassment, and (3) the behavior is functional: Averting the gaze improves performance. We speculate that averting the gaze helps people to disengage from environmental stimulation and thereby enhances the efficiency of cognitive processing directed by nonenvironmental stimulation.

The representation of space in mental models derived from text.

Mental models of text are representations of what the text is about (i.e., situations), rather than representations of the text itself. Many mental model theories propose that mental models, like real situations, are played out in a medium analogous to a Euclidean space so that distance has functional consequences. For example, when one mentally manipulates one element of the representation, one will notice other elements that are spatially close to it, and this will enhance their short-term accessibility. In a test of this noticing hypothesis, participants read texts that described the object-by-object construction of a spatial layout. According to the text, a critical object ended up close to a target object (in the spatial layout) or far from the target object. In neither case, however, was the relation between the critical object and the target object explicitly described in the text. The noticing hypothesis predicts that the accessibility of the target object will be enhanced when the critical object is close to it. We tested this prediction in seven experiments in which we also manipulated the number of objects described, whether the description was accompanied by a diagram, the presentation modality of the description, the number of dimensions in the spatial layout, and the measurement of accessibility. We failed to find consistent support for the noticing hypothesis. The data compel the conclusions that (1) spatial representations can be formed when one is reading, (2) these representations do not support automatic noticing of implicit spatial relations, (3) it is likely that the spatial representation is more topological or functional than Euclidean.

What memory is for.

Let's start from scratch in thinking about what memory is for, and consequently, how it works. Suppose that memory and conceptualization work in the service of perception and action. In this case, conceptualization is the encoding of patterns of possible physical interaction with a three-dimensional world. These patterns are constrained by the structure of the environment, the structure of our bodies, and memory. Thus, how we perceive and conceive of the environment is determined by the types of bodies we have. Such a memory would not have associations. Instead, how concepts become related (and what it means to be related) is determined by how separate patterns of actions can be combined given the constraints of our bodies. I call this combination "mesh." To avoid hallucination, conceptualization would normally be driven by the environment, and patterns of action from memory would play a supporting, but automatic, role. A significant human skill is learning to suppress the overriding contribution of the environment to conceptualization, thereby allowing memory to guide conceptualization. The effort used in suppressing input from the environment pays off by allowing prediction, recollective memory, and language comprehension. I review theoretical work in cognitive science and empirical work in memory and language comprehension that suggest that it may be possible to investigate connections between topics as disparate as infantile amnesia and mental-model theory.

On-line processing of textual illustrations in the visuospatial sketchpad: evidence from dual-task studies.

We present evidence that the comprehension of illustrated text makes use of the visuospatial sketchpad component of working memory. The results from three experiments demonstrate that the comprehending of texts that are accompanied by pictures interferes with the performance of a spatial short-term memory task more than does the comprehending of texts that have no pictures. A fourth experiment demonstrates that the interference is found only when there is a requirement to comprehend the text; that is, the simple presentation of the texts and their pictures did not degrade performance on the spatial task. Finally, we show that the comprehension of illustrated texts does not differentially interfere with a verbal short-term memory task. These results are consistent with the claim that illustrations accompanying a text encourage the formation of a spatial mental model with the use of the working memory's sketchpad.

Mental models, pictures, and text: integration of spatial and verbal information.

In the past several years, there has been an acceleration in the publication of cognitive research on the interplay between linguistic and pictorial/spatial information. To report on and encourage this sort of research, we organized a symposium at the 1991 meeting of the Midwestern Psychological Association. The articles in this special section of Memory & Cognition are based on the work presented at the symposium. In this introduction, we offer a suggestion for why the integration of linguistic and spatial information is not only a possibility, but a requirement for effective communication. Our suggestion follows the linguistic analysis of the closed-class elements that convey spatial relations, the prepositions (Talmy, 1983). The structure of language provides but a small set of prepositions to encode the vast number of spatial relations that we can perceive. Thus, to understand a situation that a speaker or a writer is conveying, the listener or reader must combine linguistic information with (perhaps metric) spatial information derived from pictures, the environment, or memory.

Pictures and anaphora: evidence for independent processes.

Pictures enhance our comprehension of written texts, but the perceptual and cognitive processes that underlie this effect have not been identified. Because integrating the information contained in a text places demands on working memory, the effect of a picture may be to expand the functional capacity of working memory and thereby to facilitate comprehension. Reasoning thus, we predicted that the availability of a diagram would interact with the difficulty of resolving anaphoric references in texts. The resolution of an anaphor distant from its antecedent (which should stress working memory) should benefit greatly from the presentation of a picture, whereas the resolution of an anaphor near to its antecedent should benefit less from the presentation of a picture. Picture availability and distance separating the anaphor from its antecedent were manipulated in experiments involving both cumulative and moving window presentations of texts. Although picture presence and ease of anaphor resolution significantly improved comprehension of the material, no evidence was found for an interaction of these factors. The results are interpreted as consistent both with dual code theory and with aspects of working memory management that do not involve anaphor resolution.

Temporal coding in rhythm tasks revealed by modality effects.

Temporal coding has been studied by examining the perception and reproduction of rhythms and by examining memory for the order of events in a list. We attempt to link these research programs both empirically and theoretically. Glenberg and Swanson (1986) proposed that the superior recall of auditory material, compared with visual material, reflects more accurate temporal coding for the auditory material. In this paper, we demonstrate that a similar modality effect can be produced in a rhythm task. Auditory rhythms composed of stimuli of two durations are reproduced more accurately than are visual rhythms. Furthermore, it appears that the auditory superiority reflects enhanced chunking of the auditory material rather than better identification of durations.

Common processes underlie enhanced recency effects for auditory and changing-state stimuli.

For some stimuli, dynamic changes are crucial for identifying just what the stimuli are. For example, spoken words (or any auditory stimuli) require change over time to be recognized. Kallman and Cameron (1989) have proposed that this sort of dynamic change underlies the enhanced recency effect found for auditory stimuli, relative to visual stimuli. The results of three experiments replicate and extend Kallman and Cameron's finding that dynamic visual stimuli (that is visual stimuli in which movement is necessary to identify the stimuli), relative to static visual stimuli, engender enhanced recency effects. In addition, an analysis based on individual differences is used to demonstrate that the processes underlying enhanced recency effects for auditory and dynamic visual stimuli are substantially similar. These results are discussed in the context of perceptual grouping processes.

Modality effects in the coding and reproduction of rhythms.

The temporal coding assumption is that time of presentation is coded more accurately for auditory events than for visual events. This assumption has been used to explain the modality effect, in which recall of recent auditory events is superior to recall of recent visual events. We tested the temporal coding assumption by examining the coding and reproduction of quintessentially temporal stimuli-rhythms. The rhythms were produced by sequences of short and long auditory stimuli or short and long visual stimuli; in either case, the task was to reproduce the temporal sequence. The results from four experiments demonstrated reproduction of auditory rhythms superior to that of visual rhythms. We conclude that speech-based explanations of modality effects cannot accommodate these findings, whereas the findings are consistent with explanations based on the temporal coding assumption.

Evidence for auditory temporal distinctiveness: modality effects in order and frequency judgments.

Two new, long-lasting phenomena involving modality of stimulus presentation are documented. In one series of experiments we investigated effects of modality of presentation on order judgments. Order judgments for auditory words were more accurate than order judgments for visual words at both the beginning and the end of lists, and the auditory advantage increased with the temporal separation of the successive items. A second series of experiments investigated effects of modality on estimates of presentation frequency. Frequency estimates of repeated auditory words exceeded frequency estimates of repeated visual words. The auditory advantage increased with frequency of presentation, and this advantage was not affected by the retention interval. These various effects were taken as support for a temporal coding assumption, that auditory presentation produces a more accurate encoding of time of presentation than does visual presentation.

A temporal distinctiveness theory of recency and modality effects.

A temporal distinctiveness theory of contextually cued retrieval from memory is presented and applied to recency and modality effects. According to this theory, one part of the mnemonic trace of an item is a representation of the item's time of presentation. Time of presentation may be encoded with a coarse grain (so that it is consistent with a wide range of times) or with a fine grain (so that it is consistent with a narrow range of times). Retrieval proceeds by constructing temporally defined search sets that include representations of items consistent with the temporal bounds of the search set. The temporal width of the search set increases as the retention interval increases. Recency effects arise from retrieval of recently presented items from narrow search sets that include representations of few items; within the context of the search set, these items are distinctive and recalled well. Superiority in recall of recently presented auditory information in comparison with recently presented visual information is attributed to differences in the grain of time of presentation representations for aurally (fine grain) and visually (coarse grain) presented information. Four experiments confirm qualitative and quantitative predictions of the theory, including the prediction of auditory superiority at the beginning of the list when the initial items are temporally distinct.

Echoic and retrieval accounts of the long-term modality effect tested using the suffix procedure.

The long-term modality effect is the advantage in recall of the last of a list of auditory to-be-remembered (TBR) items compared with the last of a list of visual TBR items when the list is followed by a filled retention interval. If the auditory advantage is due to echoic sensory memory mechanisms, then recall of the last auditory TBR item should be substantially reduced when it is followed by a redundant, not-to-be-recalled auditory suffix. Contrary to this prediction, Experiment 1 demonstrated that a redundant auditory suffix does not significantly reduce recall of the last auditory TBR item. In Experiment 2 a nonredundant auditory suffix produced a large reduction in the last auditory item. Redundancy is not the only factor controlling the effectiveness of a suffix, however. Experiment 3 demonstrated that a nonredundant visual suffix does not reduce recall of the last auditory TBR item. These results are discussed in reference to a retrieval account of the long-term modality effect.

A retrieval account of the long-term modality effect.

When to-be-remembered (TBR) word pairs are separated by distractor activity, recall of the last few audibly presented pairs is greater than recall of the last few visually presented pairs. The effect is found even after a considerably long distractor-filled retention interval. Five experiments disconfirm echoic storage, short-term storage and long-term storage accounts of these effects, as well as demonstrating that the effect is not an artifact of differential use of a recency-first output strategy. The data are generally consistent with the proposition that retrieval is disrupted by modality-specific similarity between to-be-remembered items and distractor information.