Cross-Modal Contributions to Episodic Memory for Voices.

Multisensory context often facilitates perception and memory. In fact, encoding items within a multisensory context can improve memory even on strictly unisensory tests (i.e., when the multisensory context is absent). Prior studies that have consistently found these multisensory facilitation effects have largely employed multisensory contexts in which the stimuli were meaningfully related to the items targeting for remembering (e.g., pairing canonical sounds and images). Other studies have used unrelated stimuli as multisensory context. A third possible type of multisensory context is one that is environmentally related simply because the stimuli are often encountered together in the real world. We predicted that encountering such a multisensory context would also enhance memory through cross-modal associations, or representations relating to one's prior multisensory experience with that sort of stimuli in general. In two memory experiments, we used faces and voices of unfamiliar people as everyday stimuli individuals have substantial experience integrating the perceptual features of. We assigned participants to face- or voice-recognition groups and ensured that, during the study phase, half of the face or voice targets were encountered also with information in the other modality. Voices initially encoded along with faces were consistently remembered better, providing evidence that cross-modal associations could explain the observed multisensory facilitation.

Effect of impoverished information on multisensory integration in judgments of learning.

According to the principle of inverse effectiveness (PIE), weaker responses to information in one modality (i.e., unisensory) benefit more from additional information in a second modality (i.e., multisensory; Meredith & Stein, 1986). We suggest that the PIE may also inform whether perceptual fluency affects judgments of learning (JOLs). If JOLs follow the PIE, the differences in JOLs for multisensory and unisensory items should increase as the unisensory study items become harder to perceive. That is, an influence of perceptual fluency should prompt a similar, interactive pattern across perceptual responses and JOLs. In 3 experiments, we systematically varied the signal intensity or noise in 1 modality to examine how responses might change with the inclusion of information in a second modality. In Experiment 1, written words in several font sizes were sometimes accompanied by spoken equivalents. In Experiments 2 and 3, spoken words in various background noise levels were sometimes accompanied by visual speech articulations. Consistent with the PIE, the multisensory benefits in response time and/or correct identification increased as responses to unisensory information decreased. Also, the multisensory formats received higher JOLs than the unisensory formats; however, unlike the predictions from PIE, this difference did not increase as study items became harder to perceive. Experiment 3 extended this finding to participants' explicit beliefs. In multisensory settings, JOLs may rely more on theory- than data-driven processes. We suggest that broadly defined processing fluency may always contribute to JOLs, but, regarding perceptual information, JOLs appear to track perceptual attributes rather than perceptual fluency. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Face-voice space: Integrating visual and auditory cues in judgments of person distinctiveness.

Faces and voices each convey multiple cues enabling us to tell people apart. Research on face and voice distinctiveness commonly utilizes multidimensional space to represent these complex, perceptual abilities. We extend this framework to examine how a combined face-voice space would relate to its constituent face and voice spaces. Participants rated videos of speakers for their dissimilarity in face only, voice only, and face-voice together conditions. Multiple dimensional scaling (MDS) and regression analyses showed that whereas face-voice space more closely resembled face space, indicating visual dominance, face-voice distinctiveness was best characterized by a multiplicative integration of face-only and voice-only distinctiveness, indicating that auditory and visual cues are used interactively in person-distinctiveness judgments. Further, the multiplicative integration could not be explained by the small correlation found between face-only and voice-only distinctiveness. As an exploratory analysis, we next identified auditory and visual features that correlated with the dimensions in the MDS solutions. Features pertaining to facial width, lip movement, spectral centroid, fundamental frequency, and loudness variation were identified as important features in face-voice space. We discuss the implications of our findings in terms of person perception, recognition, and face-voice matching abilities.

Judgments of Learning in Context: Backgrounds Can Both Reduce and Produce Metamemory Illusions.

Varying item-specific features such as size (Rhodes & Castel, 2008) or blur (Yue, Castel, & Bjork, 2013) often produces metamemory illusions in which one type of item receives higher judgments of learning (JOLs) without being recalled better. In this study, we explored how similar manipulations to context would influence JOLs. When to-be-recalled words varying in size (or blur) were accompanied by backgrounds also varying in size (or blur), the traditional JOL illusions were reduced (Experiments 1, 2, 4, and 5) compared to when there were no backgrounds (Experiments 3a, 3b, and 4). Thus, the item-specific and contextual cues were used interactively. Further, the background manipulations also sometimes themselves led to metamemory illusions regarding JOLs for the to-be-remembered items. In general, there were robust individual differences in how participants used the cues, including how they incorporated the contextual cues into their JOL decisions. In part, this may explain why interactive cue utilization did not always emerge at the group level. In sum, we showed that context may affect JOLs both directly and indirectly by influencing participants' use of item-specific cues. These findings broaden our understanding of how cues may be utilized (e.g., Koriat, 1997) and integrated (e.g., Undorf, Söllner, and Bröder, 2018) in JOLs.

Intensifying the intensity illusion in judgments of learning: Modality and cue combinations.

We showed that judgments of learning (JOLs) were not affected by presentation modality in a list-learning task, although the typical font-size and loudness illusions emerged in that large-font visual presentations and loud auditory presentations elicited higher JOLs than their less intense counterparts. Further, when items were presented in both modalities simultaneously, large-font/quiet and small-font/loud items received similar JOLs (and were recalled similarly). Most importantly, when the intensity manipulation was compounded across modalities, the magnitude of the illusion increased beyond that observed in a single modality, showing the influence of combining cues. Whereas recall was still the same, large-font/loud items received higher JOLs than either small-font/loud items or large-font/quiet items, and not-intense items received very low JOLs. These differences emerged only when all conditions were presented within a single list and not in a between-subjects design, underscoring the importance of comparative judgments.

Matching Speaking to Singing Voices and the Influence of Content.

OBJECTIVES AND HYPOTHESIS: We tested whether speaking voices of unfamiliar people could be matched to their singing voices, and, if so, whether the content of the utterances would influence this matching performance. Our hypothesis was that enough acoustic features would remain the same between speaking and singing voices such that their identification as belonging to the same or different individuals would be possible even upon a single hearing. We also hypothesized that the contents of the utterances would influence this identification process such that voices uttering words would be easier to match than those uttering vowels. STUDY DESIGN: We used a within-participant design with blocked stimuli that were counterbalanced using a Latin square design. In one block, mode (speaking vs singing) was manipulated while content was held constant; in another block, content (word vs syllable) was manipulated while mode was held constant, and in the control block, both mode and content were held constant. METHOD: Participants indicated whether the voices in any given pair of utterances belonged to the same person or to different people. RESULTS: Cross-mode matching was above chance level, although mode-congruent performance was better. Further, only speaking voices were easier to match when uttering words. CONCLUSIONS: We can identify speaking and singing voices as the same or different even on just a single hearing. However, content interacts with mode such that words benefit matching of speaking voices but not of singing voices. Results are discussed within an attentional framework.

Event-Based Prospective Memory Is Resistant but Not Immune to Proactive Interference.

Recent evidence suggests that proactive interference (PI) does not hurt event-based prospective memory (ProM) the way it does retrospective memory (RetroM) (Oates, Peynircioglu, & Bates, 2015). We investigated this apparent resistance further. Introduction of a distractor task to ensure we were testing ProM rather than vigilance in Experiment 1 and tripling the number of lists to provide more opportunity for PI buildup in Experiment 2 still did not produce performance decrements. However, when the ProM task was combined with a RetroM task in Experiment 3, a comparable buildup and release was observed also in the ProM task. It appears that event based ProM is indeed somewhat resistant to PI, but this resistance can break down when the ProM task comprises the same stimuli as in an embedded RetroM task. We discuss the results using the ideas of cue overload and distinctiveness as well as shared attentional and working memory resources.

Flexibility of temporal order in musical and linguistic recognition.

In Experiments 1-3 participants heard pairs of audio clips, each corresponding to half of a phrase (linguistic or musical), in a forward direction or a backward direction, where the second half came first. They judged whether the second clip was from a familiar source or from the same source as the first clip. In both tasks participants were faster and more accurate when the clips were from the same source and faster with linguistic stimuli. Longer temporal distances impaired performance, although greater flexibility was shown with linguistic materials. In Experiment 4, single extended clips were played in temporal or scrambled order. Judgments of familiarity were slower with scrambled song melodies than with instrumental melodies, and processing of music was disrupted more than that of language when temporal order was violated. These results suggest that semantic meaning enhances processing of temporal order information and modulates access.

The importance of source and cue type in time-based everyday prospective memory.

We examined the effects of the source of a prospective memory task (provided or generated) and the type of cue (specific or general) triggering that task in everyday settings. Participants were asked to complete both generated and experimenter-provided tasks and to send a text message when each task was completed. The cue/context for the to-be-completed tasks was either a specific time or a general deadline (time-based cue), and the cue/context for the texting task was the completion of the task itself (activity-based cue). Although generated tasks were completed more often, generated cues/contexts were no more effective than provided ones in triggering the intention. Furthermore, generated tasks were completed more often when the cue/context comprised a specific time, whereas provided tasks were completed more often when the cue/context comprised a general deadline. However, texting was unaffected by the source of the cue/context. Finally, emotion modulated the effects. Results are discussed within a process-driven framework.

Flexibility of event boundaries in autobiographical memory.

Events have clear and consistent boundaries that are defined during perception in a manner that influences memory performance. The natural process of event segmentation shapes event definitions during perception, and appears to play a critical role in defining distinct episodic memories at encoding. However, the role of retrieval processes in modifying event definitions is not clear. We explored how such processes changed event boundary definitions at recall. In Experiment 1 we showed that distance from encoding is related to boundary flexibility. Participants were more likely to move self-reported event boundaries to include information reported beyond those boundaries when recalling more distant events compared to more recent events. In Experiment 2 we showed that age also influenced boundary flexibility. Older Age adults were more likely to move event boundaries than College Age adults, and the relationship between distance from encoding and boundary flexibility seen in Experiment 1 was present only in College Age and Middle Age adults. These results suggest that factors at retrieval have a direct impact on event definitions in memory and that, although episodic memories may be initially defined at encoding, these definitions are not necessarily maintained in long-term memory.

Feeling-of-knowing for songs and instrumental music.

We explored the differences between metamemory judgments for titles as well as for melodies of instrumental music and those for songs with lyrics. Participants were given melody or title cues and asked to provide the corresponding titles or melodies or feeling of knowing (FOK) ratings. FOK ratings were higher but less accurate for titles with melody cues than vice versa, but only in instrumental music, replicating previous findings. In a series of seven experiments, we ruled out style, instrumentation, and strategy differences as explanations for this asymmetry. A mediating role of lyrics between the title and the melody in songs was also ruled out. What emerged as the main explanation was the degree of familiarity with the musical pieces, which was manipulated either episodically or semantically, and within this context, lyrics appeared to serve as an additional source of familiarity. Results are discussed using the Interactive Theory of how FOK judgments are made.

Levels-of-processing effects on "remember" responses in recognition for familiar and unfamiliar tunes.

We investigated the effect of level-of-processing manipulations on "remember" and "know" responses in episodic melody recognition (Experiments 1 and 2) and how this effect is modulated by item familiarity (Experiment 2). In Experiment 1, participants performed 2 conceptual and 2 perceptual orienting tasks while listening to familiar melodies: judging the mood, continuing the tune, tracing the pitch contour, and counting long notes. The conceptual mood task led to higher d' rates for "remember" but not "know" responses. In Experiment 2, participants either judged the mood or counted long notes of tunes with high and low familiarity. A level-of-processing effect emerged again in participants' "remember" d' rates regardless of melody familiarity. Results are discussed within the distinctive processing framework.

Revelation effect in metamemory.

Test items are more likely to be judged as previously studied if they need to be discovered before the recognition decision. In the present experiments, this revelation effect was extended to metamemory judgments. Participants studied word pairs and then tried to recall the second word of each pair when given the first word as a cue. In Experiment 1, a fragment of the target was either gradually increased in size or held constant, and in Experiment 2, sometimes an anagram of the cue was given instead of the cue itself. Thus, for some items, there was a revelation task before a recall attempt. If recall failed, the participants gave feeling-of-knowing (FOK) ratings. In both experiments, the participants gave higher FOK ratings after a revelation task, even though the items that these FOKs referred to remained unrecalled. Analyses showed a criterion shift but no differences in sensitivity.

Feeling of knowing for names in response to faces.

Participants were asked to recall the names when shown photographs of faces in both a semantic task (Experiment 1) and an episodic task (Experiments 2 and 3). When recall failed, feeling of knowing (FOK) ratings were solicited. In addition, participants reported on the strategies that they used to make their ratings, whether they could recall other pieces of information (the target-accessibility strategy, e.g., Koriat, A. (1993). How do we know that? The accessibility model of the feeling of knowing. Psychological Review, 100, 609-639) or whether the faces simply looked familiar (the cue-familiarity strategy, e.g., Schwartz, B. L., & Metcalfe, J. (1992). Cue familiarity but not target accessibility enhances feeling of knowing ratings. Journal of Experimental Psychology: Learning, Memory, and Cognition, 18, 1074-1083). In all experiments, FOK ratings were fairly accurate in that participants were successful in predicting their performance on a subsequent recognition test. More importantly, participants reported using the cue-familiarity strategy more often, although they gave higher FOK ratings when they reported using the target-accessibility strategy. The FOK ratings that were given using the two strategies were equally accurate.

The role of familiarity in episodic memory and metamemory for music.

Participants heard music snippets of varying melodic and instrumental familiarity paired with animal-name titles. They then recalled the target when given either the melody or the title as a cue, or they gave name feeling-of-knowing (FOK) ratings. In general, recall for titles was better than it was for melodies, and recall was enhanced with increasing melodic familiarity of both the cues and the targets. Accuracy of FOK ratings, but not magnitude, also increased with increasing familiarity. Although similar ratings were given after melody and title cues, accuracy was better with title cues. Finally, knowledge of the real titles of the familiar music enhanced recall but had, by and large, no effect on the FOK ratings.

Effects of bilingual and monolingual presentations and time on concept- and language-matching tasks.

Balanced Turkish-English bilingual participants viewed word pairs, presented both monolingually (English-English or Turkish-Turkish) or bilingually (English-Turkish or Turkish-English) and both for short and long durations. They made decisions on whether the simultaneously presented words in a pair were in the same language or not, or whether they denoted the same concept or not. In the short presentation condition, we found no evidence for subliminal processing. In cases in which both words were consciously identified, participants were more accurate, although not faster in the long than in the short presentation condition for both language and concept decisions. In the long presentation condition, language decisions were more accurate than concept decisions, although not faster. In addition, language decisions were not affected by whether the words were synonyms (concept identity), and concept decisions were not affected by whether the presentation was monolingual or bilingual (language identity), although in the monolingual conditions, "same" decisions were faster but not more accurate, and in the bilingual conditions a speed-accuracy trade-off was observed in that "same" decisions were faster but "different" decisions were more accurate.

Effects of age on flashbulb memories.

The authors investigated the effect of aging on flashbulb (FB) memories. In 1996, elderly Turks recalled how they had heard about 2 remote events--the death of the first president of Turkey (in 1938) and another event involving a change in the national borders of the country (in 1939)--and both elderly and younger adults recalled how they heard about the recent death of the 8th president of Turkey (in 1993). Seventy percent of the elderly had FB memories for the 1938 death; critical variables for the formation of FB memories were personal importance attached to the event and rehearsal. Ninety percent of younger Turks and 72% of elderly Turks had FB memories for the 1993 death; the only variable that differed between the 2 groups was rehearsal.