Stimulus-locked auditory information facilitates real-time visuo-motor sequence learning.

Prior research investigating whether and how multisensory information facilitates skill learning is quite mixed; whereas some research points to congruent information improving learning, other work suggests that people become reliant on the redundant information, such that its removal ultimately detracts from the ability to perform a unisensory task. We examined this question using the Serial Interception Sequence Learning (SISL) task, a visuo-motor paradigm in which participants implicitly learn a sequence embedded in noise. We investigated whether adding auditory information in different ways would enhance real time sequence learning and whether any benefits of multisensory learning would persist with visual-only testing. Auditory information was used either as feedback on the visuo-motor task (Experiments 1 and 2) or was presented synchronously with visual information during learning (Experiment 3). Robust sequence-specific performance advantages occurred across conditions and experiments; however, auditory information enhanced real-time performance only when it was synchronized with visual information. Participants were significantly more accurate, faster, and more precise with stimulus-locked auditory information during training. Notably, these benefits did not generalize to the visual-only context, suggesting that the benefits of stimulus-locked auditory information are primarily useful only when the perceptual information is present.

Mask wearing affects emotion perception.

To reduce the spread of COVID-19, mask wearing has become ubiquitous in much of the world. We studied the extent to which masks impair emotion recognition and dampen the perceived intensity of facial expressions by naturalistically inducing positive, neutral, and negative emotions in individuals while they were masked and unmasked. Two groups of online participants rated the emotional intensity of each presented image. One group rated full faces (N=104); the other (N=102) rated cropped images where only the upper face was visible. We found that masks impaired the recognition of and rated intensity of positive emotions. This happened even when the faces were cropped and the lower part of the face was not visible. Masks may thus reduce positive emotion and/or expressivity of positive emotion. However, perception of negativity was unaffected by masking, perhaps because unlike positive emotions like happiness which are signaled more in the mouth, negative emotions like anger rely more on the upper face.

Visual cortex responds to sound onset and offset during passive listening.

Sounds enhance our ability to detect, localize, and respond to co-occurring visual targets. Research suggests that sounds improve visual processing by resetting the phase of ongoing oscillations in visual cortex. However, it remains unclear what information is relayed from the auditory system to visual areas and if sounds modulate visual activity even in the absence of visual stimuli (e.g., during passive listening). Using intracranial electroencephalography (iEEG) in humans, we examined the sensitivity of visual cortex to three forms of auditory information during a passive listening task: auditory onset responses, auditory offset responses, and rhythmic entrainment to sounds. Because some auditory neurons respond to both sound onsets and offsets, visual timing and duration processing may benefit from each. In addition, if auditory entrainment information is relayed to visual cortex, it could support the processing of complex stimulus dynamics that are aligned between auditory and visual stimuli. Results demonstrate that in visual cortex, amplitude-modulated sounds elicited transient onset and offset responses in multiple areas, but no entrainment to sound modulation frequencies. These findings suggest that activity in visual cortex (as measured with iEEG in response to auditory stimuli) may not be affected by temporally fine-grained auditory stimulus dynamics during passive listening (though it remains possible that this signal may be observable with simultaneous auditory-visual stimuli). Moreover, auditory responses were maximal in low-level visual cortex, potentially implicating a direct pathway for rapid interactions between auditory and visual cortices. This mechanism may facilitate perception by time-locking visual computations to environmental events marked by auditory discontinuities.NEW & NOTEWORTHY Using intracranial electroencephalography (iEEG) in humans during a passive listening task, we demonstrate that sounds modulate activity in visual cortex at both the onset and offset of sounds, which likely supports visual timing and duration processing. However, more complex auditory rate information did not affect visual activity. These findings are based on one of the largest multisensory iEEG studies to date and reveal the type of information transmitted between auditory and visual regions.

Perspective-taking increases emotionality and empathy but does not reduce harmful biases against American Indians: Converging evidence from the museum and lab.

Given the problematic depictions of Native Americans and the pervasive cultural biases that exist, we sought to understand how contemporary educational practices in museums might encourage viewers to consider the context of their preconceptions rather than passively absorb conventional representations. In this two-part study, we tested whether and how viewers (mis)perceptions and interpretations of Native peoples might be influenced by encouraging empathy-specifically by taking the perspective of a Native individual depicted in a photograph they are visually analyzing. We randomly assigned participants in a lab setting (N = 120) and in a museum setting (N = 75) to one of three conditions (perspective-taking, stereotype-suppression, or control), and examined eye movements, self-reports, and verbal and written responses while participants viewed portrait photographs of American Indians. Notably, perspective-taking led viewers to interpret American Indians in a more emotional, empathetic, and human-centered manner than in control and suppression conditions. This was reflected in eye movements such that control and suppression participants attended to decorative features (e.g. jewelry) more than to the eyes of the depicted individual, whereas perspective-takers' attention was more balanced. Similarly, perspective-takers used more empathetic and emotion-related language, whereas participants in control and suppression groups used more "objective" visually-descriptive language. Crucially, regardless of condition, cultural biases were stubbornly resistant to change and, in some cases, appeared even more frequently for participants adopting others' perspectives. We argue that despite the positive outcomes associated with perspective-taking, the continued presence of cultural biases across conditions demonstrates that cultural competency-based interventions must be more complex and culturally-specific.

Multisensory processing of facial expressions in binocular rivalry.

Binocular rivalry occurs when two percepts, each presented to a single eye, compete for perceptual dominance. Across two experiments, we investigated whether emotional music influenced perceptual dominance of an emotionally congruent face. In the first experiment, participants heard music (happy, threatening, none) while viewing a positive or negative emotional face pitted against a neutral face or emotional faces pitted against each other. Several key findings emerged. As expected, emotional faces significantly dominated over neutral faces, irrespective of music. For emotional face pairings, negative faces were predominantly reported as initial percepts. Interestingly, this negativity bias was transient and did not persist for the duration of the trial. Rather, positive faces dominated perception throughout trials. Moreover, emotional music affected rivalry dynamics such that congruent music drove attention toward congruent emotional percepts and incongruent music suppressed incongruent percepts. In a second experiment with the same group of participants, we investigated whether explicit attention modulated binocular rivalry of emotional faces. We demonstrated that attention affected both initial and sustained percepts by suppressing automatic emotional biases and stabilizing attention-congruent expressions. Together, our results demonstrate the importance of investigating multisensory expression perception in transient and sustained contexts, the role of emotion as a mediator of sensory integration across perceptual modalities, and the influence of attention on emotional competition in binocular rivalry. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

What Is Art Good For? The Socio-Epistemic Value of Art.

Scientists, humanists, and art lovers alike value art not just for its beauty, but also for its social and epistemic importance; that is, for its communicative nature, its capacity to increase one's self-knowledge and encourage personal growth, and its ability to challenge our schemas and preconceptions. However, empirical research tends to discount the importance of such social and epistemic outcomes of art engagement, instead focusing on individuals' preferences, judgments of beauty, pleasure, or other emotional appraisals as the primary outcomes of art appreciation. Here, we argue that a systematic neuroscientific study of art appreciation must move beyond understanding aesthetics alone, and toward investigating the social importance of art appreciation. We make our argument for such a shift in focus first, by situating art appreciation as an active social practice. We follow by reviewing the available psychological and cognitive neuroscientific evidence that art appreciation cultivates socio-epistemic skills such as self- and other-understanding, and discuss philosophical frameworks which suggest a more comprehensive empirical investigation. Finally, we argue that focusing on the socio-epistemic values of art engagement highlights the important role art plays in our lives. Empirical research on art appreciation can thus be used to show that engagement with art has specific social and personal value, the cultivation of which is important to us as individuals, and as communities.

Pattern classification of EEG signals reveals perceptual and attentional states.

Pattern classification techniques have been widely used to differentiate neural activity associated with different perceptual, attentional, or other cognitive states, often using fMRI, but more recently with EEG as well. Although these methods have identified EEG patterns (i.e., scalp topographies of EEG signals occurring at certain latencies) that decode perceptual and attentional states on a trial-by-trial basis, they have yet to be applied to the spatial scope of attention toward global or local features of the display. Here, we initially used pattern classification to replicate and extend the findings that perceptual states could be reliably decoded from EEG. We found that visual perceptual states, including stimulus location and object category, could be decoded with high accuracy peaking between 125-250 ms, and that the discriminative spatiotemporal patterns mirrored and extended our (and other well-established) ERP results. Next, we used pattern classification to investigate whether spatiotemporal EEG signals could reliably predict attentional states, and particularly, the scope of attention. The EEG data were reliably differentiated for local versus global attention on a trial-by-trial basis, emerging as a specific spatiotemporal activation pattern over posterior electrode sites during the 250-750 ms interval after stimulus onset. In sum, we demonstrate that multivariate pattern analysis of EEG, which reveals unique spatiotemporal patterns of neural activity distinguishing between behavioral states, is a sensitive tool for characterizing the neural correlates of perception and attention.

In the working memory of the beholder: Art appreciation is enhanced when visual complexity is compatible with working memory.

What shapes art appreciation? Much research has focused on the importance of visual features themselves (e.g., symmetry, natural scene statistics) and of the viewer's experience and expertise with specific artworks. However, even after taking these factors into account, there are considerable individual differences in art preferences. Our new result suggests that art preference is also influenced by the compatibility between visual properties and the characteristics of the viewer's visual system. Specifically, we have demonstrated, using 120 artworks from diverse periods, cultures, genres, and styles, that art appreciation is increased when the level of visual complexity within an artwork is compatible with the viewer's visual working memory capacity. The result highlights the importance of the interaction between visual features and the beholder's general visual capacity in shaping art appreciation.

Auditory rhythms are systemically associated with spatial-frequency and density information in visual scenes.

A variety of perceptual correspondences between auditory and visual features have been reported, but few studies have investigated how rhythm, an auditory feature defined purely by dynamics relevant to speech and music, interacts with visual features. Here, we demonstrate a novel crossmodal association between auditory rhythm and visual clutter. Participants were shown a variety of visual scenes from diverse categories and asked to report the auditory rhythm that perceptually matched each scene by adjusting the rate of amplitude modulation (AM) of a sound. Participants matched each scene to a specific AM rate with surprising consistency. A spatial-frequency analysis showed that scenes with greater contrast energy in midrange spatial frequencies were matched to faster AM rates. Bandpass-filtering the scenes indicated that greater contrast energy in this spatial-frequency range was associated with an abundance of object boundaries and contours, suggesting that participants matched more cluttered scenes to faster AM rates. Consistent with this hypothesis, AM-rate matches were strongly correlated with perceived clutter. Additional results indicated that both AM-rate matches and perceived clutter depend on object-based (cycles per object) rather than retinal (cycles per degree of visual angle) spatial frequency. Taken together, these results suggest a systematic crossmodal association between auditory rhythm, representing density in the temporal domain, and visual clutter, representing object-based density in the spatial domain. This association may allow for the use of auditory rhythm to influence how visual clutter is perceived and attended.

Laughter exaggerates happy and sad faces depending on visual context.

Laughter is an auditory stimulus that powerfully conveys positive emotion. We investigated how laughter influenced the visual perception of facial expressions. We presented a sound clip of laughter simultaneously with a happy, a neutral, or a sad schematic face. The emotional face was briefly presented either alone or among a crowd of neutral faces. We used a matching method to determine how laughter influenced the perceived intensity of the happy, neutral, and sad expressions. For a single face, laughter increased the perceived intensity of a happy expression. Surprisingly, for a crowd of faces, laughter produced an opposite effect, increasing the perceived intensity of a sad expression in a crowd. A follow-up experiment revealed that this contrast effect may have occurred because laughter made the neutral distractor faces appear slightly happy, thereby making the deviant sad expression stand out in contrast. A control experiment ruled out semantic mediation of the laughter effects. Our demonstration of the strong context dependence of laughter effects on facial expression perception encourages a reexamination of the previously demonstrated effects of prosody, speech content, and mood on face perception, as they may be similarly context dependent.

The role of stereopsis, motion parallax, perspective and angle polarity in perceiving 3-D shape.

We studied how stimulus attributes (angle polarity and perspective) and data-driven signals (motion parallax and binocular disparity) affect recovery of 3-D shape. We used physical stimuli, which consisted of two congruent trapezoids forming a dihedral angle. To study the effects of the stimulus attributes, we used 2 × 2 combinations of convex/concave angles and proper/reverse perspective cues. To study the effects of binocular disparity and motion parallax, we used 2 × 2 combinations of monocular/binocular viewing with moving/stationary observers. The task was to report the depth of the right vertical edge relative to a fixation point positioned at a different depth. In Experiment 1 observers also had the option of reporting that the right vertical edge and fixation point were at the same depth. However, in Experiment 2, observers were only given two response options: is the right vertical edge in front of/behind the fixation point? We found that across all stimulus configurations, perspective is a stronger cue than angle polarity in recovering 3-D shape; we also confirm the bias to perceive convex compared to concave angles. In terms of data-driven signals, binocular disparity recovered 3-D shape better than motion parallax. Interestingly, motion parallax improved performance for monocular viewing but not for binocular viewing.