Facial expressions affect the memory of facial colors.

Facial color influences the perception of facial expressions, and emotional expressions bias how facial color is remembered. However, it remains unclear whether facial expressions affect daily facial color memory. The memory color effect demonstrates that knowledge about typical colors affects the perception of the actual color of given objects. To investigate the effect of facial color memory, we examined whether the memory color effect for faces varies depending on facial expression. We calculated the subjective achromatic point of the facial expression image stimulus and compared the degree to which it was shifted from the actual achromatic point between facial expression conditions. We hypothesized that if the memory of facial color is influenced by the facial expression color (e.g., anger is a warm color, fear is a cold color), then the subjective achromatic point would vary with facial expression. In Experiment 1, we recruited 13 participants who adjusted the color of facial expression stimuli (anger, neutral, and fear) and a banana stimulus to be achromatic. No significant differences in the subjective achromatic point between facial expressions were observed. Subsequently, we conducted Experiment 2 with 23 participants because Experiment 1 did not account for the sensitivity to color changes on the face; humans perceive greater color differences in faces than in non-faces. Participants selected which facial color they believed the expression stimulus appeared to be, choosing one of two options provided to them. The results indicated that the subjective achromatic points of anger and fear faces significantly shifted toward the opposite color direction compared with neutral faces in the brief presentation condition. This research suggests that the memory color of faces differs depending on facial expressions and supports the idea that the perception of emotional expressions can bias facial color memory.

Glossiness perception and its pupillary response.

Recent studies have revealed that pupillary response changes depend on perceptual factors such as subjective brightness caused by optical illusions and luminance. However, the manner in which the perceptual factor that is derived from the glossiness perception of object surfaces affects the pupillary response remains unclear. We investigated the relationship between the glossiness perception and pupillary response through a glossiness rating experiment that included recording the pupil diameter. We prepared general object images (original) and randomized images (shuffled) that comprised the same images with randomized small square regions as stimuli. The image features were controlled by matching the luminance histogram. The observers were asked to rate the perceived glossiness of the stimuli presented for 3,000 ms and the changes in their pupil diameters were recorded. Images with higher glossiness ratings constricted the pupil size more than those with lower glossiness ratings at the peak constriction of the pupillary responses during the stimulus duration. The linear mixed-effects model demonstrated that the glossiness rating, image category (original/shuffled), variance of the luminance histogram, and stimulus area were most effective in predicting the pupillary responses. These results suggest that the illusory brightness obtained by the image regions of high-glossiness objects, such as specular highlights, induce pupil constriction.

The effect of facial colour on implicit facial expressions.

Humans recognise reddish-coloured faces as angry. However, does facial colour also affect "implicit" facial expression perception of which humans are not explicitly aware? In this study, we investigated the effects of facial colour on implicit facial expression perception. The experimental stimuli were "hybrid faces", in which the low-frequency component of the neutral facial expression image was replaced with the low-frequency component of the facial expression image of happiness or anger. In Experiment 1, we confirmed that the hybrid face stimuli were perceived as neutral and, therefore, supported implicit facial expression perception. In Experiment 2, the hybrid face stimuli were adjusted to natural and reddish facial colours, and their friendliness ratings were compared. The results showed that the expression of happiness was rated as more friendly than the expression of anger. In addition, the expression of happiness was rated as friendlier when the low-frequency happy component was red, but the friendliness rating of the expression of anger did not change when it was presented in red. In Experiment 3, we affirmed the implicit facial expression perception even in reddish colours. These results suggest that facial colour modulates the perception of implicit facial expressions in hybrid facial stimuli.

Brightness Perception in World-Centered Coordinates Assessed by Pupillometry.

Subjective brightness perception reportedly differs among the peripheral visual fields owing to lower- and higher-order cognition. However, there is still a lack of information associated with subjective brightness perception in the world-centered coordinates, not in the visual fields. In this study, we aimed to investigate the anisotropy of subjective brightness perception in the world-centered coordinates based on pupillary responses to the stimuli in five locations by manipulating the world-centered coordinates through active (requiring head movement) and passive scenes (without head movement) in a virtual reality environment. Specifically, this study aimed to elucidate if there is an ecological advantage in the five different locations in the world-centered coordinates. The pupillary responses to glare and halo stimuli indicated that the brightness perception differed among the five locations in the world-centered coordinates. Furthermore, we found that the pupillary response to stimuli at the top location might be influenced by ecological factors (such as from the bright sky and the sun's existence). Thus, we have contributed to the understanding of the extraretinal information influence on subjective brightness perception in the world-centered coordinates, demonstrating that the pupillary response is independent of head movement.

Anisotropy in the peripheral visual field based on pupil response to the glare illusion.

Visual-field (VF) anisotropy has been investigated in terms of spatial resolution of attention, spatial frequency, and semantic processing. Brightness perception has also been reported to vary between VFs. However, the influence of VF anisotropy on brightness perception using pupillometry has not been investigated. The present study measured participants' pupil size during glare illusion, in which converging luminance gradients evoke brightness enhancement and a glowing impression on the central white area of the stimulus, and halo stimuli, in which the same physical brightness of the glare illusion is used with a diverging luminance pattern. The results revealed greater stimulus-evoked pupillary dilation and glare-related dilated pupil reduction in the upper VF (UVF) compared with other VFs and halo-related pupillary changes, respectively. The stimulus-evoked pupillary dilation was affected by poor contrast sensitivity. However, owing to the superior cognitive bias formed by statistical regularity in natural scene processing of the glare illusion in the UVF, we found reduced pupillary dilation compared with the response to halo stimuli and the response from other VFs. These findings offer valuable insight into a method to reduce the potential glare effect of any VF anisotropy induced by the glare effect experienced in daily vision. An important practical implication of our study may be in informing the design of applications aimed at improving nighttime driving behavior. We also believe that our study makes a significant contribution to the literature because it offers valuable insights on VF anisotropy using evidence from pupillometry and the glare illusion.

The effect of red/blue color stimuli on temporal perception under different pupillary responses induced by different equiluminant methods.

As time plays a fundamental role in our social activities, scholars have studied temporal perception since the earliest days of experimental psychology. Since the 1960s, the ubiquity of color has been driving research on the potential effects of the colors red and blue on temporal perception and on its underlying mechanism. However, the results have been inconsistent, which could be attributed to the difficulty of controlling physical properties such as hue and luminance within and between studies. Therefore, we conducted a two-interval duration-discrimination task to evaluate the perceived duration of color stimuli under different equiluminant conditions: subjective or pupillary light reflex (PLR)-based equiluminance. The results, based on psychometric functional analyses and simultaneous pupillary recordings, showed that the perceived duration of red was overestimated compared with blue even when the intensity of the stimulus was controlled based on subjective equiluminance (Experiment 1). However, since blue is known to induce a larger PLR than red despite equiluminance, we conducted a controlled study to distinguish the indirect effect of pupillary response to temporal perception. Interestingly, the effect observed in Experiment 1 faded when the luminance levels of the two stimuli were matched based on PLR response (Experiment 2). These results indicate that duration judgement can be affected not only by the hue but also by different equiluminance methods. Furthermore, this causality between the equiluminance method and temporal perception can be explained by the fluctuations in incident light entering the pupil.

Backward and forward neck tilt affects perceptual bias when interpreting ambiguous figures.

The relationships between posture and perception have already been investigated in several studies. However, it is still unclear how perceptual bias and experiential contexts of human perception affect observers' perception when posture is changed. In this study, we hypothesized that a change in the perceptual probability caused by perceptual bias also depends on posture. In order to verify this hypothesis, we used the Necker cube with two types of appearance, from above and below, although the input is constant, and investigated the change of the probability of perceptual content. Specifically, we asked observers their perception of the appearance of the Necker cube placed at any of the five angles in the space of virtual reality. There were two patterns of neck movement, vertical and horizontal. During the experiment, pupil diameter, one of the cognitive indices, was also measured. Results showed that during the condition of looking down vertically, the probability of the viewing-from-above perception of the Necker cube was significantly greater than during the condition of looking up. Interestingly, the pupillary results were also consistent with the probability of the perception. These results indicate that perception was modulated by the posture of the neck and suggest that neck posture is incorporated into ecological constraints.

Universality and superiority in preference for chromatic composition of art paintings.

Color composition in paintings is a critical factor affecting observers' aesthetic judgments. We examined observers' preferences for the color composition of Japanese and Occidental paintings when their color gamut was rotated. In the experiment, observers were asked to select their preferred image from original and three hue-rotated images in a four-alternative forced choice paradigm. Despite observers' being unfamiliar with the presented artwork, the original paintings (0 degrees) were preferred more frequently than the hue-rotated ones. Furthermore, the original paintings' superiority was observed when the images were divided into small square pieces and their positions randomized (Scrambled condition), and when the images were composed of square pieces collected from different art paintings and composed as patchwork images (Patchwork condition). Therefore, the original paintings' superiority regarding preference was quite robust, and the specific objects in the paintings associated with a particular color played only a limited role. Rather, the original paintings' general trend in color statistics influenced hue-angle preference. Art paintings likely share common statistical regulations in color distributions, which may be the basis for the universality and superiority of the preference for original paintings.

Pupillary response reflects attentional modulation to sound after emotional arousal.

There have been various studies on the effects of emotional visual processing on subsequent non-emotional auditory stimuli. A previous study with EEG has shown that responses to deviant sounds presented after presenting negative pictures collected more attentional resources than those for neutral pictures. To investigate such a compelling between emotional and cognitive processing, this study aimed to examined pupillary responses to an auditory stimulus after a positive, negative, or neutral emotional state was elicited by an emotional image. An emotional image was followed by a beep sound that was either repetitive or unexpected, and the pupillary dilation was measured. As a result, we found that the early component of the pupillary response to the beep sound was larger for negative and positive emotional states than the neutral emotional state, whereas the late component was larger for the positive emotional state than the negative and neutral emotional states. In addition, the peak latency of the pupillary response was earlier for negative than neutral or positive images. Further, to compensate for the disadvantage of low-temporal resolution of the pupillary data, the pupillary responses were deconvoluted and used in the analysis. The deconvolution analysis of pupillary responses confirmed that the responses to beep sound were more likely to be modulated by the emotional state rather than being influenced by the short presentation interval between the images and sounds. These findings suggested that pupil size index modulations in the compelling situation between emotional and cognitive processing.

Effect of glare illusion-induced perceptual brightness on temporal perception.

Temporal perception and the ability to precisely ascertain time duration are central to essentially all behaviors. Since stimulus magnitude is assumed to be positively related to the perceived duration from the early days of experimental psychology, most studies so far have assessed this effect by presenting stimuli with relatively different intensities in physical quantity. However, it remains unclear how perceptual magnitude itself directly affects temporal perception. In this study (n = 21, n = 20), we conducted a two-interval duration-discrimination task adapting a glare illusion (a visual illusion that enhances perceived brightness without changing physical luminance) to investigate whether the temporal perception is also influenced by perceptual magnitude. Based on the mean difference in the point of subjective equality derived from a psychometric function and pupil diameter, we found that temporal perception is influenced by the illusory brightness of glare stimuli. Interestingly, the perceived duration of the apparently brighter stimuli (glare stimuli; larger pupillary light reflex) was shorter than that of control stimuli (halo stimuli; smaller pupillary light reflex) despite the stimuli remaining physically equiluminant, in contrast with the well-known "magnitude effect." Furthermore, this temporal modulation did not occur when the physical luminance of the stimuli was manipulated to match the illusory-induced magnitude. These results indicate that temporal processing depends on the confluence of both external and perceived subjective magnitude and even illusory brightness is sufficient to affect the sense of duration; which may be explained by the internal magnitude decrease of the glare stimuli due to pupillary constriction decreasing the light entering the eye.

Steady-state visually evoked potential is modulated by the difference of recognition condition.

Recent researches revealed that the EEG component caused by the flickering visual stimulus, which is called steady-state visually evoked potential (SSVEP), might be a potential index for object recognition. This study examined whether SSVEP reflects different states during object recognition. In one trial, a binary image (BI), which is difficult to recognize, was followed by a grayscale image (GI) of the same object as the answer. Both BI and GI were presented in a flickering manner at a frequency of 7.5 Hz. Participants were first asked to answer whether they could recognize BI. Then, after GI was shown, participants were requested to answer whether they recognized it. We analyzed the evoked and induced component of SSVEPs from the two recognition conditions. As a result, the SSVEPs to BI were significantly larger than that to GI. In addition, induced component to GI after the BI was unrecognized was smaller than after the BI was recognized. The present data provide evidence that SSVEPs reflect a transition of cognitive state to ambiguous figures is reflected.

Pupil dilation reflects English /l//r/ discrimination ability for Japanese learners of English: a pilot study.

The importance of the English language has been increasing as various fields have become more globalized. When Japanese people try to acquire foreign language such as English, learners find it difficult to perceive speech-sounds such as the phonemes /l/ and /r/ that are absent in their native language (e.g., "light"/lάIt/ and "right"/rάIt/). Recent studies report that a unique sound that deviates from a repetitive background sound induces pupillary dilation response (PDR) regardless of whether attention is directed to the sound or not. In this study, we investigated whether deviation in higher-order processing such as language processing induces PDR, and the possibility of determining implicit subjective English proficiency. A behavioural auditory distinguishing ability test was performed prior to the main experiment to quantitatively evaluate participants' ability to distinguish English words. Then, by conducting an oddball paradigm-employing stimulus including the phonemes /l/ and /r/ with simultaneous pupil diameter recording, a significant dilation was evoked by /l/-/r/ speech sounds presented as deviant stimuli. Moreover, a strong correlation between the PDR amplitude and participants' ability to distinguish English words was found; that is, individuals with higher ability to distinguish such words displayed a prominent PDR. Also, the PDR difference between the two groups classified by discrimination ability suggests that PDR might be sensitive to higher-order characteristics involved in language processing, which is independent from the aspects of physical sound and cognitive load.

Asymmetrical characteristics of emotional responses to pictures and sounds: Evidence from pupillometry.

In daily life, our emotions are often elicited by a multimodal environment, mainly visual and auditory stimuli. Therefore, it is crucial to investigate the symmetrical characteristics of emotional responses to pictures and sounds. In this study, we aimed to elucidate the relationship of attentional states to emotional unimodal stimuli (pictures or sounds) and emotional responses by measuring the pupil diameter, which reflects the emotional arousal associated with increased sympathetic activity. Our hypothesis was that the emotional responses to both the image and sound stimuli are symmetrical: emotion might be suppressed when attentional resources are allocated to another stimulus of the same modality as the emotional stimulus-such as a dot presented at the same time as an emotional image, and a beep sound presented at the same time as an emotional sound. In our two experiments, data for 24 participants were analyzed for a pupillary response. In experiment 1, we investigated the relationship of the attentional state with emotional visual stimuli (International Affective Picture System) and emotional responses by using pupillometry. We set four task conditions to modulate the attentional state (emotional task, no task, visual detection task, and auditory detection task). We observed that the velocity of pupillary dilation was faster during the presentation of emotionally arousing pictures compared to that of neutral ones, regardless of the valence of the pictures. Importantly, this effect was not dependent on the task condition. In experiment 2, we investigated the relationship of the attentional state with emotional auditory sounds (International Affective Digitized Sounds) and emotional responses. We observed a trend towards a significant interaction between the stimulus and the task conditions with regard to the velocity of pupillary dilation. In the emotional and auditory detection tasks, the velocity of pupillary dilation was faster with positive and neutral sounds than negative sounds. However, there were no significant differences between the no task and visual detection task conditions. Taken together, the current data reveal that different pupillary responses were elicited to emotional visual and auditory stimuli, at least in the point that there is no attentional effect to emotional responses to visual stimuli, despite both experiments being sufficiently controlled to be of symmetrical experimental design.

Cueing the Necker cube: Pupil dilation reflects the viewing-from-above constraint in bistable perception.

We hypothesized that a perceptually ambiguous or bistable object (Necker cube) can be more effectively biased to assume a point of view-from-above (VFA) than from below the object by cueing attention. Participants viewed a Necker cube in which one surface was temporarily shaded so as to prime a specific perspective on the cube. Subsequently, the standard (wireframe) Necker cube was viewed for 3 seconds, and participants reported what perspective they had seen initially and whether their perception shifted to the alternative perspective during the brief viewing. Concomitantly, pupil size was monitored with an eye-tracker to obtain an index of cognitive effort. There were two conditions: passive viewing and forced attention to sustain the initially primed perspective. We confirmed the presence of a VFA bias with forced attention, which was accompanied by reduced attentional effort, as indexed by a reduced pupil diameter, compared with the view-from-below. Participants showed no bias during passive viewing. We suggest that the level of intensive attention, when retrieving and maintaining a specific view from memory, is mirrored in the size of the eye pupils and may reflect ecological constraints on visual perception.

Cooperative update of beliefs and state-transition functions in human reinforcement learning.

It is widely known that reinforcement learning systems in the brain contribute to learning via interactions with the environment. These systems are capable of solving multidimensional problems, in which some dimensions are relevant to a reward, while others are not. To solve these problems, computational models use Bayesian learning, a strategy supported by behavioral and neural evidence in human. Bayesian learning takes into account beliefs, which represent a learner's confidence in a particular dimension being relevant to the reward. Beliefs are given as a posterior probability of the state-transition (reward) function that maps the optimal actions to the states in each dimension. However, when it comes to implementing this learning strategy, the order in which beliefs and state-transition functions update remains unclear. The present study investigates this update order using a trial-by-trial analysis of human behavior and electroencephalography signals during a task in which learners have to identify the reward-relevant dimension. Our behavioral and neural results reveal a cooperative update-within 300 ms after the outcome feedback, the state-transition functions are updated, followed by the beliefs for each dimension.

Pupil Constriction in the Glare Illusion Modulates the Steady-State Visual Evoked Potentials.

The glare illusion enhances the perceived brightness of a central white area surrounded by a luminance gradient, without any actual change in light intensity. In this study, we measured the varied brightness and neurophysiological responses of electroencephalography (EEG) and pupil size with the several luminance contrast patterns of the glare illusion to address the question of whether the illusory brightness changes to the glare illusion process in the early visual cortex. We hypothesized that if the illusory brightness enhancement was created in the early stages of visual processing, the neural response would be similar to how it processes an actual change in light intensity. To test this, we observed the sustained visual cortical response of steady-state visual evoked potentials (SSVEPs), while participants watched flickering dots displayed in the central white area of both the varied luminance contrast of glare illusion and a control stimulus (no glare condition). We found the SSVEP amplitude was lower in the glare illusion than in the control condition, especially under high luminance contrast conditions. Furthermore, we found the probable mechanisms of the inhibited SSVEP amplitude to the high luminance contrast of glare illusion based on the greater pupil constriction, thereby decreasing the amount of light entering the pupil. Thus, the brightness enhancement in the glare illusion is already represented at the primary stage of visual processing linked to the larger pupil constriction.

Colorful glares: Effects of colors on brightness illusions measured with pupillometry.

We hypothesized that pupil constrictions to the glare illusion, where converging luminance gradients subjectively enhance the perception of brightness, would be stronger for 'blue' than for other colors. Such an expectation was based on reflections about the ecology of vision, where the experience of dazzling light is common when one happens to look directly at sunlight through some occluders. Thus, we hypothesized that pupil constrictions to 'blue' reflect an ecologically-based expectation of the visual system from the experience of sky's light and color, which also leads to interpret the blue gradients of illusory glare to act as effective cues to impending probable intense light. We therefore manipulated the gradients color of glare illusions and measured changes in subjective brightness of identical shape stimuli. We confirmed that the blue resulted in what was subjectively evaluated as the brightest condition, despite all colored stimuli were equiluminant. This enhanced brightness effect was observed both in a psychophysical adjustment task and in changes in pupil size, where the maximum pupil constriction peak was observed with the 'blue' converging gradients over and above to the pupil response to blue in other conditions (i.e., diverging gradients and homogeneous patches). Moreover, glare-related pupil constrictions for each participant were correlated to each individual's subjective brightness adjustments. Homogenous blue hues also constricted the pupil more than other hues, which represents a pupillometric analog of the Helmholtz-Kohlrausch effect on brightness perception. Together, these findings show that pupillometry constitutes an easy tool to assess individual differences in color brightness perception.

Hemifield Crossings during Multiple Object Tracking Affect Task Performance and Steady-State Visual Evoked Potentials.

The ability to track multiple objects is important for daily life activities such as driving, but it is subject to some restrictions. One limitation concerns the hemifields in which objects move. A previous study showed that when subjects were restricted to the use of one hemifield, both the maximum number of tracked objects and the tracking accuracy were lower than when they were permitted to use both hemifields. However, daily life involves many tracked objects moving between hemifields. In this study, we investigated the effects of such hemifield crossings on behavioral performance (Behavioral experiment) and on the amplitudes and phase synchronization of steady-state visual evoked potentials (SSVEPs) (SSVEP experiment) by comparing the Within condition, in which tracked objects moved within their respective hemifields, and the Crossover condition, in which tracked objects moved between hemifields. In the Behavioral experiment, tracking performance was worse under the Crossover condition than under the Within condition. In the SSVEP experiment, SSVEP amplitudes for target and distractor frequencies differed under the Within condition but did not differ under the Crossover condition. However, phase synchronization between the left and right hemifields exhibited the opposite trend. This study provides evidence that attention to objects moving between hemifields is suppressed relative to attention to objects moving within hemifields and that Crossover tracking diminishes attentional modulation at an early sensory processing level while modulating interhemispheric functional connectivity.

Effects of Face and Background Color on Facial Expression Perception.

Detecting others' emotional states from their faces is an essential component of successful social interaction. However, the ability to perceive emotional expressions is reported to be modulated by a number of factors. We have previously found that facial color modulates the judgment of facial expression, while another study has shown that background color plays a modulatory role. Therefore, in this study, we directly compared the effects of face and background color on facial expression judgment within a single experiment. Fear-to-anger morphed faces were presented in face and background color conditions. Our results showed that judgments of facial expressions was influenced by both face and background color. However, facial color effects were significantly greater than background color effects, although the color saturation of faces was lower compared to background colors. These results suggest that facial color is intimately related to the judgment of facial expression, over and above the influence of simple color.

Association between pupil dilation and implicit processing prior to object recognition via insight.

Insight refers to the sudden conscious shift in the perception of a situation following a period of unconscious processing. The present study aimed to investigate the implicit neural mechanisms underlying insight-based recognition, and to determine the association between these mechanisms and the extent of pupil dilation. Participants were presented with ambiguous, transforming images comprised of dots, following which they were asked to state whether they recognized the object and their level of confidence in this statement. Changes in pupil dilation were not only characterized by the recognition state into the ambiguous object but were also associated with prior awareness of object recognition, regardless of meta-cognitive confidence. Our findings indicate that pupil dilation may represent the level of implicit integration between memory and visual processing, despite the lack of object awareness, and that this association may involve noradrenergic activity within the locus coeruleus-noradrenergic (LC-NA) system.