Impact of Personalized Avatars and Motion Synchrony on Embodiment and Users' Subjective Experience: Empirical Study.

BACKGROUND: Embodiment through a virtual avatar is a key element for people to feel that they are in the virtual world. OBJECTIVE: This study aimed to elucidate the interaction between 2 methods of eliciting embodiment through a virtual avatar: motion synchronization and appearance similarity between a human and avatar, to understand embodiment (agency, body ownership, and self-location) and subjective experience (presence, simulator sickness, and emotion) in virtual reality. METHODS: Using a full-body motion capture system, 24 participants experienced their virtual avatars with a 3D-scanned face and size-matched body from a first-person perspective. This study used a 2 (motion; sync and async) × 2 (appearance; personalized and generic) within-subject design. RESULTS: The results indicated that agency and body ownership increased when motion and appearance were matched, whereas self-location, presence, and emotion were affected by motion only. Interestingly, if the avatar's appearance was similar to the participants (personalized avatar), they formed an agency toward the avatar's motion that was not performed by themselves. CONCLUSIONS: Our findings would be applicable in the field of behavioral therapy, rehabilitation, and entertainment applications, by eliciting higher agency with a personalized avatar.

Measuring recognition of body changes over time: A human-computer interaction tool using dynamic morphing and body ownership illusion.

Measuring body image is crucial at both personal and social levels. Previous studies have attempted to quantitatively measure body image but methods for measuring body change recognition over time have not yet been established. The present study proposes a novel human-computer interaction technique using dynamic morphing and body ownership illusion, and we conducted a user study to investigate how body ownership illusion and gender would affect to body change recognition. The results showed that a participant's body change recognition was weak when the body ownership illusion was strong. In addition, female participants were less sensitive than male participants. With our proposed technique, we demonstrated that we were able to quantitatively measure body change recognition and our empirical data indicated that body change recognition varied depending on body ownership illusion and gender, suggesting that our methodology could not only be used in future body image studies but also in eating disorder treatments.

Load-sensitive impairment of working memory for biological motion in schizophrenia.

Impaired working memory (WM) is a core cognitive deficit in schizophrenia. Nevertheless, past studies have reported that patients may also benefit from increasing salience of memory stimuli. Such efficient encoding largely depends upon precise perception. Thus an investigation on the relationship between perceptual processing and WM would be worthwhile. Here, we used biological motion (BM), a socially relevant stimulus that schizophrenics have difficulty discriminating from similar meaningless motions, in a delayed-response task. Non-BM stimuli and static polygons were also used for comparison. In each trial, one of the three types of stimuli was presented followed by two probes, with a short delay in between. Participants were asked to indicate whether one of them was identical to the memory item or both were novel. The number of memory items was one or two. Healthy controls were more accurate in recognizing BM than non-BM regardless of memory loads. Patients with schizophrenia exhibited similar accuracy patterns to those of controls in the Load 1 condition only. These results suggest that information contained in BM could facilitate WM encoding in general, but the effect is vulnerable to the increase of cognitive load in schizophrenia, implying inefficient encoding driven by imprecise perception.

Facilitating Effects of Emotion on the Perception of Biological Motion: Evidence for a Happiness Superiority Effect.

It has been reported that visual perception can be influenced not only by the physical features of a stimulus but also by the emotional valence of the stimulus, even without explicit emotion recognition. Some previous studies reported an anger superiority effect while others found a happiness superiority effect during visual perception. It thus remains unclear as to which emotion is more influential. In the present study, we conducted two experiments using biological motion (BM) stimuli to examine whether emotional valence of the stimuli would affect BM perception; and if so, whether a specific type of emotion is associated with a superiority effect. Point-light walkers with three emotion types (anger, happiness, and neutral) were used, and the threshold to detect BM within noise was measured in Experiment 1. Participants showed higher performance in detecting happy walkers compared with the angry and neutral walkers. Follow-up motion velocity analysis revealed that physical difference among the stimuli was not the main factor causing the effect. The results of the emotion recognition task in Experiment 2 also showed a happiness superiority effect, as in Experiment 1. These results show that emotional valence (happiness) of the stimuli can facilitate the processing of BM.

Biological Motion Perception, Brain Responses, and Schizotypal Personality Disorder.

IMPORTANCE: Exploration of the ability to process socially relevant events portrayed by biological motion and to identify underlying neuronal processes can provide clues for understanding the pathophysiology of psychosis. Individuals with schizotypal personality disorder (SPD) have pervasive interpersonal deficits and odd behaviors. An understanding of the neural mechanisms involved in the perception of biological motion and the relation of activity to clinical symptoms in those mechanisms is needed. OBJECTIVE: To investigate the specificity of brain regions responsive to biological motion perception in individuals with SPD compared with healthy control individuals. DESIGN, SETTING, AND PARTICIPANTS: Twenty-one patients diagnosed as having SPD and 38 age-, sex-, and IQ-matched controls underwent event-related functional magnetic resonance imaging. The SPD group completed the Scale for the Assessment of Positive Symptoms, the Scale for the Assessment of Negative Symptoms, and the Schizotypal Personality Questionnaire for assessment of symptom severity. During scanning, all participants were required to discriminate biological from scrambled sequences of point-light animations. Data were collected from September 21. 2011, to July 13, 2013, and analyzed from March to May 2015. MAIN OUTCOMES AND MEASURES: Blood oxygenation level-dependent signals during event-related scanning and symptom severity in the SPD group. RESULTS: The 21 individuals with SPD (16 men and 5 women) and 38 controls (29 men and 9 women) had a mean (SD) age, 22.8 (3.8) vs 22.2 (2.5) years and a mean (SD) IQ, 115.00 (12.55) vs 120.24 )7.68). Brain imaging revealed the presence of neuronal activation specific to biological motion within the posterior superior temporal sulcus. However, the individuals with SPD exhibited regions of neural responsiveness within brain regions forming the reward network, which consisted of the dorsal striatum and bilateral superior medial frontal cortex (all t ≥ 2.99, P of clusters <.002). The individuals with SPD also exhibited reduced activation in the anterior and middle cingulate cortices and the lingual and superior occipital gyri, which are brain areas responsive to biological motion perception and executive control of perception (all t ≥ 3.29, P of clusters <.001). In addition, significant correlations between the hyperdopaminergic clinical symptoms and enhanced neuronal activation in the caudate nucleus and frontal cortex were observed in the SPD group (all r ≥ 0.52, P < .02). CONCLUSIONS AND RELEVANCE: Individuals with SPD display heightened activation in the neural circuitry involved in reward and decision making when viewing biological motion stimuli in addition to a positive correlation between increased blood oxygenation level-dependent signal responses related to biological motions and clinical symptoms. These findings suggest that enhanced responses arise within the reward network in individuals with SPD and are possibly related to the peculiar ways that individuals with SPD behave in social contexts.

A new technique for generating disordered point-light animations for the study of biological motion perception.

Studies of biological motion perception often use stimuli depicting human actions portrayed via point-light (PL) displays. Typically, counterpart, or control, stimuli for PL biological motion are created by spatially scrambling motion trajectories of individual PL dots. Depending on the purpose of the study, however, this procedure may be inappropriate as a foil for genuine PL animations, because spatial scrambling not only disrupts coherent motion activity but also eliminates pair-wise motion relationships among dots and, unless corrected, alters the spatial spread of PL dot motions. We introduce a new technique for producing perturbed PL animations, called pair-wise shuffled motion, that preserves the elementary features of biological motion in spatial and motion energy domains and only disrupts the specific sense of global, coherent perception of biological motion. First we describe the procedure for creating pair-wise shuffled motion sequences. Next we compare unperturbed PL animations to pair-wise shuffled motion, to spatially scrambled motion, and to spatially constrained scrambled motion in terms of spatial distributions of the dots, spatiotemporal amplitude spectra derived from Fourier analysis of those sequences, and space-time motion energy associated with those perturbed animations. We then show that the results from those analyses generalize to a large family of PL animations, including the widely used PL walker. Finally we present results from a two-interval forced-choice biological-motion discrimination experiment comparing the robustness of scrambled and pair-wise shuffled motions as foil stimuli. Results from these comparisons suggest that pair-wise shuffled motion offers advantages as a foil stimulus compared to foils using the conventional scrambling technique. Pair-wise shuffled motion provides an additional, effective control display for evaluating PL biological motion perception in future psychophysical, computational, and imaging studies that focus on mechanisms of processing spatiotemporal information signifying biological motion within PL displays.

Failure to benefit from target novelty during encoding contributes to working memory deficits in schizophrenia.

INTRODUCTION: Although working memory (WM) impairments are well documented in schizophrenic patients (PSZ), the underlying mechanisms are poorly understood. The aim of this study was to investigate the role of target salience during encoding to determine whether impaired visual attention in PSZ leads to poor WM. METHODS: Thirty-one PSZ and 28 demographically matched healthy controls (HC) performed a spatial delayed-response task. Attentional demands were manipulated during WM encoding by presenting high salient (novel) or low salient (familiar) targets. Participants also rated their level of response confidence at the end of each trial, allowing us to analyse different response types. RESULTS: WM was impaired in PSZ. Increasing target salience by increasing novelty improved WM performance in HC but not in PSZ. Poor WM performance in PSZ was largely due to an increase in the proportion of incorrect but high confident responses most likely reflecting a failure to encode the correct target. CONCLUSIONS: Our findings suggest that dysfunctions of non-mnemonic attentional processes during encoding contribute to WM impairments in schizophrenia and may represent an important target for cognitive remediation strategies.

Deficient biological motion perception in schizophrenia: results from a motion noise paradigm.

BACKGROUND: Schizophrenia patients exhibit deficient processing of perceptual and cognitive information. However, it is not well-understood how basic perceptual deficits contribute to higher level cognitive problems in this mental disorder. Perception of biological motion, a motion-based cognitive recognition task, relies on both basic visual motion processing and social cognitive processing, thus providing a useful paradigm to evaluate the potentially hierarchical relationship between these two levels of information processing. METHODS: In this study, we designed a biological motion paradigm in which basic visual motion signals were manipulated systematically by incorporating different levels of motion noise. We measured the performances of schizophrenia patients (n = 21) and healthy controls (n = 22) in this biological motion perception task, as well as in coherent motion detection, theory of mind, and a widely used biological motion recognition task. RESULTS: Schizophrenia patients performed the biological motion perception task with significantly lower accuracy than healthy controls when perceptual signals were moderately degraded by noise. A more substantial degradation of perceptual signals, through using additional noise, impaired biological motion perception in both groups. Performance levels on biological motion recognition, coherent motion detection and theory of mind tasks were also reduced in patients. CONCLUSION: The results from the motion-noise biological motion paradigm indicate that in the presence of visual motion noise, the processing of biological motion information in schizophrenia is deficient. Combined with the results of poor basic visual motion perception (coherent motion task) and biological motion recognition, the association between basic motion signals and biological motion perception suggests a need to incorporate the improvement of visual motion perception in social cognitive remediation.

Changes in effective connectivity according to working memory load: an FMRI study of face and location working memory tasks.

OBJECTIVE: The functional strategic mechanisms in the brain during performing visuospatial working memory tasks, especially tasks with heavy load, are controversial. We conducted the functional magnetic resonance imaging (fMRI) while sixteen subjects were performing face- and location-matching n-back tasks to examine causal relations within the frontoparietal networks. METHODS: We applied a sophisticated method, the structural equation modeling (SEM), to the fMRI data. The imaging data were analyzed by extracting the task-related eigenseries using the principal component analysis (PCA) and then by applying a form of data-driven model called the automated search method. RESULTS: The SEM analyses revealed a functional shift of network connectivity from the right to the left hemisphere with increasing load in the face-matching n-back tasks while the location-matching tasks required bilateral activation. In the locating matching n-back tasks, a pattern of parallel processing was observed in the left phonological loop and the right inferior parietal regions. Furthermore, object working memory-related activities in the left hemisphere reliably contributed to performance of both the face- and location-matching 2-back tasks. CONCLUSION: Our results are consistent with previous reports in terms of demonstrating parallel and distributed information processing during performing working memory tasks with heavy loads. Our results additionally suggest a dynamic shift between the fast imagery circuit (right hemisphere) and the stable verbal circuit (left hemisphere), depending on task load.

Perception of biological motion in schizophrenia and healthy individuals: a behavioral and FMRI study.

BACKGROUND: Anomalous visual perception is a common feature of schizophrenia plausibly associated with impaired social cognition that, in turn, could affect social behavior. Past research suggests impairment in biological motion perception in schizophrenia. Behavioral and functional magnetic resonance imaging (fMRI) experiments were conducted to verify the existence of this impairment, to clarify its perceptual basis, and to identify accompanying neural concomitants of those deficits. METHODOLOGY/FINDINGS: In Experiment 1, we measured ability to detect biological motion portrayed by point-light animations embedded within masking noise. Experiment 2 measured discrimination accuracy for pairs of point-light biological motion sequences differing in the degree of perturbation of the kinematics portrayed in those sequences. Experiment 3 measured BOLD signals using event-related fMRI during a biological motion categorization task. Compared to healthy individuals, schizophrenia patients performed significantly worse on both the detection (Experiment 1) and discrimination (Experiment 2) tasks. Consistent with the behavioral results, the fMRI study revealed that healthy individuals exhibited strong activation to biological motion, but not to scrambled motion in the posterior portion of the superior temporal sulcus (STSp). Interestingly, strong STSp activation was also observed for scrambled or partially scrambled motion when the healthy participants perceived it as normal biological motion. On the other hand, STSp activation in schizophrenia patients was not selective to biological or scrambled motion. CONCLUSION: Schizophrenia is accompanied by difficulties discriminating biological from non-biological motion, and associated with those difficulties are altered patterns of neural responses within brain area STSp. The perceptual deficits exhibited by schizophrenia patients may be an exaggerated manifestation of neural events within STSp associated with perceptual errors made by healthy observers on these same tasks. The present findings fit within the context of theories of delusion involving perceptual and cognitive processes.

Reduced cognitive control of a visually bistable image in schizophrenia.

Schizophrenia is associated with the inability to control and coordinate thoughts, actions, and perceptions. In conventional assessments of cognitive control, multiple sensory features of stimuli are concomitantly manipulated, introducing a confounding role of bottom-up perceptual information. To overcome this difficulty, we used an ambiguous visual stimulus (Necker cube), which allowed measurement of cognitive control with constant sensory input. Subjects (20 patients, 20 controls) were asked to control their perception of a transparent Necker cube by keeping a designated plane at the front or back of the stimulus, the position of which is perceptually bistable. Patients were highly deficient at controlling their perception of the cube. When a visual feature (the luminance contrast between a designated cube plane and the other planes) was systematically manipulated, an interaction was found whereby schizophrenia patients no longer under-performed on the highest contrast condition. These results show patients' impairment of controlling perception in the absence of visual modulation and suggest the potential utility of perceptually based approaches to cognitive remediation in schizophrenia.

Enhancing visual working memory encoding: The role of target novelty.

Perceptual salience improves the encoding of information into visual working memory (WM). However, the factors that contribute to this facilitation effect are not well understood. This study tested the influence of target familiarity on WM encoding. In each trial, participants were presented with either one or three targets and asked to encode their locations into WM. In Experiment 1, target familiarity was manipulated by presenting either an upright (familiar target) or upside-down (unfamiliar/novel target) A. Increasing the novelty of the targets led to improved performance in the spatial WM task. Experiment 2 showed that participants were faster in responding to novel versus familiar targets in a spatial detection task. Experiment 3 demonstrated that the beneficial effect of target novelty on WM encoding was not driven by differences in low-level features. Our results suggest that target novelty enhances the processes required for WM encoding, just as it facilitates perceptual processing.

An event-related FMRI study of phonological verbal working memory in schizophrenia.

BACKGROUND: While much is known about the role of prefrontal cortex (PFC) in working memory (WM) deficits of schizophrenia, the nature of the relationship between cognitive components of WM and brain activation patterns remains unclear. We aimed to elucidate the neural correlates of the maintenance component of verbal WM by examining correct and error trials with event-related fMRI. METHODOLOGY/FINDINGS: Twelve schizophrenia patients (SZ) and thirteen healthy control participants (CO) performed a phonological delayed-matching-to-sample-task in which a memory set of three nonsense words was presented, followed by a 6-seconds delay after which a probe nonsense word appeared. Participants decided whether the probe matched one of the targets, and rated the confidence of their decision. Blood-oxygen-level-dependent (BOLD) activity during WM maintenance was analyzed in relation to performance (correct/error) and confidence ratings. Frontal and parietal regions exhibited increased activation on correct trials for both groups. Correct and error trials were further segregated into true memory, false memory, guess, and true error trials. True memory trials were associated with increased bilateral activation of frontal and parietal regions in both groups but only CO showed deactivation in PFC. There was very little maintenance-related cortical activity during guess trials. False memory was associated with increased left frontal and parietal activation in both groups. CONCLUSION: These findings suggest that a wider network of frontal and parietal regions support WM maintenance in correct trials compared with error trials in both groups. Furthermore, a more extensive and dynamic pattern of recruitment of the frontal and parietal networks for true memory was observed in healthy controls compared with schizophrenia patients. These results underscore the value of parsing the sources of memory errors in fMRI studies because of the non-linear nature of the brain-behavior relationship, and suggest that group comparisons need to be interpreted in more specific behavioral contexts.

Selective impairment in visual perception of biological motion in obsessive-compulsive disorder.

Obsessive-compulsive disorder (OCD) is associated with a variety of well-documented cognitive deficits such as deficits in memory and executive functioning, but little is known about basic perceptual concomitants of OCD. This study investigated global, configural processing in OCD using dynamic (moving) and static stimuli with minimal demands on cognitive function. Twenty OCD patients and 16 age- and education-matched healthy control subjects were tested on four perceptual tasks: two motion tasks involved detection and discrimination of human activity portrayed by point-light animations ("biological" motion). The other two tasks involved detection of coherent, translational motion defined by random-dot cinematograms and detection of static global shape defined by spatially distributed contours. OCD patients exhibited impaired performance on biological motion tasks; in contrast, their performance on tasks of coherent motion detection and global form perception were comparable to those of healthy controls. These results indicate that OCD patients have a specific deficit in perceiving biological motion signals, whereas their perception of non-biological coherent motion and static global shape is intact. Because efficient social interactions depend on accurate and rapid perception of subtle socially relevant cues, deficits in biological motion perception may compromise social functioning in people with OCD.

Weakened center-surround interactions in visual motion processing in schizophrenia.

Schizophrenia is often accompanied by a range of visual perception deficits, with many involving impairments in motion perception. The presence of perceptual abnormalities may impair neural processes that depend on normal visual analysis, which in turn may affect overall functioning in dynamic visual environments. Here, we examine the integrity of suppressive center-surround mechanisms in motion perception of schizophrenic patients. Center-surround suppression has been implicated in a range of visual functions, including figure-ground segregation and pursuit eye movements, visual functions that are impaired in schizophrenia. In control subjects, evidence of center-surround suppression is found in a reduced ability to perceive motion of a high-contrast stimulus as its size increases. This counterintuitive finding is likely a perceptual correlate of center-surround mechanisms in cortical area MT. We now show that schizophrenic patients exhibit abnormally weak center-surround suppression in motion, an abnormality that is most pronounced in patients with severe negative symptoms. Interestingly, patients with the weakest surround suppression outperformed control subjects in motion discriminations of large high-contrast stimuli. This enhanced motion perception of large high-contrast stimuli is consistent with an MT abnormality in schizophrenia and has a potential to disrupt smooth pursuit eye movements and other visual functions that depend on unimpaired center-surround interactions in motion.

Self-initiated encoding facilitates object working memory in schizophrenia: implications for the etiology of working memory deficit.

BACKGROUND: Working memory (WM) deficit is present in a majority of patients with schizophrenia but it is unclear which components of WM are impaired. Past studies suggest that encoding may be compromised. One important determinant of encoding is the deployment of selective attention to the target stimulus. In addition, attention and encoding are modulated by motivational factors. In this study, we investigated the effects of self-initiated encoding (i.e., voluntary attention) on WM. METHODS: 19 patients with schizophrenia and 19 matched control subjects participated in visual WM and control tasks. Encoding was manipulated by asking subjects to select from two face targets and memorize 1) one of the two identical faces (Non-preference condition), 2) one that is marked (Non-choice condition), and 3) one they prefer (Preference condition). WM accuracy for both location (spatial) and identity (object) was measured. RESULTS: Overall, patients with schizophrenia were less accurate and slower than the control subjects but the deficit was greater for object WM. However, patients were more accurate in object WM when they selected a preferred face as their target during encoding (preference condition) compared with the other two conditions. This effect was not significant for spatial WM. CONCLUSIONS: These results suggest that voluntary, self-initiated attention may facilitate object encoding especially if the selection of the target involves affective choice, and that attention may play different roles in encoding 'what' versus 'where' in WM. Since encoding affects all forms of memory, these results may have a more general implication for memory.

Impaired visual recognition of biological motion in schizophrenia.

BACKGROUND: Motion perception deficits have been suggested to be an important feature of schizophrenia but the behavioral consequences of such deficits are unknown. Biological motion refers to the movements generated by living beings. The human visual system rapidly and effortlessly detects and extracts socially relevant information from biological motion. A deficit in biological motion perception may have significant consequences for detecting and interpreting social information. METHODS: Schizophrenia patients and matched healthy controls were tested on two visual tasks: recognition of human activity portrayed in point-light animations (biological motion task) and a perceptual control task involving detection of a grouped figure against the background noise (global-form task). Both tasks required detection of a global form against background noise but only the biological motion task required the extraction of motion-related information. RESULTS: Schizophrenia patients performed as well as the controls in the global-form task, but were significantly impaired on the biological motion task. In addition, deficits in biological motion perception correlated with impaired social functioning as measured by the Zigler social competence scale [Zigler, E., Levine, J. (1981). Premorbid competence in schizophrenia: what is being measured? Journal of Consulting and Clinical Psychology, 49, 96-105.]. CONCLUSION: The deficit in biological motion processing, which may be related to the previously documented deficit in global motion processing, could contribute to abnormal social functioning in schizophrenia.