Differentiating facial incongruity and flatness in schizophrenia, using structured light camera data.

Incongruity between emotional experience and its outwardly expression is one of the prominent symptoms in schizophrenia. Though widely reported and used in clinical evaluation, this symptom is inadequately defined in the literature and may be confused with mere affect flattening. In this study we used structured-light depth camera and dedicated software to automatically measure facial activity of schizophrenia patients and healthy individuals during an emotionally evocative task. We defined novel measures for the congruence of emotional experience and emotional expression and for Flat Affect, compared them between patients and controls, and examined their consistency with clinical evaluation. We found incongruity in schizophrenia to be manifested in a less specific range of facial expressions in response to similar emotional stimuli, while the emotional experience remains intact. Our study also suggests that when taking into consideration affect flatness, no contextually inappropriate facial expressions are evident.

Beyond novelty detection: incongruent events, when general and specific classifiers disagree.

Unexpected stimuli are a challenge to any machine learning algorithm. Here, we identify distinct types of unexpected events when general-level and specific-level classifiers give conflicting predictions. We define a formal framework for the representation and processing of incongruent events: Starting from the notion of label hierarchy, we show how partial order on labels can be deduced from such hierarchies. For each event, we compute its probability in different ways, based on adjacent levels in the label hierarchy. An incongruent event is an event where the probability computed based on some more specific level is much smaller than the probability computed based on some more general level, leading to conflicting predictions. Algorithms are derived to detect incongruent events from different types of hierarchies, different applications, and a variety of data types. We present promising results for the detection of novel visual and audio objects, and new patterns of motion in video. We also discuss the detection of Out-Of- Vocabulary words in speech recognition, and the detection of incongruent events in a multimodal audiovisual scenario.

Differential category learning processes: The neural basis of comparison-based learning and induction.

Findings from numerous studies suggest that multiple neural systems are involved in category learning. Specifically, it is often argued that acquiring a representation of different category structures (e.g., rule-based vs. prototype-based representation) involves different computational challenges, which are resolved by different neural circuitries in the human brain. Here we present an alternative approach for studying neural mechanisms of category learning: We refer to the idea that any category learning task involves mapping common features shared by same-category members, distinctive features discriminating members of different categories, or both. We argue that since these processes are psychologically and computationally distinct, they differ in their usability for category learning. Our participants learned novel categories of complex visual stimuli by comparing either pairs of objects from the same novel category or pairs of objects from different categories. Object pairs were chosen so that the objective amount of information they contained was identical in the two category learning conditions, equally enabling learning the predefined objective category structure. We find that the neural circuitry involved in detecting important between-categories differences is associated mainly with the dorsal striatum (bilaterally) and the right hippocampus. On the other hand, mapping within-category similarities and differences is restricted to high-level visual brain areas. We suggest that multiple neural mechanisms are involved in category learning enabling us to face different computational challenges associated with different basic types of induction processes that differ in their usability for learning different category structures.

The development of category learning strategies: what makes the difference?

Category learning can be achieved by identifying common features among category members, distinctive features among non-members, or both. These processes are psychologically and computationally distinct, and may have implications for the acquisition of categories at different hierarchical levels. The present study examines an account of children's difficulty in acquiring categories at the subordinate level grounded on these distinct comparison processes. Adults and children performed category learning tasks in which they were exposed either to pairs of objects from the same novel category or pairs of objects from different categories. The objects were designed so that for each category learning task, two features determined category membership whereas two other features were task irrelevant. In the learning stage participants compared pairs of objects noted to be either from the same category or from different categories. Object pairs were chosen so that the objective amount of information provided to the participants was identical in the two learning conditions. We found that when presented only with object pairs noted to be from the same category, young children (6 < or = YO < or = 9.5) learned the novel categories just as well as older children (10 < or = YO < or = 14) and adults. However, when presented only with object pairs known to be from different categories, unlike older children and adults, young children failed to learn the novel categories. We discuss cognitive and computational factors that may give rise to this comparison bias, as well as its expected outcomes.

Category learning from equivalence constraints.

Information for category learning may be provided as positive or negative equivalence constraints (PEC/NEC)-indicating that some exemplars belong to the same or different categories. To investigate categorization strategies, we studied category learning from each type of constraint separately, using a simple rule-based task. We found that participants use PECs differently than NECs, even when these provide the same amount of information. With informative PECs, categorization was rapid, reasonably accurate and uniform across participants. With informative NECs, performance was rapid and highly accurate for only some participants. When given directions, all participants reached high-performance levels with NECs, but the use of PECs remained unchanged. These results suggest that people may use PECs intuitively, but not perfectly. In contrast, using informative NECs enables a potentially more accurate categorization strategy, but a less natural, one which many participants initially fail to implement-even in this simplified setting.

Comparison processes in category learning: from theory to behavior.

Recent studies stressed the importance of comparing exemplars both for improving performance by artificial classifiers as well as for explaining human category-learning strategies. In this report we provide a theoretical analysis for the usability of exemplar comparison for category-learning. We distinguish between two types of comparison -- comparison of exemplars identified to belong to the same category vs. comparison of exemplars identified to belong to two different categories. Our analysis suggests that these two types of comparison differ both qualitatively and quantitatively. In particular, in most everyday life scenarios, comparison of same-class exemplars will be far more informative than comparison of different-class exemplars. We also present behavioral findings suggesting that these properties of the two types of comparison shape the category-learning strategies that people implement. The predisposition for use of one strategy in preference to the other often results in a significant gap between the actual information content provided, and the way this information is eventually employed. These findings may further suggest under which conditions the reported category-learning biases may be overcome.

Motion segmentation and depth ordering using an occlusion detector.

We present a novel method for motion segmentation and depth ordering from a video sequence in general motion. We first compute motion segmentation based on differential properties of the spatio-temporal domain, and scale-space integration. Given a motion boundary, we describe two algorithms to determine depth ordering from two- and three- frame sequences. An remarkable characteristic of our method is its ability compute depth ordering from only two frames. The segmentation and depth ordering algorithms are shown to give good results on 6 real sequences taken in general motion. We use synthetic data to show robustness to high levels of noise and illumination changes; we also include cases where no intensity edge exists at the location of the motion boundary, or when no parametric motion model can describe the data. Finally, we describe human experiments showing that people, like our algorithm, can compute depth ordering from only two frames, even when the boundary between the layers is not visible in a single frame.

The distortion of reality perception in schizophrenia patients, as measured in Virtual Reality.

BACKGROUND: Virtual Reality is an interactive three-dimensional computer generated environment. Providing a complex and multi-modal environment, VR can be particularly useful for the study of complex cognitive functions and brain disorders. Here we used a VR world to measure the distortion in reality perception in schizophrenia patients. METHODS: 43 schizophrenia patients and 29 healthy controls navigated in a VR environment and were asked to detect incoherencies, such as a cat barking or a tree with red leaves. RESULTS: Whereas the healthy participants reliably detected incoherencies in the virtual experience, 88% of the patients failed in this task. The patients group had specific difficulty in the detection of audio-visual incoherencies; this was significantly correlated with the hallucinations score of the PANSS. CONCLUSIONS: By measuring the distortion in reality perception in schizophrenia patients, we demonstrated that Virtual Reality can serve as a powerful experimental tool to study complex cognitive processes.

Improving the accuracy of the diagnosis of schizophrenia by means of virtual reality.

OBJECTIVE: The authors' goal was to improve the diagnosis of schizophrenia by using virtual reality technology to build a complex, multimodal environment in which cognitive functions can be studied (and measured) in parallel. METHOD: The authors studied sensory integration within working memory by means of computer navigation through a virtual maze. The simulated journey consisted of a series of rooms, each of which included three doors. Each door was characterized by three features (color, shape, and sound), and a single combination of features--the door-opening rule--was correct. Subjects had to learn the rule and use it. The participants were 39 schizophrenic patients and 21 healthy comparison subjects. RESULTS: Upon completion, each subject was assigned a performance profile, including various error scores, response time, navigation ability, and strategy. A classification procedure based on the subjects' performance profile correctly predicted 85% of the schizophrenic patients (and all of the comparison subjects). Several performance variables showed significant correlations with scores on a standard diagnostic measure (Positive and Negative Syndrome Scale), suggesting potential use of these measurements for the diagnosis of schizophrenia. On the other hand, the patients did not show unusual repetition of response despite stimulus cessation (called "perseveration" in classical studies of schizophrenia), which is a common symptom of the disease. This deficit appeared only when the subjects did not receive proper explanation of the task. CONCLUSIONS: The ability to study multimodal performance simultaneously by using virtual reality technology opens new possibilities for the diagnosis of schizophrenia with objective procedures.

Virtual reality testing of multi-modal integration in schizophrenic patients.

Our goal is to develop a new family of automatic tools for the diagnosis of schizophrenia, using Virtual Reality Technology (VRT). VRT is specifically suitable for this purpose, because it allows for multi-modal stimulation in a complex setup, and the simultaneous measurement of multiple parameters. In this work we studied sensory integration within working memory, in a navigation task through a VR maze. Along the way subjects pass through multiple rooms that include three doors each, only one of which can be used to legally exit the room. Specifically, each door is characterized by three features (color, shape and sound), and only one combination of features -- as determined by a transient opening rule -- is legal. The opening rule changes over time. Subjects must learn the rule and use it for successful navigation throughout the maze. 39 schizophrenic patients and 21 healthy controls participated in this study. Upon completion, each subject was assigned a performance profile, including various error scores, response time, navigation ability and strategy. We developed a classification procedure based on the subjects' performance profile, which correctly predicted 85% of the schizophrenic patients (and all the controls). We observed that a number of parameters showed significant correlation with standard diagnosis scores (PANSS), suggesting the potential use of our measurements for future diagnosis of schizophrenia. On the other hand, our patients did not show unusual repetition of response despite stimulus cessation (called perseveration in classical studies of schizophrenia), which is usually considered a robust marker of the disease. Interestingly, this deficit only appeared in our study when subjects did not receive proper explanation of the task.