Bayes factor of model selection validates FLMP.

The fuzzy logical model of perception (FLMP; Massaro, 1998) has been extremely successful at describing performance across a wide range of ecological domains as well as for a broad spectrum of individuals. An important issue is whether this descriptive ability is theoretically informative or whether it simply reflects the model's ability to describe a wider range of possible outcomes. Previous tests and contrasts of this model with others have been adjudicated on the basis of both a root mean square deviation (RMSD) for goodness-of-fit and an observed RMSD relative to a benchmark RMSD if the model was indeed correct. We extend the model evaluation by another technique called Bayes factor (Kass & Raftery, 1995; Myung & Pitt, 1997). The FLMP maintains its significant descriptive advantage with this new criterion. In a series of simulations, the RMSD also accurately recovers the correct model under actual experimental conditions. When additional variability was added to the results, the models continued to be recoverable. In addition to its descriptive accuracy, RMSD should not be ignored in model testing because it can be justified theoretically and provides a direct and meaningful index of goodness-of-fit. We also make the case for the necessity of free parameters in model testing. Finally, using Newton's law of universal gravitation as an analogy, we argue that it might not be valid to expect a model's fit to be invariant across the whole range of possible parameter values for the model. We advocate that model selection should be analogous to perceptual judgment, which is characterized by the optimal use of multiple sources of information (e.g., the FLMP). Conclusions about models should be based on several selection criteria.

Modeling face identification processing in children and adults.

Two face identification experiments were carried out to study whether and how children (5-year-olds) and adults integrate single facial features to identify faces. Using the paradigm of the Fuzzy Logical Model of Perception each experiment used the same expanded factorial design, with three levels of eyes variations crossed with three levels of mouth variations as well as their corresponding half-face conditions. In Experiment 1, an integration of facial features was observed in adults only. But, in adjusting the salience of the features varied, the results of Experiment 2 indicate that children and adults evaluated and integrated information from both features to identify a face. A weighted Fuzzy Logical Model of Perception fit the judgments significantly better than a Single Channel Model and questions previous claims of holistic face processing. Although no developmental differences in the stage of the integration of facial information were observable, differences between children and adults appeared in the information used for face identification.

Tests of auditory-visual integration efficiency within the framework of the fuzzy logical model of perception.

The fuzzy logical model of perception [FLMP, Massaro, Perceiving Talking Faces: From Speech Perception to a Behavioral Principle (MIT Press, Cambridge, MA, 1998)] has been extremely successful at describing performance across a wide range of ecological domains as well as for a broad spectrum of individuals. Because the model predicts optimal or maximally efficient integration, an important issue is whether this is the case for most individuals. Three databases are evaluated to determine to what extent a significant quantitative improvement in predictive ability can be obtained if integration is assumed to be somewhat inefficient. For the most part, there were no significant signs of inefficient integration. The previous differences found by Grant and Seitz [J. Acoust. Soc. Am. 104, 2438-2450 (1998)] must be due to their measures of efficiency, which appear to be invalid and/or conflate information with integration efficiency. Finally, the descriptive ability of the FLMP is shown to be theoretically informative and not simply the model's ability to describe any possible outcome.

Speech perception in perceivers with hearing loss: synergy of multiple modalities.

Although we would expect that the role of visible speech in multimodal speech perception would have the greatest relevance to individuals with hearing impairment, relatively few analytic studies have been done with these participants. Their adequate understanding of spoken language usually requires information from several modalities or other sources of information. The framework of the fuzzy logical model of perception (FLMP) is used to assess how individuals with hearing impairment evaluate and integrate multiple sources of information. Given this framework, a distinction can be made between information and information processing. Within this framework, we can ask what information differences and information-processing differences exist among individuals with normal hearing and those with hearing impairment. Four experimental studies from the literature are analyzed to address these questions. Test items are presented under both unimodal and bimodal conditions. Of central interest is the nature of the bimodal performance as a function of the unimodal performance. The findings show that, although information differences obviously exist across different populations, their information processing involved in pattern recognition appears to be the same and is well described by the FLMP.

Laterality in visual speech perception.

The lateralization of visual speech perception was examined in 3 experiments. Participants were presented with a realistic computer-animated face articulating 1 of 4 consonant-vowel syllables without sound. The face appeared at 1 of 5 locations in the visual field. The participants' task was to identify each test syllable. To prevent eye movement during the presentation of the face, participants had to carry out a fixation task simultaneously with the speechreading task. In one study, an eccentricity effect was found along with a small but significant difference in favor of the right visual field (left hemisphere). The same results were found with the face articulating nonlinguistic mouth movements (e.g., kiss). These results suggest that the left-hemisphere advantage is based on the processing of dynamic visual information rather than on the extraction of linguistic significance from facial movements.

Featural evaluation, integration, and judgment of facial affect.

The paradigm of the fuzzy logical model of perception (FLMP) is extended to the domain of perception and recognition of facial affect. Two experiments were performed using a highly realistic computer-generated face varying on 2 features of facial affect. Each experiment used the same expanded factorial design, with 5 levels of brow deflection crossed with 5 levels of mouth deflection, as well as their corresponding half-face conditions, for a total stimulus set of 35 faces. Experiment 1 used a 2-alternative, forced-choice paradigm (either happy or angry), whereas Experiment 2 used 9 rating steps from happy to angry. Results indicate that participants evaluated and integrated information from both features to perceive affective expressions. Both choice probabilities and ratings showed that the influence of 1 feature was greater to the extent that the other feature was ambiguous. The FLMP fit the judgments from both experiments significantly better than an additive model. Our results question previous claims of categorical and holistic perception of affect.

Perception of visible speech: influence of spatial quantization.

Visible speech reading was studied to determine which features are functional and to test several models of pattern recognition. Nine test syllables differing in their initial consonant were presented in intact form or under various levels of spatial quantization. Performance decreased in increasing quantization but remained relatively good at moderate levels of degradation. Different models were tested against the confusion matrices. Six features were identified as functional in distinguishing among the nine consonant--vowel syllables. These features were used as sources of information in a fuzzy-logical model of perception and an additive model. The fuzzy-logical model provided a significantly better description of the confusion matrices, showing that speech reading is analogous to other domains of pattern recognition such as face recognition and facial-affect perception.

Perceptual recognition of facial affect: cross-cultural comparisons.

Previous research has shown that the perception of affect in faces is well described by the fuzzy logical model of perception (FLMP). In this study, we asked whether the processes involved in recognition depended on the race/culture of the face and/or of the perceiver. A computer-generated face was used to manipulate two features of facial affect: brow deflection and mouth deflection. An expanded-factorial design was used, with four levels of brow deflection crossed with four levels of mouth deflection, as well as their corresponding half-face conditions. Participants identified these faces as either happy or angry. Japanese and U.S. students were tested on faces from these two countries that were texture-mapped onto the animated face. The FLMP gave the best description of performance for both groups and for both types of faces. These findings challenge previous claims of holistic perception, categorical perception, and additive feature integration.

Perceiving speech from inverted faces.

We examined whether the orientation of the face influences speech perception in face-to-face communication. Participants identified auditory syllables, visible syllables, and bimodal syllables presented in an expanded factorial design. The syllables were /ba/, /va/, /oa/, or /da/. The auditory syllables were taken from natural speech whereas the visible syllables were produced by computer animation of a realistic talking face. The animated face was presented either as viewed in normal upright orientation or inverted orientation (180 degrees frontal rotation). The central intent of the study was to determine if an inverted view of the face would change the nature of processing bimodal speech or simply influence the information available in visible speech. The results with both the upright and inverted face views were adequately described by the fuzzy logical model of perception (FLMP). The observed differences in the FLMP's parameter values corresponding to the visual information indicate that inverting the view of the face influences the amount of visible information but does not change the nature of the information processing in bimodal speech perception.

Perception of asynchronous and conflicting visual and auditory speech.

Previous research has shown that perceivers naturally integrate auditory and visual information in face-to-face speech perception. Two experiments were carried out to study whether integration would be disrupted by differences in the stimulus onset asynchrony (SOA), the temporal arrival of the two sources of information. Synthetic visible and natural and synthetic auditory syllables off/ba/, /va/, [symbol: see text]a/, and /da/ were used in an expanded factorial design to present all possible combinations of the auditory and visual syllables, as well as the unimodal syllables. The fuzzy logical model of perception (FLMP), which accurately describes integration, was used to measure the degree to which integration of audible and visible speech occurred. These findings provide information about the temporal window of integration and its apparent dependence on the range of speech events in the test.

Integration of orthographic, conceptual, and episodic information on implicit and explicit tests.

An experiment was conducted to determine how orthographic and conceptual information are integrated during incidental and intentional retrieval. Subjects studied word lists with either a shallow (counting vowels) or deep (rating pleasantness) processing task, then received either an implicit or explicit word fragment completion (WFC) test. At test, word fragments contained 0, 1, 2, or 4 letters, and were accompanied by 0, 1, 2, or 3 semantically related words. On both the implicit and explicit tests, performance improved with increases in the numbers of letters and words. When semantic cues were presented with the word fragments, the implicit test became more conceptually drive. Still, conceptual processing had a larger effect in intentional than in incidental retrieval. The Fuzzy Logical Model of Perception (FLMP) provided a good description of how orthographic, semantic, and episodic information were combined during retrieval.

Perceiving affect from the voice and the face.

This experiment examines how emotion is perceived by using facial and vocal cues of a speaker. Three levels of facial affect were presented using a computer-generated face. Three levels of vocal affect were obtained by recording the voice of a male amateur actor who spoke a semantically neutral word in different simulated emotional states. These two independent variables were presented to subjects in all possible permutations-visual cues alone, vocal cues alone, and visual and vocal cues together-which gave a total set of 15 stimuli. The subjects were asked to judge the emotion of the stimuli in a two-alternative forced choice task (either HAPPY or ANGRY). The results indicate that subjects evaluate and integrate information from both modalities to perceive emotion. The influence of one modality was greater to the extent that the other was ambiguous (neutral). The fuzzy logical model of perception (FLMP) fit the judgments significantly better than an additive model, which weakens theories based on an additive combination of modalities, categorical perception, and influence from only a single modality.

Continuous versus discrete information processing in pattern recognition.

A discrete feature model (DFM) and the fuzzy logical model (FLMP) were formulated to predict the distribution of rating judgments in a pattern recognition task. The distinction was between the spoken vowels /i/ and /I/, as in beet and bit. Subjects were instructed to rate the vowel on a nine-point scale from /i/ to /I/. Two features, the first formant frequency (F1) and the vowel duration, were orthogonally varied: The vowel /i/ has a lower (F1) and a longer duration compared to a somewhat higher (F1) and shorter duration for /I/. The DFM predicts that the separate features are recognized discretely, whereas the FLMP assumes that continuous information is available about each feature. Tests of these models on the observed data indicated that the continuous information assumption of the FLMP gave a significantly better description of the distribution of rating judgments.

Independence of lexical context and phonological information in speech perception.

M. A. Pitt (1995a) studied the joint influence of phonological information and lexical context in W. F. Ganong's (1980) task. Pitt improved on earlier studies by collecting enough observations to make possible the quantitative analyses of an individual's data. The present article shows that the results of such analyses demonstrate that the integration of phonological information and lexical context is very well accounted for by the fuzzy logical model of perception (FLMP). Although Pitt concluded that the results of his research argued against the FLMP in favor of an interactive feedback system, his conclusion was based on an analysis of transformed results. It is argued that this use of a response transformation led to incorrect conclusions and that ultimately, models must be tested directly against observed behavior.

Cross-linguistic comparisons in the integration of visual and auditory speech.

We examined how speakers of different languages perceive speech in face-to-face communication. These speakers identified synthetic unimodal and bimodal speech syllables made from synthetic auditory and visual five-step /ba/-/da/ continua. In the first experiment, Dutch speakers identified the test syllables as either /ba/ or /da/. To explore the robustness of the results, Dutch and English speakers were given a completely open-ended response task. Tasks in previous studies had always specified a set of alternatives. Similar results were found in the two-alternative and open-ended task. Identification of the speech segments was influenced by both the auditory and the visual sources of information. The results falsified an auditory dominance model (ADM) which assumes that the contribution of visible speech is dependent on poor-quality audible speech. The results also falsified an additive model of perception (AMP) in which the auditory and visual sources are linearly combined. The fuzzy logical model of perception (FLMP) provided a good description of performance, supporting the claim that multiple sources of continuous information are evaluated and integrated in speech perception. These results replicate previous results found with English, Spanish, and Japanese speakers. Although there were significant performance differences, the model analyses indicated no differences in the nature of information processing across language groups. The performance differences across languages were caused by information differences due to different phonologies in Dutch and English. These results suggest that the underlying mechanisms for speech perception are similar across languages.

Visual, orthographic, phonological, and lexical influences in reading.

The fuzzy logical model of perception (FLMP), used as a framework to analyze phenomena in reading, asserts that there are multiple influences on reading performance. Experiments on the dynamic interaction of letter information and orthographic context are presented. Previous results have supported the FLMP over a variety of interactive activation models. These findings indicated that lateral masking and the time course of processing must be accounted for in the theoretical prediction of letter and word recognition. A new finding is that the word superiority effect is not influenced by the nature of the backward masking stimulus, nor is it diminished with letter and word masks, contrary to the predictions of several extant explanations. The FLMP is extended to account for reaction time in reading. Perceptual recognition, naming, and lexical decision tasks reflect the influence of multiple sources of information. These include orthographic structure, spelling-to-speech correspondences, and word frequency. Reading can be productively analyzed as a prototypical pattern recognition situation in which the reader exploits multiple sources of information in perception and action.

A pattern recognition account of decision making.

In the domain of pattern recognition, experiments have shown that perceivers integrate multiple sources of information in an optimal manner. In contrast, other research has been interpreted to mean that decision making is nonoptimal. As an example, Tversky and Kahneman (1983) have shown that subjects commit a conjunction fallacy because they judge it more likely that a fictitious person named Linda is a bank teller and a feminist than just a bank teller. This judgment supposedly violates probability theory, because the probability of two events can never be greater than the probability of either event alone. The present research tests the hypothesis that subjects interpret this judgment task as a pattern recognition task. If this hypothesis is correct, subjects' judgments should be described accurately by the fuzzy logical model of perception (FLMP)--a successful model of pattern recognition. In the first experiment, the Linda task was extended to an expanded factorial design with five vocations and five avocations. The probability ratings were described well by the FLMP and described poorly by a simple probability model. The second experiment included (1) two fictitious people, Linda and Joan, as response alternatives and (2) both ratings and categorization judgments. Although the ratings were accurately described by both the FLMP and an averaging of the sources of information, the categorization judgments were described better by the FLMP. These results reveal important similarities in recognizing patterns and in decision making. Given that the FLMP is an optimal method for combining multiple sources of information, the probability judgments appear to be optimal in the same manner as pattern-recognition judgments.

Children's integration of speech and pointing gestures in comprehension.

We examined 4- and 9-year-old's referential comprehension when given pointing gestures and spoken labels, in two types of contextually ambiguous situations. In one situation, speech/gesture discordance was produced in conditions where labels for one of four objects being referred to sounded either alike, or different from each other. In the other, the contextual set contained the same two objects, and ambiguity was produced by factorially combining speech on a continuum ranging between /bcl/ and /dcl/ with a pointing gesture from a continuum ranging between an unambiguous point to a ball and to a doll. Results showed that the speech modality had a far greater influence on word comprehension than gestures. Second, the influence of gestures was greater for the older children. Mathematical models of speech-gesture understanding were tested against the data. Selection models assume that one dimension of information is used on a given trial, and that the selection of a modality depends on the ambiguity of information encoded on the dominant dimension. The Fuzzy Logical Model of Perception (FLMP) assumes that both modalities are evaluated independently of one another and then integrated to achieve comprehension. The results from both age groups were best described by the assumptions of the FLMP. Results are related to general claims about perceptual development during childhood concerning the quality of representations formed and dimensional selectivity of visual-spoken language.

Long-term training, transfer, and retention in learning to lipread.

A long-term training paradigm in lipreading was used to test the fuzzy logical model of perception (FLMP). This model has been used successfully to describe the joint contribution of audible and visible speech in bimodal speech perception. Tests of the model were extended in the present experiment to include the prediction of confusion matrices, as well as performance at several different levels of skill. The predictions of the FLMP were contrasted with the predictions of a prelabeling integration model (PRLM). Subjects were taught to lipread 22 initial consonants in three different vowel contexts. Training involved a variety of discrimination and identification lessons with the consonant-vowel syllables. Repeated testing was given on syllables, words, and sentences. The test items were presented visually, auditorily, and bimodally, at normal rate or three times normal rate. The subjects improved in their lipreading ability across all three types of test items. Replicating previous results, the present study illustrates that substantial gains in lipreading performance are possible. Relative to the PRLM, the FLMP gave a better description of the confusion matrices at both the beginning and the end of practice. One new finding from the present study is that the FLMP can account for the gains in bimodal speech perception as subjects improve their lipreading and listening abilities.

The paradigm and the fuzzy logical model of perception are alive and well.

Cutting, Bruno, Brady, and Moore (1992) criticized the paradigm for inquiry and the fuzzy logical model of perception (FLMP) presented in Massaro (1988a). In this reply to their remarks, it is shown that (a) the properties of the paradigm are ideal for inquiry; (b) models are best tested against the results of individual subjects and not average group data; (c) model fitting and analysis of variance do not give contradictory results; (d) the FLMP can be proven false and does not have a superpower to predict a plethora of functions or to absorb random variability; and (e) various extraneous characteristics of a model, such as equation length, cannot account for the success of the FLMP. On the other hand, the empirical findings of Cutting et al. give important new properties of pattern recognition. Finally, Cutting's theory of directed perception is compared with the FLMP.