Distinct but related abilities for visual and haptic object recognition.

People vary in their ability to recognize objects visually. Individual differences for matching and recognizing objects visually is supported by a domain-general ability capturing common variance across different tasks (e.g., Richler et al., Psychological Review, 126, 226-251, 2019). Behavioral (e.g., Cooke et al., Neuropsychologia, 45, 484-495, 2007) and neural evidence (e.g., Amedi, Cerebral Cortex, 12, 1202-1212, 2002) suggest overlapping mechanisms in the processing of visual and haptic information in the service of object recognition, but it is unclear whether such group-average results generalize to individual differences. Psychometrically validated measures are required, which have been lacking in the haptic modality. We investigate whether object recognition ability is specific to vision or extends to haptics using psychometric measures we have developed. We use multiple visual and haptic tests with different objects and different formats to measure domain-general visual and haptic abilities and to test for relations across them. We measured object recognition abilities using two visual tests and four haptic tests (two each for two kinds of haptic exploration) in 97 participants. Partial correlation and confirmatory factor analyses converge to support the existence of a domain-general haptic object recognition ability that is moderately correlated with domain-general visual object recognition ability. Visual and haptic abilities share about 25% of their variance, supporting the existence of a multisensory domain-general ability while leaving a substantial amount of residual variance for modality-specific abilities. These results extend our understanding of the structure of object recognition abilities; while there are mechanisms that may generalize across categories, tasks, and modalities, there are still other mechanisms that are distinct between modalities.

Evidence for an amodal domain-general object recognition ability.

A general object recognition ability predicts performance across a variety of high-level visual tests, categories, and performance in haptic recognition. Does this ability extend to auditory recognition? Vision and haptics tap into similar representations of shape and texture. In contrast, features of auditory perception like pitch, timbre, or loudness do not readily translate into shape percepts related to edges, surfaces, or spatial arrangement of parts. We find that an auditory object recognition ability correlates highly with a visual object recognition ability after controlling for general intelligence, perceptual speed, low-level visual ability, and memory ability. Auditory object recognition was a stronger predictor of visual object recognition than all control measures across two experiments, even though those control variables were also tested visually. These results point towards a single high-level ability used in both vision and audition. Much work highlights how the integration of visual and auditory information is important in specific domains (e.g., speech, music), with evidence for some overlap of visual and auditory neural representations. Our results are the first to reveal a domain-general ability, o, that predicts object recognition performance in both visual and auditory tests. Because o is domain-general, it reveals mechanisms that apply across a wide range of situations, independent of experience and knowledge. As o is distinct from general intelligence, it is well positioned to potentially add predictive validity when explaining individual differences in a variety of tasks, above and beyond measures of common cognitive abilities like general intelligence and working memory.

Relating a Spiking Neural Network Model and the Diffusion Model of Decision-Making.

Many models of decision making assume accumulation of evidence to threshold as a core mechanism to predict response probabilities and response times. A spiking neural network model (Wang, 2002) instantiates these mechanisms at the level of biophysically-plausible pools of neurons with excitatory and inhibitory connections, and has numerous model parameters tuned by physiological measures. The diffusion model (Ratcliff, 1978) is a cognitive model that can be fitted to a range of behaviors and conditions. We investigated how parameters of the cognitive-level diffusion model relate to the parameters of a neural-level spiking model. In each simulated "experiment", we generated "data" from the spiking neural network by factorially combining a manipulation of choice difficulty (via the input to the spiking model) and a manipulation of one of the core parameters of the spiking model. We then fitted the diffusion model to these simulated data to observe how manipulation of each core spiking model parameter mapped on to fitted drift rate, response threshold, and non-decision time. Manipulations of parameters in the spiking model related to input sensitivity, threshold, and stimulus processing time mapped on to their conceptual analogues in the diffusion model, namely drift rate, threshold, and non-decision time. Manipulations of parameters in the spiking model with no direct analogue to the diffusion model, non-stimulus-specific background input, strength of recurrent excitation, and receptor conductances, mapped on to threshold in the diffusion model. We discuss implications of these results for interpretations of fits of the diffusion model to behavioral data.

Visual object recognition ability is not related to experience with visual arts.

Visual arts require the ability to process, categorize, recognize, and understand a variety of visual inputs. These challenges may engage and even influence mechanisms that are also relevant for visual object recognition beyond visual arts. A domain-general object recognition ability that applies broadly across a range of visual tasks was recently discovered. Here, we ask whether experience with visual arts is correlated with this domain-general ability. We developed a new survey to measure general visual arts experience and use it to measure arts experience in 142 individuals in whom we also estimated domain-general object recognition ability. Despite our measures demonstrating high reliability in a large sample size, we found substantial evidence (BF01 = 9.52) for no correlation between visual arts experience and general object recognition ability. This suggests that experience in visual arts has little influence on object recognition skills or vice versa, at least in our sample ranging from low to moderately high levels of arts experience. Our methods can be extended to other populations and our results should be replicated, as they suggest some limitations for the generalization of programs targeting visual literacy beyond the visual arts.

Salience by competitive and recurrent interactions: Bridging neural spiking and computation in visual attention.

Decisions about where to move the eyes depend on neurons in frontal eye field (FEF). Movement neurons in FEF accumulate salience evidence derived from FEF visual neurons to select the location of a saccade target among distractors. How visual neurons achieve this salience representation is unknown. We present a neuro-computational model of target selection called salience by competitive and recurrent interactions (SCRI), based on the competitive interaction model of attentional selection and decision-making (Smith & Sewell, 2013). SCRI selects targets by synthesizing localization and identification information to yield a dynamically evolving representation of salience across the visual field. SCRI accounts for neural spiking of individual FEF visual neurons, explaining idiosyncratic differences in neural dynamics with specific parameters. Many visual neurons resolve the competition between search items through feedforward inhibition between signals representing different search items, some also require lateral inhibition, and many act as recurrent gates to modulate the incoming flow of information about stimulus identity. SCRI was tested further by using simulated spiking representations of visual salience as input to the gated accumulator model of FEF movement neurons (Purcell et al., 2010, 2012). Predicted saccade response times fit those observed for search arrays of different set sizes and different target-distractor similarities, and accumulator trajectories replicated movement neuron discharge rates. These findings offer new insights into visual decision-making through converging neuro-computational constraints and provide a novel computational account of the diversity of FEF visual neurons. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Revealing a competitive dynamic in rapid categorization with object substitution masking.

Categorization at different levels of abstraction have distinct time courses, but the different levels are often considered separately. Superordinate-level categorization is typically faster than basic-level categorization at ultra-rapid exposure durations (< 33 ms) while basic-level categorization is faster than superordinate-level categorization at longer exposure durations. This difference may be due to a competitive dynamic between levels of categorization. By leveraging object substitution masking, we found a distinct time course of masking effects for each level of categorization. Superordinate-level categorization showed a masking effect earlier than basic-level categorization. However, when basic-level categorization first showed a masking effects, superordinate-level categorization was spared despite its earlier masking effect. This unique pattern suggests a trade-off between the two levels of categorization over time. Such an effect supports an account of categorization that depends on the interaction of perceptual encoding, selective attention, and competition between levels of category representation.

The Role of Cue-Based Strategies in Skilled Diagnosis Among Pathologists.

OBJECTIVE: This research was designed to test whether behavioral indicators of pathology-related cue utilization were associated with performance on a diagnostic task. BACKGROUND: Across many domains, including pathology, successful diagnosis depends on pattern recognition that is supported by associations in memory in the form of cues. Previous studies have focused on the specific information or knowledge on which medical image expertise relies. The target in this study is the more general ability to identify and interpret relevant information. METHOD: Data were collected from 54 histopathologists in both conference and online settings. The participants completed a pathology edition of the Expert Intensive Skills Evaluation 2.0 (EXPERTise 2.0) to establish behavioral indicators of context-related cue utilization. They also completed a separate diagnostic task designed to examine related diagnostic skills. RESULTS: Behavioral indicators of higher or lower cue utilization were based on the participants' performance across five tasks. Accounting for the number of cases reported per year, higher cue utilization was associated with greater accuracy on the diagnostic task. A post hoc analysis suggested that higher cue utilization may be associated with a greater capacity to recognize low prevalence cases. CONCLUSION: This study provides support for the role of cue utilization in the development and maintenance of skilled diagnosis amongst pathologists. APPLICATION: Pathologist training needs to be structured to ensure that learners have the opportunity to form cue-based strategies and associations in memory, especially for less commonly seen diseases.

Haptic object recognition based on shape relates to visual object recognition ability.

Visual object recognition depends in large part on a domain-general ability (Richler et al. Psychol Rev 126(2): 226-251, 2019). Given evidence pointing towards shared mechanisms for object perception across vision and touch, we ask whether individual differences in haptic and visual object recognition are related. We use existing validated visual tests to estimate visual object recognition ability and relate it to performance on two novel tests of haptic object recognition ability (n = 66). One test includes complex objects that participants chose to explore with a hand grasp. The other test uses a simpler stimulus set that participants chose to explore with just their fingertips. Only performance on the haptic test with complex stimuli correlated with visual object recognition ability, suggesting a shared source of variance across task structures, stimuli, and modalities. A follow-up study using a visual version of the haptic test with simple stimuli shows a correlation with the original visual tests, suggesting that the limited complexity of the stimuli did not limit correlation with visual object recognition ability. Instead, we propose that the manner of exploration may be a critical factor in whether a haptic test relates to visual object recognition ability. Our results suggest a perceptual ability that spans at least across vision and touch, however, it may not be recruited during just fingertip exploration.

Differentiating Experience From Cue Utilization in Radiological Assessments.

OBJECTIVE: This research was designed to examine the contribution of self-reported experience and cue utilization to diagnostic accuracy in the context of radiology. BACKGROUND: Within radiology, it is unclear how task-related experience contributes to the acquisition of associations between features with events in memory, or cues, and how they contribute to diagnostic performance. METHOD: Data were collected from 18 trainees and 41 radiologists. The participants completed a radiology edition of the established cue utilization assessment tool EXPERTise 2.0, which provides a measure of cue utilization based on performance on a number of domain-specific tasks. The participants also completed a separate image interpretation task as an independent measure of diagnostic performance. RESULTS: Consistent with previous research, a k-means cluster analysis using the data from EXPERTise 2.0 delineated two groups, the pattern of centroids of which reflected higher and lower cue utilization. Controlling for years of experience, participants with higher cue utilization were more accurate on the image interpretation task compared to participants who demonstrated relatively lower cue utilization (p = .01). CONCLUSION: This study provides support for the role of cue utilization in assessments of radiology images among qualified radiologists. Importantly, it also demonstrates that cue utilization and self-reported years of experience as a radiologist make independent contributions to performance on the radiological diagnostic task. APPLICATION: Task-related experience, including training, needs to be structured to ensure that learners have the opportunity to acquire feature-event relationships and internalize these associations in the form of cues in memory.

Combining convolutional neural networks and cognitive models to predict novel object recognition in humans.

Object representations from convolutional neural network (CNN) models of computer vision (LeCun, Bengio, & Hinton, 2015) were used to drive a cognitive model of decision making, the linear ballistic accumulator (LBA) model (Brown & Heathcote, 2008), to predict errors and response times (RTs) in a novel object recognition task in humans. CNNs have become very successful at visual tasks like classifying objects in real-world images (e.g., He, Zhang, Ren, & Sun, 2015; Krizhevsky, Sutskever, & Hinton, 2012). We asked whether object representations learned by CNNs previously trained on a large corpus of natural images could be used to predict performance recognizing novel objects the network has never been trained on; we used novel Greebles, Ziggerins, and Sheinbugs that have been used in a number of previous object recognition studies. We specifically investigated whether a model combining high-level CNN representations of these novel objects could be used to drive an LBA model of decision making to account for errors and RTs in a same-different matching task (from Richler et al., 2019). Combining linearly transformed CNN object representations with the LBA provided reasonable accounts of performance not only on average, but at the individual-participant level and the item level as well. We frame the findings in the context of growing interest in using CNN models to understand visual object representations and the promise of using CNN representations to extend cognitive models to explain more complex aspects of human behavior. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Countermanding Perceptual Decision-Making.

We investigated whether a task requiring concurrent perceptual decision-making and response control can be performed concurrently, whether evidence accumulation and response control are accomplished by the same neurons, and whether perceptual decision-making and countermanding can be unified computationally. Based on neural recordings in a prefrontal area of macaque monkeys, we present behavioral, neural, and computational results demonstrating that perceptual decision-making of varying difficulty can be countermanded efficiently, that single prefrontal neurons instantiate both evidence accumulation and response control, and that an interactive race between stochastic GO evidence accumulators for each alternative and a distinct STOP accumulator fits countermanding choice behavior and replicates neural trajectories. Thus, perceptual decision-making and response control, previously regarded as distinct mechanisms, are actually aspects of a common neuro-computational mechanism supporting flexible behavior.

Thermodynamic Integration and Steppingstone Sampling Methods for Estimating Bayes Factors: A Tutorial.

One of the more principled methods of performing model selection is via Bayes factors. However, calculating Bayes factors requires marginal likelihoods, which are integrals over the entire parameter space, making estimation of Bayes factors for models with more than a few parameters a significant computational challenge. Here, we provide a tutorial review of two Monte Carlo techniques rarely used in psychology that efficiently compute marginal likelihoods: thermodynamic integration (Friel & Pettitt, 2008; Lartillot & Philippe, 2006) and steppingstone sampling (Xie, Lewis, Fan, Kuo, & Chen, 2011). The methods are general and can be easily implemented in existing MCMC code; we provide both the details for implementation and associated R code for the interested reader. While Bayesian toolkits implementing standard statistical analyses (e.g., JASP Team, 2017; Morey & Rouder, 2015) often compute Bayes factors for the researcher, those using Bayesian approaches to evaluate cognitive models are usually left to compute Bayes factors for themselves. Here, we provide examples of the methods by computing marginal likelihoods for a moderately complex model of choice response time, the Linear Ballistic Accumulator model (Brown & Heathcote, 2008), and compare them to findings of Evans and Brown (2017), who used a brute force technique. We then present a derivation of TI and SS within a hierarchical framework, provide results of a model recovery case study using hierarchical models, and show an application to empirical data. A companion R package is available at the Open Science Framework: https://osf.io/jpnb4.

Neurally constrained modeling of speed-accuracy tradeoff during visual search: gated accumulation of modulated evidence.

Stochastic accumulator models account for response times and errors in perceptual decision making by assuming a noisy accumulation of perceptual evidence to a threshold. Previously, we explained saccade visual search decision making by macaque monkeys with a stochastic multiaccumulator model in which accumulation was driven by a gated feed-forward integration to threshold of spike trains from visually responsive neurons in frontal eye field that signal stimulus salience. This neurally constrained model quantitatively accounted for response times and errors in visual search for a target among varying numbers of distractors and replicated the dynamics of presaccadic movement neurons hypothesized to instantiate evidence accumulation. This modeling framework suggested strategic control over gate or over threshold as two potential mechanisms to accomplish speed-accuracy tradeoff (SAT). Here, we show that our gated accumulator model framework can account for visual search performance under SAT instructions observed in a milestone neurophysiological study of frontal eye field. This framework captured key elements of saccade search performance, through observed modulations of neural input, as well as flexible combinations of gate and threshold parameters necessary to explain differences in SAT strategy across monkeys. However, the trajectories of the model accumulators deviated from the dynamics of most presaccadic movement neurons. These findings demonstrate that traditional theoretical accounts of SAT are incomplete descriptions of the underlying neural adjustments that accomplish SAT, offer a novel mechanistic account of decision-making mechanisms during speed-accuracy tradeoff, and highlight questions regarding the identity of model and neural accumulators. NEW & NOTEWORTHY A gated accumulator model is used to elucidate neurocomputational mechanisms of speed-accuracy tradeoff. Whereas canonical stochastic accumulators adjust strategy only through variation of an accumulation threshold, we demonstrate that strategic adjustments are accomplished by flexible combinations of both modulation of the evidence representation and adaptation of accumulator gate and threshold. The results indicate how model-based cognitive neuroscience can translate between abstract cognitive models of performance and neural mechanisms of speed-accuracy tradeoff.

The Quality of Response Time Data Inference: A Blinded, Collaborative Assessment of the Validity of Cognitive Models.

Most data analyses rely on models. To complement statistical models, psychologists have developed cognitive models, which translate observed variables into psychologically interesting constructs. Response time models, in particular, assume that response time and accuracy are the observed expression of latent variables including 1) ease of processing, 2) response caution, 3) response bias, and 4) non-decision time. Inferences about these psychological factors, hinge upon the validity of the models' parameters. Here, we use a blinded, collaborative approach to assess the validity of such model-based inferences. Seventeen teams of researchers analyzed the same 14 data sets. In each of these two-condition data sets, we manipulated properties of participants' behavior in a two-alternative forced choice task. The contributing teams were blind to the manipulations, and had to infer what aspect of behavior was changed using their method of choice. The contributors chose to employ a variety of models, estimation methods, and inference procedures. Our results show that, although conclusions were similar across different methods, these "modeler's degrees of freedom" did affect their inferences. Interestingly, many of the simpler approaches yielded as robust and accurate inferences as the more complex methods. We recommend that, in general, cognitive models become a typical analysis tool for response time data. In particular, we argue that the simpler models and procedures are sufficient for standard experimental designs. We finish by outlining situations in which more complicated models and methods may be necessary, and discuss potential pitfalls when interpreting the output from response time models.

On Testing and Developing Cognitive Models.

The target article, "Robust Modeling in Cognitive Science", proposes a number of recommended practices in computational modeling in response to the growing "crisis of confidence" facing many scientific disciplines, including psychology and neuroscience. Those of us who do modeling, write about modeling, teach modeling, and mentor modelers worry deeply about best practices and any new suggestions for making modeling more transparent, trusted, and robust is welcome. Many of the recommendations seem uncontroversial. My commentary focuses on forms of preregistration and postregistration, which constitute three of the four key ideas highlighted as take-home recommendations at the conclusion of the target article. I have chosen to consider these recommendations by reflecting on my own past experiences developing new models and modeling approaches.

Modeling memory dynamics in visual expertise.

The development of visual expertise is accompanied by enhanced visual object recognition memory within an expert domain. We aimed to understand the relationship between expertise and memory by modeling cognitive mechanisms. Participants with a measured range of birding expertise were recruited and tested on memory for birds (expert domain) and cars (novice domain). Participants performed an old-new continuous recognition memory task whereby on each trial an image of a bird or car was presented that was either new or had been presented earlier with lag j. The Linear Ballistic Accumulator model (LBA; Brown & Heathcote, 2008) was first used to decompose accuracy and response time (RT) into drift rate, response threshold, and nondecision time, with the measured level of visual expertise as a potential covariate on each model parameter. An Expertise × Category interaction was observed on drift rates such that expertise was positively correlated with memory performance recognizing bird images but not car images as old versus new. To then model the underlying processes responsible for variation in drift rate with expertise, we used a model of drift rates building on the Exemplar-Based Random Walk model (Nosofsky, Cox, Cao, & Shiffrin, 2014; Nosofsky & Palmeri, 1997), which revealed that expertise was associated with increases in memory strength and increases in the distinctiveness of stored exemplars. Taken together, we provide insight using formal cognitive modeling into how improvements in recognition memory with expertise are driven by enhancements in the representations of objects in an expert domain. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Composite faces are not (necessarily) processed coactively: A test using systems factorial technology and logical-rule models.

Upright faces are thought to be processed more holistically than inverted faces. In the widely used composite face paradigm, holistic processing is inferred from interference in recognition performance from a to-be-ignored face half for upright and aligned faces compared with inverted or misaligned faces. We sought to characterize the nature of holistic processing in composite faces in computational terms. We use logical-rule models (Fific, Little, & Nosofsky, 2010) and Systems Factorial Technology (Townsend & Nozawa, 1995) to examine whether composite faces are processed through pooling top and bottom face halves into a single processing channel-coactive processing-which is one common mechanistic definition of holistic processing. By specifically operationalizing holistic processing as the pooling of features into a single decision process in our task, we are able to distinguish it from other processing models that may underlie composite face processing. For instance, a failure of selective attention might result even when top and bottom components of composite faces are processed in serial or in parallel without processing the entire face coactively. Our results show that performance is best explained by a mixture of serial and parallel processing architectures across all 4 upright and inverted, aligned and misaligned face conditions. The results indicate multichannel, featural processing of composite faces in a manner inconsistent with the notion of coactivity. (PsycINFO Database Record

High-resolution Functional Magnetic Resonance Imaging Reveals Configural Processing of Cars in Right Anterior Fusiform Face Area of Car Experts.

Visual object expertise correlates with neural selectivity in the fusiform face area (FFA). Although behavioral studies suggest that visual expertise is associated with increased use of holistic and configural information, little is known about the nature of the supporting neural representations. Using high-resolution 7-T functional magnetic resonance imaging, we recorded the multivoxel activation patterns elicited by whole cars, configurally disrupted cars, and car parts in individuals with a wide range of car expertise. A probabilistic support vector machine classifier was trained to differentiate activation patterns elicited by whole car images from activation patterns elicited by misconfigured car images. The classifier was then used to classify new combined activation patterns that were created by averaging activation patterns elicited by individually presented top and bottom car parts. In line with the idea that the configuration of parts is critical to expert visual perception, car expertise was negatively associated with the probability of a combined activation pattern being classified as a whole car in the right anterior FFA, a region critical to vision for categories of expertise. Thus, just as found for faces in normal observers, the neural representation of cars in right anterior FFA is more holistic for car experts than car novices, consistent with common mechanisms of neural selectivity for faces and other objects of expertise in this area.

Bayesian statistical approaches to evaluating cognitive models.

Cognitive models aim to explain complex human behavior in terms of hypothesized mechanisms of the mind. These mechanisms can be formalized in terms of mathematical structures containing parameters that are theoretically meaningful. For example, in the case of perceptual decision making, model parameters might correspond to theoretical constructs like response bias, evidence quality, response caution, and the like. Formal cognitive models go beyond verbal models in that cognitive mechanisms are instantiated in terms of mathematics and they go beyond statistical models in that cognitive model parameters are psychologically interpretable. We explore three key elements used to formally evaluate cognitive models: parameter estimation, model prediction, and model selection. We compare and contrast traditional approaches with Bayesian statistical approaches to performing each of these three elements. Traditional approaches rely on an array of seemingly ad hoc techniques, whereas Bayesian statistical approaches rely on a single, principled, internally consistent system. We illustrate the Bayesian statistical approach to evaluating cognitive models using a running example of the Linear Ballistic Accumulator model of decision making (Brown SD, Heathcote A. The simplest complete model of choice response time: linear ballistic accumulation. Cogn Psychol 2008, 57:153-178). WIREs Cogn Sci 2018, 9:e1458. doi: 10.1002/wcs.1458 This article is categorized under: Neuroscience > Computation Psychology > Reasoning and Decision Making Psychology > Theory and Methods.

RELATING ACCUMULATOR MODEL PARAMETERS AND NEURAL DYNAMICS.

Accumulator models explain decision-making as an accumulation of evidence to a response threshold. Specific model parameters are associated with specific model mechanisms, such as the time when accumulation begins, the average rate of evidence accumulation, and the threshold. These mechanisms determine both the within-trial dynamics of evidence accumulation and the predicted behavior. Cognitive modelers usually infer what mechanisms vary during decision-making by seeing what parameters vary when a model is fitted to observed behavior. The recent identification of neural activity with evidence accumulation suggests that it may be possible to directly infer what mechanisms vary from an analysis of how neural dynamics vary. However, evidence accumulation is often noisy, and noise complicates the relationship between accumulator dynamics and the underlying mechanisms leading to those dynamics. To understand what kinds of inferences can be made about decision-making mechanisms based on measures of neural dynamics, we measured simulated accumulator model dynamics while systematically varying model parameters. In some cases, decision- making mechanisms can be directly inferred from dynamics, allowing us to distinguish between models that make identical behavioral predictions. In other cases, however, different parameterized mechanisms produce surprisingly similar dynamics, limiting the inferences that can be made based on measuring dynamics alone. Analyzing neural dynamics can provide a powerful tool to resolve model mimicry at the behavioral level, but we caution against drawing inferences based solely on neural analyses. Instead, simultaneous modeling of behavior and neural dynamics provides the most powerful approach to understand decision-making and likely other aspects of cognition and perception.