Transfer of visual perceptual learning over a task-irrelevant feature through feature-invariant representations: Behavioral experiments and model simulations.

A large body of literature has examined specificity and transfer of perceptual learning, suggesting a complex picture. Here, we distinguish between transfer over variations in a "task-relevant" feature (e.g., transfer of a learned orientation task to a different reference orientation) and transfer over a "task-irrelevant" feature (e.g., transfer of a learned orientation task to a different retinal location or different spatial frequency), and we focus on the mechanism for the latter. Experimentally, we assessed whether learning a judgment of one feature (such as orientation) using one value of an irrelevant feature (e.g., spatial frequency) transfers to another value of the irrelevant feature. Experiment 1 examined whether learning in eight-alternative orientation identification with one or multiple spatial frequencies transfers to stimuli at five different spatial frequencies. Experiment 2 paralleled Experiment 1, examining whether learning in eight-alternative spatial-frequency identification at one or multiple orientations transfers to stimuli with five different orientations. Training the orientation task with a single spatial frequency transferred widely to all other spatial frequencies, with a tendency to specificity when training with the highest spatial frequency. Training the spatial frequency task fully transferred across all orientations. Computationally, we extended the identification integrated reweighting theory (I-IRT) to account for the transfer data (Dosher, Liu, & Lu, 2023; Liu, Dosher, & Lu, 2023). Just as location-invariant representations in the original IRT explain transfer over retinal locations, incorporating feature-invariant representations effectively accounted for the observed transfer. Taken together, we suggest that feature-invariant representations can account for transfer of learning over a "task-irrelevant" feature.

Enabling identification of component processes in perceptual learning with nonparametric hierarchical Bayesian modeling.

Perceptual learning is a multifaceted process, encompassing general learning, between-session forgetting or consolidation, and within-session fast relearning and deterioration. The learning curve constructed from threshold estimates in blocks or sessions, based on tens or hundreds of trials, may obscure component processes; high temporal resolution is necessary. We developed two nonparametric inference procedures: a Bayesian inference procedure (BIP) to estimate the posterior distribution of contrast threshold in each learning block for each learner independently and a hierarchical Bayesian model (HBM) that computes the joint posterior distribution of contrast threshold across all learning blocks at the population, subject, and test levels via the covariance of contrast thresholds across blocks. We applied the procedures to the data from two studies that investigated the interaction between feedback and training accuracy in Gabor orientation identification over 1920 trials across six sessions and estimated learning curve with block sizes L = 10, 20, 40, 80, 160, and 320 trials. The HBM generated significantly better fits to the data, smaller standard deviations, and more precise estimates, compared to the BIP across all block sizes. In addition, the HBM generated unbiased estimates, whereas the BIP only generated unbiased estimates with large block sizes but exhibited increased bias with small block sizes. With L = 10, 20, and 40, we were able to consistently identify general learning, between-session forgetting, and rapid relearning and adaptation within sessions. The nonparametric HBM provides a general framework for fine-grained assessment of the learning curve and enables identification of component processes in perceptual learning.

Estimating the Trial-by-Trial Learning Curve in Perceptual Learning with Hierarchical Bayesian Modeling.

The learning curve serves as a crucial metric for assessing human performance in perceptual learning. It may encompass various component processes, including general learning, between-session forgetting or consolidation, and within-session rapid relearning and adaptation or deterioration. Typically, empirical learning curves are constructed by aggregating tens or hundreds of trials of data in blocks or sessions. Here, we devised three inference procedures for estimating the trial-by-trial learning curve based on the multi-component functional form identified in Zhao et al. (submitted): general learning, between-session forgetting, and within-session rapid relearning and adaptation. These procedures include a Bayesian inference procedure (BIP) estimating the posterior distribution of parameters for each learner independently, and two hierarchical Bayesian models (HBMv and HBMc) computing the joint posterior distribution of parameters and hyperparameters at the population, subject, and test levels. The HBMv and HBMc incorporate variance and covariance hyperparameters, respectively, between and within subjects. We applied these procedures to data from two studies investigating the interaction between feedback and training accuracy in Gabor orientation identification across about 2000 trials spanning six sessions (Liu et al., 2010, 2012) and estimated the trial-by-trial learning curves at both the subject and population levels. The HBMc generated best fits to the data and the smallest half width of 68.2% credible interval of the learning curves compared to the BIP and HBMv. The parametric HBMc with the multi-component functional form provides a general framework for trial-by-trial analysis of the component processes in perceptual learning and for predicting the learning curve in unmeasured time points.

Informational feedback accelerates learning in multi-alternative perceptual judgements of orientation.

Experience or training can substantially improve perceptual performance through perceptual learning, and the extent and rate of these improvements may be affected by feedback. In this paper, we first developed a neural network model based on the integrated reweighting theory (Dosher et al., 2013) to account for perceptual learning and performance in n-alternative identification tasks and the dependence of learning on different forms of feedback. We then report an experiment comparing the effectiveness of response feedback (RF) versus accuracy feedback (AF) or no feedback (NF) (full versus partial versus no supervision) in learning a challenging eight-alternative visual orientation identification (8AFC) task. Although learning sometimes occurred in the absence of feedback (NF), RF had a clear advantage above AF or NF in this task. Using hybrid supervision learning rules, a new n-alternative identification integrated reweighting theory (I-IRT) explained both the differences in learning curves given different feedback and the dynamic changes in identification confusion data. This study shows that training with more informational feedback (RF) is more effective, though not necessary, in these challenging n-alternative tasks, a result that has implications for developing training paradigms in realistic tasks.

Learning spatial frequency identification through reweighted decoding.

Perceptual learning, the improvement of perceptual judgments with practice, occurs in many visual tasks. There are, however, relatively fewer studies examining perceptual learning in spatial frequency judgments. In addition, perceptual learning has generally been studied in two-alternative tasks, occasionally in n-alternative tasks, and infrequently in identification. Recently, perceptual learning was found in an orientation identification task (eight-alternatives) and was well accounted for by a new identification integrated reweighting theory (I-IRT) (Liu et al., submitted). Here, we examined perceptual learning in a similar eight-alternative spatial frequency absolute identification task in two different training protocols, finding learning in the majority but not all observers. We fit the I-IRT to the spatial frequency learning data and discuss possible model explanations for variations in learning.

Hierarchical Bayesian perceptual template modeling of mechanisms of spatial attention in central and peripheral cuing.

The external noise paradigm and perceptual template model (PTM) have successfully been applied to characterize observer properties and mechanisms of observer state changes (e.g. attention and perceptual learning) in several research domains, focusing on individual level analysis. In this study, we developed a new hierarchical Bayesian perceptual template model (HBPTM) to model the trial-by-trial data from all individuals and conditions in a published spatial cuing study within a single structure and compared its performance to that of a Bayesian Inference Procedure (BIP), which separately infers the posterior distributions of the model parameters for each individual subject without the hierarchical structure. The HBPTM allowed us to compute the joint posterior distribution of the hyperparameters and parameters at the population, observer, and experiment levels and make statistical inferences at all these levels. In addition, we ran a large simulation study that varied the number of observers and number of trials in each condition and demonstrated the advantage of the HBPTM over the BIP across all the simulated datasets. Although it is developed in the context of spatial attention, the HBPTM and its extensions can be used to model data from the external noise paradigm in other domains and enable predictions of human performance at both the population and individual levels.

Current directions in visual perceptual learning.

The visual expertise of adult humans is jointly determined by evolution, visual development, and visual perceptual learning. Perceptual learning refers to performance improvements in perceptual tasks after practice or training in the task. It occurs in almost all visual tasks, ranging from simple feature detection to complex scene analysis. In this Review, we focus on key behavioral aspects of visual perceptual learning. We begin by describing visual perceptual learning tasks and manipulations that influence the magnitude of learning, and then discuss specificity of learning. Next, we present theories and computational models of learning and specificity. We then review applications of visual perceptual learning in visual rehabilitation. Finally, we summarize the general principles of visual perceptual learning, discuss the tension between plasticity and stability, and conclude with new research directions.

Visual attention in spatial cueing and visual search.

To characterize internal processes of an observer conducting perceptual tasks, we developed an observer model that combines the perceptual template model (PTM), the attention mechanisms in the PTM framework (Lu & Dosher, 1998), and uncertainty of signal detection theory (Green & Swets, 1966). The model was evaluated with a visual search experiment conducted in a range of external noise, signal contrast, and target-distractor similarity conditions. In each trial, eight Gabor patches were shown in each of two brief intervals, with one target at a different orientation from the distractors in one of the presentations. Subjects were precued to a subset of the stimuli (1, 2, 4, or 8) and asked to report (a) which interval contained the target and (b) where the target was. Individual roles of uncertainty and of attention in visual search were investigated by comparing models with and without an attention component. The results showed that decision uncertainty alone was sufficient to account for the set-size effect, even in conditions with high target-distractor similarity. Our theoretical model and empirical results provide a coherent picture regarding how visual information is selected and processed during feature search.

Roving: The causes of interference and re-enabled learning in multi-task visual training.

People routinely perform multiple visual judgments in the real world, yet, intermixing tasks or task variants during training can damage or even prevent learning. This paper explores why. We challenged theories of visual perceptual learning focused on plastic retuning of low-level retinotopic cortical representations by placing different task variants in different retinal locations, and tested theories of perceptual learning through reweighting (changes in readout) by varying task similarity. Discriminating different (but equivalent) and similar orientations in separate retinal locations interfered with learning, whereas training either with identical orientations or sufficiently different ones in different locations released rapid learning. This location crosstalk during learning renders it unlikely that the primary substrate of learning is retuning in early retinotopic visual areas; instead, learning likely involves reweighting from location-independent representations to a decision. We developed an Integrated Reweighting Theory (IRT), which has both V1-like location-specific representations and higher level (V4/IT or higher) location-invariant representations, and learns via reweighting the readout to decision, to predict the order of learning rates in different conditions. This model with suitable parameters successfully fit the behavioral data, as well as some microstructure of learning performance in a new trial-by-trial analysis.

Evaluating the performance of the staircase and quick Change Detection methods in measuring perceptual learning.

The staircase method has been widely used in measuring perceptual learning. Recently, Zhao, Lesmes, and Lu (2017, 2019) developed the quick Change Detection (qCD) method and applied it to measure the trial-by-trial time course of dark adaptation. In the current study, we conducted two simulations to evaluate the performance of the 3-down/1-up staircase and qCD methods in measuring perceptual learning in a two-alternative forced-choice task. In Study 1, three observers with different time constants (40, 80, and 160 trials) of an exponential learning curve were simulated. Each simulated observer completed staircases with six step sizes (1%, 5%, 10%, 20%, 30%, and 60%) and a qCD procedure, each starting at five levels (+50%, +25%, 0, -25%, and -50% different from the true threshold in the first trial). We found the following results: Staircases with 1% and 5% step sizes failed to generate more than five reversals half of the time; and the bias and standard deviations of thresholds estimated from the post hoc segment-by-segment qCD analysis were much smaller than those from the staircase method with the other four step sizes. In Study 2, we simulated thresholds in the transfer phases with the same time constants and 50% transfer for each observer in Study 1. We found that the estimated transfer indexes from qCD showed smaller biases and standard deviations than those from the staircase method. In addition, rescoring the simulated data from the staircase method using the Bayesian estimation component of the qCD method resulted in much-improved estimates. We conclude that the qCD method characterizes the time course of perceptual learning and transfer more accurately, precisely, and efficiently than the staircase method, even with the optimal 10% step size.

Assessing the detailed time course of perceptual sensitivity change in perceptual learning.

The learning curve in perceptual learning is typically sampled in blocks of trials, which could result in imprecise and possibly biased estimates, especially when learning is rapid. Recently, Zhao, Lesmes, and Lu (2017, 2019) developed a Bayesian adaptive quick Change Detection (qCD) method to accurately, precisely, and efficiently assess the time course of perceptual sensitivity change. In this study, we implemented and tested the qCD method in assessing the learning curve in a four-alternative forced-choice global motion direction identification task in both simulations and a psychophysical experiment. The stimulus intensity in each trial was determined by the qCD, staircase or random stimulus selection (RSS) methods. Simulations showed that the accuracy (bias) and precision (standard deviation or confidence bounds) of the estimated learning curves from the qCD were much better than those obtained by the staircase and RSS method; this is true for both trial-by-trial and post hoc segment-by-segment qCD analyses. In the psychophysical experiment, the average half widths of the 68.2% credible interval of the estimated thresholds from the trial-by-trial and post hoc segment-by-segment qCD analyses were both quite small. Additionally, the overall estimates from the qCD and staircase methods matched extremely well in this task where the behavioral rate of learning is relatively slow. Our results suggest that the qCD method can precisely and accurately assess the trial-by-trial time course of perceptual learning.

High reward enhances perceptual learning.

Studies of perceptual learning have revealed a great deal of plasticity in adult humans. In this study, we systematically investigated the effects and mechanisms of several forms (trial-by-trial, block, and session rewards) and levels (no, low, high, subliminal) of monetary reward on the rate, magnitude, and generalizability of perceptual learning. We found that high monetary reward can greatly promote the rate and boost the magnitude of learning and enhance performance in untrained spatial frequencies and eye without changing interocular, interlocation, and interdirection transfer indices. High reward per se made unique contributions to the enhanced learning through improved internal noise reduction. Furthermore, the effects of high reward on perceptual learning occurred in a range of perceptual tasks. The results may have major implications for the understanding of the nature of the learning rule in perceptual learning and for the use of reward to enhance perceptual learning in practical applications.

Mixture of easy trials enables transient and sustained perceptual improvements through priming and perceptual learning.

The sense of vision allows us to discriminate fine details across a wide range of tasks. How to improve this perceptual skill, particularly within a short training session, is of substantial interest. Emerging evidence suggests that mixing easy trials can quickly improve performance in hard trials, but it is equivocal whether the improvement is short-lived or long-lasting, and additionally what accounts for this improvement. Here, by tracking objective performance (accuracy) and subjective experience (ratings of target visibility and choice confidence) over time and in a large sample of participants, we demonstrate the coexistence of transient and sustained effects of mixing easy trials, which differ markedly in their timescales, in their effects on subjective awareness, and in individual differences. In particular, whereas the transient effect was found to be ubiquitous and manifested similarly across objective and subjective measures, the sustained effect was limited to a subset of participants with weak convergence from objective and subjective measures. These results indicate that mixture of easy trials enables two distinct, co-existing forms of rapid perceptual improvements in hard trials, as mediated by robust priming and fragile learning. Placing constraints on theory of brain plasticity, this finding may also have implications for alleviating visual deficits.

Visual Perceptual Learning and Models.

Visual perceptual learning through practice or training can significantly improve performance on visual tasks. Originally seen as a manifestation of plasticity in the primary visual cortex, perceptual learning is more readily understood as improvements in the function of brain networks that integrate processes, including sensory representations, decision, attention, and reward, and balance plasticity with system stability. This review considers the primary phenomena of perceptual learning, theories of perceptual learning, and perceptual learning's effect on signal and noise in visual processing and decision. Models, especially computational models, play a key role in behavioral and physiological investigations of the mechanisms of perceptual learning and for understanding, predicting, and optimizing human perceptual processes, learning, and performance. Performance improvements resulting from reweighting or readout of sensory inputs to decision provide a strong theoretical framework for interpreting perceptual learning and transfer that may prove useful in optimizing learning in real-world applications.

Translating Perceptual Learning from the Laboratory to Applications.

Human training studies in the laboratory have demonstrated plasticity in brain systems and, in some cases, large improvements in perceptual performance, inspiring a quest to develop training apps and systems. We consider the next steps in the translation from the laboratory to the clinic and commerce.

Augmented Hebbian reweighting accounts for accuracy and induced bias in perceptual learning with reverse feedback.

Using an asymmetrical set of vernier stimuli (-15″, -10″, -5″, +10″, +15″) together with reverse feedback on the small subthreshold offset stimulus (-5″) induces response bias in performance (Aberg & Herzog, 2012; Herzog, Eward, Hermens, & Fahle, 2006; Herzog & Fahle, 1999). These conditions are of interest for testing models of perceptual learning because the world does not always present balanced stimulus frequencies or accurate feedback. Here we provide a comprehensive model for the complex set of asymmetric training results using the augmented Hebbian reweighting model (Liu, Dosher, & Lu, 2014; Petrov, Dosher, & Lu, 2005, 2006) and the multilocation integrated reweighting theory (Dosher, Jeter, Liu, & Lu, 2013). The augmented Hebbian learning algorithm incorporates trial-by-trial feedback, when present, as another input to the decision unit and uses the observer's internal response to update the weights otherwise; block feedback alters the weights on bias correction (Liu et al., 2014). Asymmetric training with reversed feedback incorporates biases into the weights between representation and decision. The model correctly predicts the basic induction effect, its dependence on trial-by-trial feedback, and the specificity of bias to stimulus orientation and spatial location, extending the range of augmented Hebbian reweighting accounts of perceptual learning.

Developing Bayesian adaptive methods for estimating sensitivity thresholds (d') in Yes-No and forced-choice tasks.

Motivated by Signal Detection Theory (SDT), we developed a family of novel adaptive methods that estimate the sensitivity threshold-the signal intensity corresponding to a pre-defined sensitivity level (d' = 1)-in Yes-No (YN) and Forced-Choice (FC) detection tasks. Rather than focus stimulus sampling to estimate a single level of %Yes or %Correct, the current methods sample psychometric functions more broadly, to concurrently estimate sensitivity and decision factors, and thereby estimate thresholds that are independent of decision confounds. Developed for four tasks-(1) simple YN detection, (2) cued YN detection, which cues the observer's response state before each trial, (3) rated YN detection, which incorporates a Not Sure response, and (4) FC detection-the qYN and qFC methods yield sensitivity thresholds that are independent of the task's decision structure (YN or FC) and/or the observer's subjective response state. Results from simulation and psychophysics suggest that 25 trials (and sometimes less) are sufficient to estimate YN thresholds with reasonable precision (s.d. = 0.10-0.15 decimal log units), but more trials are needed for FC thresholds. When the same subjects were tested across tasks of simple, cued, rated, and FC detection, adaptive threshold estimates exhibited excellent agreement with the method of constant stimuli (MCS), and with each other. These YN adaptive methods deliver criterion-free thresholds that have previously been exclusive to FC methods.

Separating decision and encoding noise in signal detection tasks.

In this article we develop an extension to the signal detection theory framework to separately estimate internal noise arising from representational and decision processes. Our approach constrains signal detection theory models with decision noise by combining a multipass external noise paradigm with confidence rating responses. In a simulation study we present evidence that representation and decision noise can be separately estimated over a range of representative underlying representational and decision noise level configurations. These results also hold across a number of decision rules and show resilience to rule miss-specification. The new theoretical framework is applied to a visual detection confidence-rating task with 3 and 5 response categories. This study compliments and extends the recent efforts of researchers (Benjamin, Diaz, & Wee, 2009; Mueller & Weidemann, 2008; Rosner & Kochanski, 2009; Kellen, Klauer, & Singmann, 2012) to separate and quantify underlying sources of response variability in signal detection tasks.

Construction and evaluation of an integrated dynamical model of visual motion perception.

Although numerous models describe the individual neural mechanisms that may be involved in the perception of visual motion, few of them have been constructed to take arbitrary stimuli and map them to a motion percept. Here, we propose an integrated dynamical motion model (IDM), which is sufficiently general to handle diverse moving stimuli, yet sufficiently precise to account for a wide-ranging set of empirical observations made on a family of random dot kinematograms. In particular, we constructed models of the cortical areas involved in motion detection, motion integration and perceptual decision. We analyzed their parameters through dynamical simulations and numerical continuation to constrain their proper ranges. Then, empirical data from a family of random dot kinematograms experiments with systematically varying direction distribution, presentation duration and stimulus size, were used to evaluate our model and estimate corresponding model parameters. The resulting model provides an excellent account of a demanding set of parametrically varied behavioral effects on motion perception, providing both quantitative and qualitative elements of evaluation.

Generating a taxonomy of spatially cued attention for visual discrimination: effects of judgment precision and set size on attention.

Attention precues improve the performance of perceptual tasks in many but not all circumstances. These spatial attention effects may depend upon display set size or workload, and have been variously attributed to external noise filtering, stimulus enhancement, contrast gain, or response gain, or to uncertainty or other decision effects. In this study, we document systematically different effects of spatial attention in low- and high-precision judgments, with and without external noise, and in different set sizes in order to contribute to the development of a taxonomy of spatial attention. An elaborated perceptual template model (ePTM) provides an integrated account of a complex set of effects of spatial attention with just two attention factors: a set-size dependent exclusion or filtering of external noise and a narrowing of the perceptual template to focus on the signal stimulus. These results are related to the previous literature by classifying the judgment precision and presence of external noise masks in those experiments, suggesting a taxonomy of spatially cued attention in discrimination accuracy.