Learning from errors versus explicit instruction in preparation for a test that counts.

BACKGROUND: Although the generation of errors has been thought, traditionally, to impair learning, recent studies indicate that, under particular feedback conditions, the commission of errors may have a beneficial effect. AIMS: This study investigates the teaching strategies that facilitate learning from errors. MATERIALS AND METHODS: This 2-year study, involving two cohorts of ~88 students each, contrasted a learning-from-errors (LFE) with an explicit instruction (EI) teaching strategy in a multi-session implementation directed at improving student performance on the high-stakes New York State Algebra 1 Regents examination. In the LFE condition, instead of receiving instruction on 4 sessions, students took mini-tests. Their errors were isolated to become the focus of 4 teacher-guided feedback sessions. In the EI condition, teachers explicitly taught the mathematical material for all 8 sessions. RESULTS: Teacher time-on in the LFE condition produced a higher rate of learning than did teacher time-on in the EI condition. The learning benefit in the LFE condition was, however, inconsistent across teachers. Second-by-second analyses of classroom activities, directed at isolating learning-relevant differences in teaching style revealed that a highly interactive mode of engaging the students in understanding their errors was more conducive to learning than was teaching directed at getting to the correct solution, either by lecturing about corrections or by interaction focused on corrections. CONCLUSION: These results indicate that engaging the students interactively to focus on errors, and the reasons for them, facilitates productive failure and learning from errors.

Going beyond the spacing effect: Does it matter how time on a task is distributed?

We assessed the effects of removing some constraints that characterise traditional experiments on the effects of spaced, rather than massed, study opportunities. In five experiments-using lists of to-be-remembered words-we examined the effects of how total study time was distributed across multiple repetitions of a given to-be-remembered word. Overall, within a given list, recall profited from study time being distributed (e.g., four 1-s presentations or two 2-s presentations vs one 4-s presentation). Among the implications of these findings is that if students choose to engage in massed studying (by virtue of constraints on their study time or a failure to appreciate the benefits of spaced study sessions), then studying the information twice but for half the time may produce memory benefits in a single study session.

Improving conceptual learning via pretests.

Although examples can be structured to emphasize diagnostic features of concepts, novice learners tend to focus on irrelevant surface features and struggle to encode deeper structures. Experiment 1 examined whether pretesting-answering questions about content before it is studied-could enhance learners' noticing of diagnostic features, making them easier to process during subsequent study. Participants studied statistical concepts with examples that emphasized surface details or deep structure, and then classified new examples of these concepts. Studying examples that emphasized deep structure increased classification performance compared to examples that emphasized surface details. Moreover, taking pretests prior to studying the examples increased classification performance and eliminated differential benefits of studying structure versus surface examples. Experiment 2 examined whether pretesting serves a role beyond directing attention. After studying different statistical concepts with only surface-emphasizing examples, classification performance was better when participants actually took pretests compared to being given the correct responses. It is the generative aspect of pretesting, beyond attention directing, that improves conceptual learning among novice learners. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Where and how to learn: The interactive benefits of contextual variation, restudying, and retrieval practice for learning.

Students are often advised to do all of their studying in one good place, but restudying to-be-learned material in a new context can enhance subsequent recall. We examined whether there are similar benefits for testing. In Experiment 1 (n = 106), participants studied a 36-word list and 48 hr later-when back in the same or a new context-either restudied or recalled the list without feedback. After another 48 hr, all participants free-recalled the list in a new context. Experiment 2 (n = 203) differed by having the testing-condition participants restudy the list before being tested. Across both experiments, testing in a new context reduced recall, which carried over to the final test, whereas restudying in a new context did not impair (and in Experiment 2, significantly enhanced) recall. These findings reveal critical interactions between contextual-variation and retrieval-practice effects, which we interpret as consistent with a distribution-of-memory-strengths framework.

Feedback at Test Can Reverse the Retrieval-Effort Effect.

The testing effect refers to the finding that retrieving previously encoded material typically improves subsequent recall performance more on a later test than does restudying that material. Storm et al. (2014) demonstrated, however, that when feedback is provided on such a later test the testing advantage then turns to a restudying advantage on subsequent tests. The goal of the present research was to examine whether there is a similar consequence of feedback when the difficulty of initial retrieval practice is modulated. Replicating prior research, we found that on an initial delayed test, recall of to-be-learned items was better following difficult than easy practice. Critically, however, providing immediate feedback on an initial delayed test reversed this pattern. Our findings are consistent with a distribution-based interpretation of how feedback at test modifies recall performance.

Forgetting as the friend of learning: implications for teaching and self-regulated learning.

One of the "important peculiarities" of human learning (Bjork RA and Bjork EL. From Learning Processes to Cognitive Processes: Essays in Honor of William K. Estes, 1992, p. 35-67) is that certain conditions that produce forgetting-that is, impair access to some to-be-learned information studied earlier-also enhance the learning of that information when it is restudied. Such conditions include changing the environmental context from when some to-be-learned material is studied to when that material is restudied; increasing the delay from when something is studied to when it is tested or restudied; and interleaving, rather than blocking, the study or practice of the components of to-be-learned knowledge or skills. In this paper, we provide some conjectures as to why conditions that produce forgetting can also enable learning, and why a misunderstanding of this peculiarity of how humans learn can result in nonoptimal teaching and self-regulated learning.

Being Suspicious of the Sense of Ease and Undeterred by the Sense of Difficulty: Looking Back at Schmidt and Bjork (1992).

Richard Schmidt and I titled our article "New Conceptualizations of Practice: Common Principles in Three Paradigms Suggest New Concepts for Training" to reflect our view that prevailing ideas about how to optimize teaching, learning, and practicing were, in our words, "at best incomplete, and at worst incorrect." We argued that teachers and trainers were susceptible to being misled by two commonsense assumptions-namely, that procedures that enhance performance during training are the procedures of choice and that the context of training needs to match in detail the posttraining context that is the target of training. A variety of then-recent experimental findings challenged both assumptions and demonstrated, in particular, that procedures posing certain difficulties and appearing to slow the rate of learning often enhanced long-term retention and transfer of to-be-learned skills and knowledge. Given the parallel nature of such findings for both motor and verbal learning, we concluded that principles of considerable generality could be deduced to upgrade teaching and training.

How should exemplars be sequenced in inductive learning? Empirical evidence versus learners' opinions.

The sequencing of exemplars during study can have a large effect on category or concept induction. Counter to learners' intuitions, interleaving exemplars from different categories is often more effective for learning the different underlying categories than is blocking all the exemplars by category (e.g., Kornell & Bjork, 2008). Prior research suggests that blocking and interleaving each support different aspects of induction: Interleaving appears to enhance between-category discrimination, whereas blocking appears to promote the learning of within-category commonalities. In Experiments 1 and 2, participants studied paintings by 12 artists and were asked to induce the different artists' painting styles. We explored whether hybrid schedules can leverage the benefits of both types of schedules, comparing blocked, interleaved, and 3 hybrid schedules-blocked-to-interleaved, interleaved-to-blocked, and miniblocks. The miniblocks and blocked-to-interleaved schedules were as effective, statistically, but not better than pure interleaving. The blocked schedule led to the worst performance. In Experiments 3 and 4, we explored participants' a priori beliefs by having them self-schedule hypothetical future category-learning tasks. Although participants demonstrated some metacognitive sophistication with respect to the relative benefits of blocked and interleaved study, pure interleaving was the least popular schedule, despite its being one of the most, effective schedules for learning. (PsycINFO Database Record

Do learners predict a shift from recency to primacy with delay?

The shift from recency to primacy with delay reflects a fundamental observation in the study of memory. As time passes, the accessibility of earlier-learned representations tends to increase relative to the accessibility of later-learned representations. In three experiments involving participants' memory for text materials, we examined whether participants understood that there might be such a shift with retention interval. In marked contrast to their actual performance, participants predicted recency effects at both shorter and longer retention intervals. Our findings add to the evidence that the storage and retrieval dynamics of the human memory system, though adaptive overall from a statistics-of-use standpoint, are both complex and poorly understood by users of the system.

Optimal sequencing during category learning: Testing a dual-learning systems perspective.

Recent studies demonstrate that interleaving the exemplars of different categories, rather than blocking exemplars by category, can enhance inductive learning-the ability to categorize new exemplars-presumably because interleaving affords discriminative contrasts between exemplars from different categories. Consistent with this view, other studies have demonstrated that decreasing between-category similarity and increasing within-category variability can eliminate or even reverse the interleaving benefit. We tested another hypothesis, one based on the dual-learning systems framework-namely, that the optimal schedule for learning categories should depend on an interaction of the cognitive system that mediates learning and the structure of the particular category being learned. Blocking should enhance rule-based category learning, which is mediated by explicit, hypothesis-testing processes, whereas interleaving should enhance information-integration category learning, which is mediated by an implicit, procedural-based learning system. Consistent with this view, we found a crossover interaction between schedule (blocked vs. interleaved) and category structure (rule-based vs. information-integration).

On the difficulty of mending metacognitive illusions: A priori theories, fluency effects, and misattributions of the interleaving benefit.

Interleaving exemplars of to-be-learned categories-rather than blocking exemplars by category-typically enhances inductive learning. Learners, however, tend to believe the opposite, even after their own performance has benefited from interleaving. In Experiments 1 and 2, the authors examined the influence of 2 factors that they hypothesized contribute to the illusion that blocking enhances inductive learning: Namely, that (a) blocking creates a sense of fluent extraction during study of the features defining a given category, and (b) learners come to the experimental task with a pre-existing belief that blocking instruction by topic is superior to intermixing topics. In Experiments 3-5, the authors attempted to uproot learners' belief in the superiority of blocking through experience-based and theory-based debiasing techniques by (a) providing detailed theory-based information as to why blocking seems better, but is not, and (b) explicitly drawing attention to the link between study schedule and subsequent performance, both of which had only modest effects. Only when they disambiguated test performance on the 2 schedules by separating them (Experiment 6) did the combination of experience- and theory-based debiasing lead a majority of learners to appreciate interleaving. Overall, the results indicate that 3 influences combine to make altering learners' misconceptions difficult: the sense of fluency that can accompany nonoptimal modes of instruction; pre-existing beliefs learners bring to new tasks; and the willingness, even eagerness, to believe that 1 is unique as a learner-that what enhances others' learning differs from what enhances one's own learning. (PsycINFO Database Record

Impaired Retrieval Inhibition of Threat Material in Generalized Anxiety Disorder.

Generalized Anxiety Disorder (GAD) is characterized by cognitive biases toward threat-relevant information, but the underlying mechanisms are unclear. We translated a retrieval-practice paradigm from cognitive science to investigate impaired inhibition of threat information as one such mechanism. Participants diagnosed with GAD and never-disordered control participants learned a series of cue-target pairs; whereas some cues were associated only with neutral targets, others were associated with both neutral and threat targets. Next, participants practiced retrieving neutral targets, which typically suppresses the subsequent recall of unpracticed associated targets (retrieval-induced forgetting; RIF). Finally, participants were tested on their recall of all targets. Despite showing intact RIF of neutral targets, the GAD group failed to exhibit RIF of threat targets. Furthermore, within the GAD group, less RIF of threat targets correlated with greater pervasiveness of worry. Deficits in inhibitory control over threat-relevant information may underlie the cognitive pathology of GAD, which has important treatment implications.

On the learning benefits of confidence-weighted testing.

Taking multiple-choice practice tests with competitive incorrect alternatives can enhance performance on related but different questions appearing on a later cued-recall test (Little et al., Psychol Sci 23:1337-1344, 2012). This benefit of multiple-choice testing, which does not occur when the practice test is a cued-recall test, appears attributable to participants attempting to retrieve not only why the correct alternative is correct but also why the other alternatives are incorrect. The present research was designed to examine whether a confidence-weighted multiple-choice format in which test-takers were allowed to indicate their relative confidence in the correctness of one alternative compared with the others (Bruno, J Econ Educ 20:5-22, 1989; Bruno, Item banking: Interactive testing and self-assessment: Volume 112 of NATO ASI Series, pp. 190-209, 1993) might increase the extent to which participants engaged in such productive retrievals. In two experiments, such confidence-weighted practice tests led to greater benefits in the ability of test-takers to answer new but related questions than did standard multiple-choice practice tests. These results point to ways to make multiple-choice testing a more powerful tool for learning.

Retrieval-induced forgetting is associated with increased positivity when imagining the future.

People often think of themselves and their experiences in a more positive light than is objectively justified. Inhibitory control processes may promote this positivity bias by modulating the accessibility of negative thoughts and episodes from the past, which then limits their influence in the construction of imagined future events. We tested this hypothesis by investigating the correlation between retrieval-induced forgetting and the extent to which individuals imagine positive and negative episodic future events. First, we measured performance on a task requiring participants to imagine personal episodic events (either positive or negative), and then we correlated that measure with retrieval-induced forgetting. As predicted, individuals who exhibited higher levels of retrieval-induced forgetting imagined fewer negative episodic future events than did individuals who exhibited lower levels of retrieval-induced forgetting. This finding provides new insight into the possible role of retrieval-induced forgetting in autobiographical memory.

The Critical Importance of Retrieval--and Spacing--for Learning.

We examined the impact of repeated testing and repeated studying on long-term learning. In Experiment 1, we replicated Karpicke and Roediger's (2008) influential results showing that once information can be recalled, repeated testing on that information enhances learning, whereas restudying that information does not. We then examined whether the apparent ineffectiveness of restudying might be attributable to the spacing differences between items that were inherent in the between-subjects design employed by Karpicke and Roediger. When we controlled for these spacing differences by manipulating the various learning conditions within subjects in Experiment 2, we found that both repeated testing and restudying improved learning, and that learners' awareness of the relative mnemonic benefits of these strategies was enhanced. These findings contribute to understanding how two important factors in learning-test-induced retrieval processes and spacing-can interact, and they illustrate that such interactions can play out differently in between-subjects and within-subjects experimental designs.

Explaining retrieval-induced forgetting: A change in mental context between the study and restudy practice phases is not sufficient to cause forgetting.

Retrieving information can impair the subsequent recall of related information. Such retrieval-induced forgetting is often attributed to inhibitory mechanisms, but Jonker, MacLeod, and Seli (2013) recently proposed an alternative account. In their view, the study and retrieval-practice phases constitute two disparate contexts, and impairment of unpractised members from practised categories is attributable to their being absent from the retrieval-practice context, which is where, according to Jonker et al., participants preferentially search at the time of final test. In evidence of this account, Jonker et al. showed that even restudy practice-which is assumed by the inhibitory account to be insufficient to cause forgetting (i.e., retrieval-specificity)-can cause forgetting when a mental context change is inserted between study and restudy. The present research sought to replicate this finding while also testing the possibility that a far mental context change would cause more forgetting than a near mental context change. In Experiment 1, participants described a vacation inside the United States (near) or outside the United States (far). In Experiments 2 and 3, participants described the layout of their own home (near) or their parents' home (far). In contrast to the predictions of the context account, however, but consistent with the predictions of the inhibitory account, none of the restudy-plus-context-change conditions resulted in significant forgetting.

Desirable Difficulties in Vocabulary Learning.

In this article we discuss the role of desirable difficulties in vocabulary learning from two perspectives, one having to do with identifying conditions of learning that impose initial challenges to the learner but then benefit later retention and transfer, and the other having to do with the role of certain difficulties that are intrinsic to language processes, are engaged during word learning, and reflect how language is understood and produced. From each perspective we discuss evidence that supports the notion that difficulties in learning and imposed costs to language processing may produce benefits because they are likely to increase conceptual understanding. We then consider the consequences of these processes for actual second-language learning and suggest that some of the domain-general cognitive advantages that have been reported for proficient bilinguals may reflect difficulties imposed by the learning process, and by the requirement to negotiate cross-language competition, that are broadly desirable. As Alice Healy and her collaborators were perhaps the first to demonstrate, research on desirable difficulties in vocabulary and language learning holds the promise of bringing together research traditions on memory and language that have much to offer each other.

Learning versus performance: an integrative review.

The primary goal of instruction should be to facilitate long-term learning-that is, to create relatively permanent changes in comprehension, understanding, and skills of the types that will support long-term retention and transfer. During the instruction or training process, however, what we can observe and measure is performance, which is often an unreliable index of whether the relatively long-term changes that constitute learning have taken place. The time-honored distinction between learning and performance dates back decades, spurred by early animal and motor-skills research that revealed that learning can occur even when no discernible changes in performance are observed. More recently, the converse has also been shown-specifically, that improvements in performance can fail to yield significant learning-and, in fact, that certain manipulations can have opposite effects on learning and performance. We review the extant literature in the motor- and verbal-learning domains that necessitates the distinction between learning and performance. In addition, we examine research in metacognition that suggests that people often mistakenly interpret their performance during acquisition as a reliable guide to long-term learning. These and other considerations suggest that the learning-performance distinction is critical and has vast practical and theoretical implications.

Multiple-choice tests stabilize access to marginal knowledge.

Marginal knowledge refers to knowledge that is stored in memory, but is not accessible at a given moment. For example, one might struggle to remember who wrote The Call of the Wild, even if that knowledge is stored in memory. Knowing how best to stabilize access to marginal knowledge is important, given that new learning often requires accessing and building on prior knowledge. While even a single opportunity to restudy marginal knowledge boosts its later accessibility (Berger, Hall, & Bahrick, 1999), in many situations explicit relearning opportunities are not available. Our question is whether multiple-choice tests (which by definition expose the learner to the correct answers) can also serve this function and, if so, how testing compares to restudying given that tests can be particularly powerful learning devices (Roediger & Karpicke, 2006). In four experiments, we found that multiple-choice testing had the power to stabilize access to marginal knowledge, and to do so for at least up to a week. Importantly, such tests did not need to be paired with feedback, although testing was no more powerful than studying. Overall, the results support the idea that one's knowledge base is unstable, with individual pieces of information coming in and out of reach. The present findings have implications for a key educational challenge: ensuring that students have continuing access to information they have learned.