Situational Judgment Tests as a method for measuring personality: Development and validity evidence for a test of Dependability.

Situational Judgment Tests (SJTs) are criterion valid low fidelity measures that have gained much popularity as predictors of job performance. A broad variety of SJTs have been studied, but SJTs measuring personality are still rare. Personality traits such as Conscientiousness are valid predictors of many educational, work and life-related outcomes and SJTs are less prone to faking than classical self-report measurements. We developed an SJT measure of Dependability, a core facet of Conscientiousness, by gathering critical incidents in semi-structured interviews using the construct definition of Dependability as a prompt. We examined the psychometric properties of the newly developed SJTs across two studies (N = 546 general population; N = 440 sales professionals). The internal validity of the SJTs was examined by correlating the SJT scores with related self-report measures of Dependability and Conscientiousness, as well as testing the unidimensionality of the measure with CFA. Additionally, we specified a bi-factor model of SJT, self-report and behavioral checklist measures of Dependability accounting for common and specific measurement variance. External validity was examined by correlating the SJT scale and specific factor with work-related outcomes. The results show that the Dependability SJTs with an expert based scoring procedure were psychometrically sound and correlated moderately to highly with traditional self-report measures of Dependability and Conscientiousness. However, a large proportion of SJT variance cannot be accounted for by personality alone. This supports the notion that SJTs measure general domain knowledge about the effectiveness of personality-related behaviors. We conclude that SJT measures of personality can be a promising addition to classical self-report assessments and can be used in a wide variety of applications beyond measurement and selection, for instance as formative assessments of personality.

The testing effect in free recall is associated with enhanced organizational processes.

In two experiments with categorized lists, we asked whether the testing effect in free recall is related to enhancements in organizational processing. During a first phase in Experiment 1, subjects studied one list over eight consecutive trials, they studied another list six times while taking two interspersed recall tests, and they learned a third list in four alternating study and test trials. On a test 2 days later, recall was directly related to the number of tests and inversely related to the number of study trials. In addition, increased testing enhanced both the number of categories accessed and the number of items recalled from within those categories. One measure of organization also increased with the number of tests. In a second experiment, different groups of subjects studied a list either once or twice before a final criterial test, or they studied the list once and took an initial recall test before the final test. Prior testing again enhanced recall, relative to studying on the final test a day later, and also improved category clustering. The results suggest that the benefit of testing in free recall learning arises because testing creates retrieval schemas that guide recall.

Comprehension as a basis for metacognitive judgments: effects of effort after meaning on recall and metacognition.

We examined free recall and metacognitive judgments of ambiguous sentences studied with and without clues to facilitate their comprehension. Sentences were either studied without clues, with clues meaningfully embedded, or with clues following a 10-s interval delay. After presentation, subjects made judgments of comprehension (JCOMPs) or judgments of learning (JOLs). Puzzling over the meaning of sentences for several seconds prior to receiving the clue enhanced recall compared with studying sentences without clues or with embedded clues. This benefit of effort after meaning was not reflected in JCOMPs or JOLs. Rather, sentences considered relatively easy to understand received higher JOLs regardless of experimental condition. Although effort after meaning enhanced recall, subjects displayed no awareness of this benefit in their judgments. Our study adds to a growing literature showing students' ignorance of factors affecting their own learning, which have important implications for education. Making learning conditions more difficult, thus requiring students to engage more cognitive effort, often leads to enhanced retention.

Using popular films to enhance classroom learning: the good, the bad, and the interesting.

Popular history films sometimes contain major historical inaccuracies. Two experiments investigated how watching such films influences people's ability to remember associated texts. Subjects watched film clips and studied texts about various historical topics. Whereas the texts contained only correct information, the film clips contained both correct information (consistent with the text) and misinformation (contradicted by the text). Before watching each clip, subjects received a specific warning, a general warning, or no warning about the misinformation. One week later, they returned for a cued-recall test about the texts. Watching a film clip increased correct recall of consistent information relative to recall of the same information when subjects did not see the clip. However, when the information in the film contradicted the text, subjects often (falsely) recalled misinformation from the film. The specific warning substantially reduced this misinformation effect. Teachers should use popular history films with caution and should warn students about major inaccuracies in the films.

The effects of "effort after meaning" on recall: differences in within- and between-subjects designs.

In four experiments, we examined free recall of ambiguous sentences with or without corresponding cues to facilitate comprehension, using Auble and Franks's (1978) paradigm to examine effort after meaning (Bartlett, 1932). The ambiguous sentences were studied without cues, with cues meaningfully embedded in them, with cues provided shortly before the sentence (precue), or with cues following the sentence after several seconds (delayed cue). When these conditions were manipulated within subjects, the process of cue integration in the pre- and delayed-cue conditions enhanced recall, relative to the no-cue and embedded-cue conditions. Furthermore, in a test condition in which subjects first attempted to recall the cues alone, recall was also best in the delayed-cue condition. The effects described above did not occur when the cue presentation conditions varied between subjects, and on a test of order reconstruction, there was even an advantage for sentences studied in the embedded-cue condition over those studied in the delayed-cue condition. The dissociative effects of experimental design on sentence recall and order reconstruction suggest that effort after meaning might enhance memory for study items at the cost of impairing memory for temporal order information.

Temporal associations and prior-list intrusions in free recall.

When asked to recall the words from a just-presented target list, subjects occasionally recall words that were not on the list. These intrusions either appeared on earlier lists (prior-list intrusions, or PLIs) or had not appeared over the course of the experiment (extra-list intrusions). The authors examined the factors that elicit PLIs in free recall. A reanalysis of earlier studies revealed that PLIs tend to come from semantic associates as well as from recently studied lists, with the rate of PLIs decreasing sharply with list recency. The authors report 3 new experiments in which some items in a given list also appeared on earlier lists. Although repetition enhanced recall of list items, subjects were significantly more likely to make PLIs following the recall of repeated items, suggesting that temporal associations formed in earlier lists can induce recall errors. The authors interpret this finding as evidence for the interacting roles of associative and contextual retrieval processes in recall. Although contextual information helps to focus recall on words in the target list, it does not form an impermeable boundary between current- and prior-list experiences.