Conflict of interest explains the size of student evaluation of teaching and learning correlations in multisection studies: a meta-analysis.

We examined the associations between the size of student evaluation of teaching and learning (SET/learning) correlations and presence of several conflicts of interest (COIs) including corporate, administrative, evaluation unit, SET author, and funder interests. Our meta-analyses of SET/learning correlations reported by multisection studies show that researchers with a vested interest in finding large positive SET/learning correlations found, on average, large positive SET/learning correlations. In contrast, researchers with no identifiable COIs found that SET/learning correlations were zero or nearly zero. The largest SET/learning correlations were reported by authors with ties to SET selling corporations. Smaller but still substantial SET/learning correlations were reported by researchers with administrative assignments and by researchers in evaluation units/departments responsible for the administration of SET. Moreover, authors with the most significant COIs were publishing their studies primarily prior to 1981 whereas authors with no or less significant COIs were publishing their studies in 1981 or afterwards. Studies published prior to 1981 reported small but significant (r = .31) SET/learning correlations whereas studies published in 1981 and after reported near zero, non-significant SET/learning correlations (r = .06). The presence of COIs was associated with earlier publication date but also with smaller samples. Finally, whereas corporate, administrative, and evaluation unit authors nearly ceased publishing multisection studies on SET/learning correlations, authors from business and economics departments are now responsible for the substantial portion of newer, larger, and higher quality studies published in 1981 and after.

Meta-analyses of positive psychology interventions: The effects are much smaller than previously reported.

For at least four decades, researchers have studied the effectiveness of interventions designed to increase well-being. These interventions have become known as positive psychology interventions (PPIs). Two highly cited meta-analyses examined the effectiveness of PPIs on well-being and depression: Sin and Lyubomirsky (2009) and Bolier et al. (2013). Sin and Lyubomirsky reported larger effects of PPIs on well-being (r = .29) and depression (r = .31) than Bolier et al. reported for subjective well-being (r = .17), psychological well-being (r = .10), and depression (r = .11). A detailed examination of the two meta-analyses reveals that the authors employed different approaches, used different inclusion and exclusion criteria, analyzed different sets of studies, described their methods with insufficient detail to compare them clearly, and did not report or properly account for significant small sample size bias. The first objective of the current study was to reanalyze the studies selected in each of the published meta-analyses, while taking into account small sample size bias. The second objective was to replicate each meta-analysis by extracting relevant effect sizes directly from the primary studies included in the meta-analyses. The present study revealed three key findings: (1) many of the primary studies used a small sample size; (2) small sample size bias was pronounced in many of the analyses; and (3) when small sample size bias was taken into account, the effect of PPIs on well-being were small but significant (approximately r = .10), whereas the effect of PPIs on depression were variable, dependent on outliers, and generally not statistically significant. Future PPI research needs to focus on increasing sample sizes. A future meta-analyses of this research needs to assess cumulative effects from a comprehensive collection of primary studies while being mindful of issues such as small sample size bias.

Prospective memory, retrospective memory, and individual differences in cognitive abilities, personality, and psychopathology.

Although individual differences in processing speed, working memory, intelligence, and other cognitive functions were found to explain individual differences in retrospective memory (RetM), much less is known about their relationship with prospective memory (ProM). Moreover, the studies that investigated the relationship between ProM and cognitive functions arrived to contradictory conclusions. The relationship between ProM, personality, and psychopathology is similarly unsettled. Meta-analytic reviews of the relationships of ProM with aging and personality suggest that the contradictory findings may be due to widespread methodological problems plaguing ProM research including the prevalent use of inefficient, unreliable binary measures; widespread ceiling effects; failure to distinguish between various ProM subdomains (e.g., episodic ProM versus vigilance/monitoring); various confounds; and, importantly, small sample sizes, resulting in insufficient statistical power. Accordingly, in a large scale study with nearly 1,200 participants, we investigated the relationship between episodic event-cued ProM, episodic RetM, and fundamental cognitive functions including intelligence, personality, and psychopathology, using reliable continuous measures of episodic event-cued ProM. Our findings show that (a) continuous measures of episodic event-cued ProM were much more reliable than binary measures, (b) episodic event-cued ProM was associated with measures of processing speed, working memory, crystallized and fluid intelligence, as well as RetM, and that such associations were similar for ProM and RetM, (c) personality factors did not improve prediction of neither ProM nor RetM beyond the variance predicted by cognitive ability, (d) symptoms of psychopathology did not improve the prediction of ProM although they slightly improved the prediction of RetM, and (e) participants' sex was not associated with ProM but showed small correlations with RetM. In addition to advancing our theoretical understanding of ProM, our findings highlight the need to avoid common pitfalls plaguing ProM research.

Student evaluations of teaching: teaching quantitative courses can be hazardous to one's career.

Anonymous student evaluations of teaching (SETs) are used by colleges and universities to measure teaching effectiveness and to make decisions about faculty hiring, firing, re-appointment, promotion, tenure, and merit pay. Although numerous studies have found that SETs correlate with various teaching effectiveness irrelevant factors (TEIFs) such as subject, class size, and grading standards, it has been argued that such correlations are small and do not undermine the validity of SETs as measures of professors' teaching effectiveness. However, previous research has generally used inappropriate parametric statistics and effect sizes to examine and to evaluate the significance of TEIFs on personnel decisions. Accordingly, we examined the influence of quantitative vs. non-quantitative courses on SET ratings and SET based personnel decisions using 14,872 publicly posted class evaluations where each evaluation represents a summary of SET ratings provided by individual students responding in each class. In total, 325,538 individual student evaluations from a US mid-size university contributed to theses class evaluations. The results demonstrate that class subject (math vs. English) is strongly associated with SET ratings, has a substantial impact on professors being labeled satisfactory vs. unsatisfactory and excellent vs. non-excellent, and the impact varies substantially depending on the criteria used to classify professors as satisfactory vs. unsatisfactory. Professors teaching quantitative courses are far more likely not to receive tenure, promotion, and/or merit pay when their performance is evaluated against common standards.

The numbers tell it all: students don't like numbers!

Undergraduate Students' interest in taking quantitative vs. non quantitative courses has received limited attention even though it has important consequences for higher education. Previous studies have collected course interest ratings at the end of the courses as part of student evaluation of teaching (SET) ratings, which may confound prior interest in taking these courses with students' actual experience in taking them. This study is the first to examine undergraduate students' interest in quantitative vs. non quantitative courses in their first year of studies before they have taken any quantitative courses. Three hundred and forty students were presented with descriptions of 44 psychology courses and asked to rate their interest in taking each course. Student interest in taking quantitative vs non quantitative courses was very low; the mean interest in statistics courses was nearly 6 SDs below the mean interest in non quantitative courses. Moreover, women were less interested in taking quantitative courses than men. Our findings have several far-reaching implications. First, evaluating professors teaching quantitative vs. non quantitative courses against the same SET standard may be inappropriate. Second, if the same SET standard is used for the evaluation of faculty teaching quantitative vs. non quantitative courses, faculty are likely to teach to SETs rather than focus on student learning. Third, universities interested primarily in student satisfaction may want to expunge quantitative courses from their curricula. In contrast, universities interested in student learning may want to abandon SETs as a primary measure of faculty teaching effectiveness. Fourth, undergraduate students who are not interested in taking quantitative courses are unlikely to pursue graduate studies in quantitative psychology and unlikely to be able to competently analyze data independently.

Prospective memory, personality, and individual differences.

A number of studies investigating the relationship between personality and prospective memory (ProM) have appeared during the last decade. However, a review of these studies reveals little consistency in their findings and conclusions. To clarify the relationship between ProM and personality, we conducted two studies: a meta-analysis of prior research investigating the relationships between ProM and personality, and a study with 378 participants examining the relationships between ProM, personality, verbal intelligence, and retrospective memory. Our review of prior research revealed great variability in the measures used to assess ProM, and in the methodological quality of prior research; these two factors may partially explain inconsistent findings in the literature. Overall, the meta-analysis revealed very weak correlations (rs ranging from 0.09 to 0.10) between ProM and three of the Big Five factors: Openness, Conscientiousness, and Agreeableness. Our experimental study showed that ProM performance was related to individual differences such as verbal intelligence as well as to personality factors and that the relationship between ProM and personality factors depends on the ProM subdomain. In combination, the two studies suggest that ProM performance is relatively weakly related to personality factors and more strongly related to individual differences in cognitive factors.

Self-report measures of prospective memory are reliable but not valid.

Are self-report measures of prospective memory (ProM) reliable and valid? To examine this question, 240 undergraduate student volunteers completed several widely used self-report measures of ProM including the Prospective Memory Questionnaire (PMQ), the Prospective and Retrospective Memory Questionnaire (PRMQ), the Comprehensive Assessment of Prospective Memory (CAPM) questionnaire, self-reports of retrospective memory (RetM), objective measures of ProM and RetM, and measures of involvement in activities and events, memory strategies and aids use, personality and verbal intelligence. The results showed that both convergent and divergent validity of ProM self-reports are poor, even though we assessed ProM using a newly developed, reliable continuous measure. Further analyses showed that a substantial proportion of variability in ProM self-report scores was due to verbal intelligence, personality (conscientiousness, neuroticism), activities and event involvement (busyness), and use of memory strategies and aids. ProM self-reports have adequate reliability, but poor validity and should not be interpreted as reflecting ProM ability.

Transparent meta-analysis: does aging spare prospective memory with focal vs. non-focal cues?

BACKGROUND: Prospective memory (ProM) is the ability to become aware of a previously-formed plan at the right time and place. For over twenty years, researchers have been debating whether prospective memory declines with aging or whether it is spared by aging and, most recently, whether aging spares prospective memory with focal vs. non-focal cues. Two recent meta-analyses examining these claims did not include all relevant studies and ignored prevalent ceiling effects, age confounds, and did not distinguish between prospective memory subdomains (e.g., ProM proper, vigilance, habitual ProM) (see Uttl, 2008, PLoS ONE). The present meta-analysis focuses on the following questions: Does prospective memory decline with aging? Does prospective memory with focal vs. non-focal cues decline with aging? Does the size of age-related declines with focal vs. non-focal cues vary across ProM subdomains? And are age-related declines in ProM smaller than age-related declines in retrospective memory? METHODS AND FINDINGS: A meta-analysis of event-cued ProM using data visualization and modeling, robust count methods, and conventional meta-analysis techniques revealed that first, the size of age-related declines in ProM with both focal and non-focal cues are large. Second, age-related declines in ProM with focal cues are larger in ProM proper and smaller in vigilance. Third, age-related declines in ProM proper with focal cues are as large as age-related declines in recall measures of retrospective memory. CONCLUSIONS: The results are consistent with Craik's (1983) proposal that age-related declines on ProM tasks are generally large, support the distinction between ProM proper vs. vigilance, and directly contradict widespread claims that ProM, with or without focal cues, is spared by aging.

Meaning of missing values in eyewitness recall and accident records.

BACKGROUND: Eyewitness recalls and accident records frequently do not mention the conditions and behaviors of interest to researchers and lead to missing values and to uncertainty about the prevalence of these conditions and behaviors surrounding accidents. Missing values may occur because eyewitnesses report the presence but not the absence of obvious clues/accident features. We examined this possibility. METHODOLOGY/PRINCIPAL FINDINGS: Participants watched car accident videos and were asked to recall as much information as they could remember about each accident. The results showed that eyewitnesses were far more likely to report the presence of present obvious clues than the absence of absent obvious clues even though they were aware of their absence. CONCLUSIONS: One of the principal mechanisms causing missing values may be eyewitnesses' tendency to not report the absence of obvious features. We discuss the implications of our findings for both retrospective and prospective analyses of accident records, and illustrate the consequences of adopting inappropriate assumptions about the meaning of missing values using the Avaluator Avalanche Accident Prevention Card.

Ceiling effects make Hughes and Nicholson's data analyses and conclusions inconclusive.

Hughes and Nicholson (2010) suggest that recognizing oneself is easier from face vs. voice stimuli, that a combined presentation of face and voice actually inhibits self-recognition relative to presentation of face or voice alone, that the left hemisphere is superior in self-recognition to the right hemisphere, and that recognizing self requires more effort than recognizing others. A re-examination of their method, data, and analyses unfortunately shows important ceiling effects that cast doubts on these conclusions.

Saccadic compression of rectangle and Kanizsa figures: now you see it, now you don't.

BACKGROUND: Observers misperceive the location of points within a scene as compressed towards the goal of a saccade. However, recent studies suggest that saccadic compression does not occur for discrete elements such as dots when they are perceived as unified objects like a rectangle. METHODOLOGY/PRINCIPAL FINDINGS: We investigated the magnitude of horizontal vs. vertical compression for Kanizsa figure (a collection of discrete elements unified into single perceptual objects by illusory contours) and control rectangle figures. Participants were presented with Kanizsa and control figures and had to decide whether the horizontal or vertical length of stimulus was longer using the two-alternative force choice method. Our findings show that large but not small Kanizsa figures are perceived as compressed, that such compression is large in the horizontal dimension and small or nil in the vertical dimension. In contrast to recent findings, we found no saccadic compression for control rectangles. CONCLUSIONS: Our data suggest that compression of Kanizsa figure has been overestimated in previous research due to methodological artifacts, and highlight the importance of studying perceptual phenomena by multiple methods.

The influence of object relative size on priming and explicit memory.

We investigated the effects of object relative size on priming and explicit memory for color photos of common objects. Participants were presented with color photos of pairs of objects displayed in either appropriate or inappropriate relative sizes. Implicit memory was assessed by speed of object size ratings whereas explicit memory was assessed by an old/new recognition test. Study-to-test changes in relative size reduced both priming and explicit memory and had large effects for objects displayed in large vs. small size at test. Our findings of substantial size-specific influences on priming with common objects under some but not other conditions are consistent with instance views of object perception and priming but inconsistent with structural description views.

Transparent meta-analysis of prospective memory and aging.

Prospective memory (ProM) refers to our ability to become aware of a previously formed plan at the right time and place. After two decades of research on prospective memory and aging, narrative reviews and summaries have arrived at widely different conclusions. One view is that prospective memory shows large age declines, larger than age declines on retrospective memory (RetM). Another view is that prospective memory is an exception to age declines and remains invariant across the adult lifespan. The present meta-analysis of over twenty years of research settles this controversy. It shows that prospective memory declines with aging and that the magnitude of age decline varies by prospective memory subdomain (vigilance, prospective memory proper, habitual prospective memory) as well as test setting (laboratory, natural). Moreover, this meta-analysis demonstrates that previous claims of no age declines in prospective memory are artifacts of methodological and conceptual issues afflicting prior research including widespread ceiling effects, low statistical power, age confounds, and failure to distinguish between various subdomains of prospective memory (e.g., vigilance and prospective memory proper).

Influence of object size on baseline identification, priming, and explicit memory.

We investigated the influence of size on identification, priming, and explicit memory for color photos of common objects. Participants studied objects displayed in small, medium, and large sizes and memory was assessed with both implicit identification and explicit recognition tests. Overall, large objects were easier to identify than small objects and study-to-test changes in object size impeded performance on explicit but not implicit memory tests. In contrast to previous findings with line-drawings of objects but consistent with predictions from the distance-as-filtering hypothesis, we found that study-test size manipulations had large effects on old/new recognition memory test for objects displayed in large size at test but not for objects displayed small or medium at test. Our findings add to the growing body of literature showing that the findings obtained using line-drawings of objects do not necessarily generalize to color photos of common objects. We discuss implications of our findings for theories of object perception, memory, and eyewitness identification accuracy for objects.

Object color affects identification and repetition priming.

We investigated the influence of color on the identification of both non-studied and studied objects. Participants studied black and white and color photos of common objects and memory was assessed with an identification test. Consistent with our meta-analysis of prior research, we found that objects were easier to identify from color than from black and white photos. We also found substantial priming in all conditions, and study-to-test changes in an object's color reduced the magnitude of priming. Color-specific priming effects were large for color-complex objects, but minimal for color-simple objects. The pattern and magnitude of priming effects was not influenced either by the extent to which an object always appears in the same color (i.e., whether a color is symptomatic of an object) or by the object's origin (natural versus fabricated). We discuss the implications of our findings for theoretical accounts of object perception and repetition priming.

Behavioral inhibition and prefrontal cortex in decision-making.

Every day we make innumerable decisions; some require no effort at all whereas others require considerable deliberation and weighing of various options. Despite the importance of decision-making in our lives and increased research interest, the specific neural mechanisms underlying decision-making remain unclear. We propose that the brain has at least two cortical pathways that independently generate a decision about appropriate behavior in given circumstances. These two pathways are extensions of the dorsal and ventral streams of the visual processing pathways. The parieto-premotor (extended dorsal) pathway makes decisions about motor actions in a largely autonomous and automatic fashion whereas the temporo-ventrolateral prefrontal (extended ventral) pathway is involved primarily in deliberate decisions and inhibitory control over behavior through the inhibitory function of the ventrolateral prefrontal cortex.

Age-related changes in event-cued visual and auditory prospective memory proper.

We rely upon prospective memory proper (ProMP) to bring back to awareness previously formed plans and intentions at the right place and time, and to enable us to act upon those plans and intentions. To examine age-related changes in ProMP, younger and older participants made decisions about simple stimuli (ongoing task) and at the same time were required to respond to a ProM cue, either a picture (visually cued ProM test) or a sound (auditorily cued ProM test), embedded in a simultaneously presented series of similar stimuli (either pictures or sounds). The cue display size or loudness increased across trials until a response was made. The cue size and cue loudness at the time of response indexed ProMP. The main results showed that both visual and auditory ProMP declined with age, and that such declines were mediated by age declines in sensory functions (visual acuity and hearing level), processing resources, working memory, intelligence, and ongoing task resource allocation.

Measurement of individual differences: lessons from memory assessment in research and clinical practice.

An examination of test manuals and published research indicates that widely used memory tests (e.g., Verbal Paired Associates and Word List tests of the Wechsler Memory Scale, Rey Auditory Verbal Learning Test, and California Verbal Learning Test) are afflicted by severe ceiling effects. In the present study, the true extent of memory ability in healthy young adults was tested by giving 208 college undergraduates verbal paired-associate and verbal learning tests of various lengths; the findings demonstrate that healthy adults can remember much more than is suggested by the normative data for the memory tests just mentioned. The findings highlight the adverse effects of low ceilings in memory assessment and underscore the severe consequences of ceiling effects on score distributions, means, standard deviations, and all variability-dependent indices, such as reliability, validity, and correlations with other tests. The article discusses the optimal test lengths for verbal paired-associate and verbal list-learning tests, shows how to identify ceiling-afflicted data in published research, and explains how proper attention to this phenomenon can improve future research and clinical practice.

Rising verbal intelligence scores: implications for research and clinical practice.

Evidence suggests that scores on various intelligence tests have been rising at a fast rate. To find out whether performance on the Wechsler Adult Intelligence Scale (WAIS) Vocabulary subtest has also been rising, the authors searched major psychology journals for investigations involving healthy younger and older adult participants and collected the reported WAIS Vocabulary scores. The meta-analysis shows that WAIS Vocabulary scores have been rising at the rate of 0.117/year (corresponding to 1.52 IQ points/decade) for younger adults and 0.367/year (corresponding to 4.79 IQ points/decade) for older adults.