Revisiting Sackett et al.'s (2022) rationale behind their recommendation against correcting for range restriction in concurrent validation studies.

Sackett et al. (2022) recommend against correcting for range restriction (RR) in concurrent validation studies. The main rationale behind their recommendation is that unless "rzx" (an unrestricted true-score correlation between the third variable Z where actual selection occurred in a top-down manner [a.k.a., suitability] and the predictor of interest, X) is as high as .90 and selection ratios are as low as .30-both unlikely events in their view, the degree of RR (ux) in concurrent validation studies is unlikely to be low enough (i.e., lower than .90) to warrant RR correction. That is, (a) the "rzx" ≥ .90 and (b) the selection ratio ≤ .30 are two critical conditions for the third condition, (c) ux ≤ .90, a need for RR correction. In this study, we revisit each of these conditions that constitute the rationale behind their recommendation: (a) whether "rzx" is unlikely to be as high as .90; (b) whether selection ratios of .30 or lower are "extreme"; and (c) whether the degree of RR is "little to no" (i.e., ux ≥ .90) in concurrent validation studies, thus no need for correcting for RR in concurrent validation studies. First, our reanalysis of their Table 1 indicates that it is not implausible that "rzx" is as high as .90. Second, several studies report that selection ratios of .30 or lower are not extreme. Finally, our reanalysis of their Table 5 indicates that Sackett et al. substantially underestimate the severity of RR and its biasing effect on operational validity in concurrent validation studies due to their use of a particular RR correction method (Case IV). We believe these findings suggest that there is not sufficient support for the rationale behind Sackett et al.'s recommendation and, thus, their recommendation itself should be reconsidered. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

A multirater perspective on personality and performance: An empirical examination of the trait-reputation-identity model.

Organizational scientists have historically assessed personality via self-reports, but there is a growing recognition that personality ratings from observers offer superior prediction of job performance compared to targets' self-reports. Yet, the origin of these differences remains unclear: do observers show predictive validity advantages (a) because they have a clearer lens into how targets' thoughts, feelings, and desires translate to their behaviors (traits), (b) because they infer personality from how targets characteristically adapt their behaviors to situations (reputation), or (c) because they omit targets' unexpressed, internal aspects of personality (identity)? With a sample of 422 cadets at a highly selective military educational institute in South Korea, we applied (McAbee & Connelly, 2016) Trait-Reputation-Identity (TRI) Model to decompose consensus and discrepancy in multirater personality data. The variance associated with reputations (the unique personality insights held by observers) dominated the prediction for conscientiousness and agreeableness in predicting all criteria. Trait factors (reflecting the consensus between self- and observer-reports) were moderately related to ratings of job performance, citizenship, and grades for most theoretically aligned personality dimensions. Identity factors (targets' unique personality self-views) were generally unrelated to performance criteria, save for some modest positive relationships for conscientiousness (predicting work and academic performance) and agreeableness (predicting citizenship). These findings suggest that personality is an important determinant of success less by depicting "who employees are" (or "who they think they are") but more by describing "what employees do." We discuss the implications of these results for how multirater assessments can be built into organizational psychology research and practice. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Meta-analytic five-factor model personality intercorrelations: Eeny, meeny, miney, moe, how, which, why, and where to go.

Meta-analysis is frequently combined with multiple regression or path analysis to examine how the Big Five/Five-Factor Model (FFM) personality traits relate to work outcomes. A common approach in such studies is to construct a synthetic correlation matrix by combining new meta-analyses of FFM-criterion correlations with previously published meta-analytic FFM intercorrelations. Many meta-analytic FFM intercorrelation matrices exist in the literature, with 3 matrices being frequently used in industrial-organizational (I-O) psychology and related fields (i.e., Mount, Barrick, Scullen, & Rounds, 2005; Ones, 1993; van der Linden, te Nijenhuis, & Bakker, 2010). However, it is unknown how the choice of FFM matrix influences study conclusions, why we observe such differences in the matrices, and which matrix researchers and practitioners should use for their specific studies. We conducted 3 studies to answer these questions. In Study 1, we demonstrate that researchers' choice of FFM matrix can substantively alter conclusions from meta-analytic regressions or path analyses. In Study 2, we present a new meta-analysis of FFM intercorrelations using measures explicitly constructed around the FFM and based on employee samples. In Study 3, we systematically explore the sources of differences in FFM intercorrelations using second-order meta-analyses of 44 meta-analytic FFM matrices. We find that personality rating source (self vs. other) and inventory-specific substantive and methodological features are the primary moderators of meta-analytic FFM intercorrelations. Based on the findings from these studies, we provide a framework to guide future researchers in choosing a meta-analytic FFM matrix that is most appropriate for their specific studies, research questions, and contexts. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Using beta coefficients to impute missing correlations in meta-analysis research: Reasons for caution.

Meta-analysis has become a well-accepted method for synthesizing empirical research about a given phenomenon. Many meta-analyses focus on synthesizing correlations across primary studies, but some primary studies do not report correlations. Peterson and Brown (2005) suggested that researchers could use standardized regression weights (i.e., beta coefficients) to impute missing correlations. Indeed, their beta estimation procedures (BEPs) have been used in meta-analyses in a wide variety of fields. In this study, the authors evaluated the accuracy of BEPs in meta-analysis. We first examined how use of BEPs might affect results from a published meta-analysis. We then developed a series of Monte Carlo simulations that systematically compared the use of existing correlations (that were not missing) to data sets that incorporated BEPs (that impute missing correlations from corresponding beta coefficients). These simulations estimated ρ̄ (mean population correlation) and SDρ (true standard deviation) across a variety of meta-analytic conditions. Results from both the existing meta-analysis and the Monte Carlo simulations revealed that BEPs were associated with potentially large biases when estimating ρ̄ and even larger biases when estimating SDρ. Using only existing correlations often substantially outperformed use of BEPs and virtually never performed worse than BEPs. Overall, the authors urge a return to the standard practice of using only existing correlations in meta-analysis. (PsycINFO Database Record

Linking job-relevant personality traits, transformational leadership, and job performance via perceived meaningfulness at work: A moderated mediation model.

By integrating the fundamental principles of the theory of purposeful work behavior (TPWB; Barrick, Mount, & Li, 2013) with cognitive-affective personality system (CAPS) theory (Mischel, 1977; Mischel & Shoda, 1995), we examine how and when salespeople's job-relevant personality traits relate to their performance. We argue that individuals with personality traits that fit outdoor sales jobs (i.e., conscientious, extraversion, openness to experience) will perceive their work as more meaningful and as a result achieve heightened performance. Moreover, drawing from TPWB and CAPS theory, we expect that as an important element of the social context, transformational leadership moderates the indirect effect of salespeople's job-relevant personality traits on their job performance via enhanced perceptions of meaningfulness at work. Results based on data from 496 outdoor salespeople and their 218 supervisors and regional managers provide support for the hypotheses pertaining to conscientiousness and openness, but not extraversion. Specifically, the conditional indirect effects of conscientiousness or openness on performance through perceived meaningfulness are more positive under low, rather than high, levels of transformational leadership. Implications for research and practice are discussed along with study limitations and future research directions. (PsycINFO Database Record

Who r u?: On the (in)accuracy of incumbent-based estimates of range restriction in criterion-related and differential validity research.

Correcting validity estimates for selection procedures for range restriction typically involves comparing variance in predictor scores between all job applicants and applicants who were selected. However, some research on criterion-related and differential validity of cognitive ability tests has relied on range restriction corrections based on data from job incumbents. Unfortunately, there remains ambiguity concerning the accuracy of this incumbent-based approach vis-à-vis the applicant-based approach. To address this issue, we conducted several Monte Carlo simulations, as well as an analysis of college admissions data. Our first simulation study showed that incumbent-based range restriction corrections result in downwardly biased estimates of criterion-related validity, whereas applicant-based corrections were quite accurate. Our second set of simulations showed that incumbent-based range restriction corrections can produce evidence of differential validity when there is no differential validity in the population. In contrast, applicant-based corrections tended to accurately estimate population parameters and showed little, if any, evidence of differential validity when there is no differential validity in the population. Analysis of data for the ACT as a predictor of academic performance revealed similar patterns of bias for incumbent-based corrections in an academic setting. Overall, the present findings raise serious concerns regarding the use of incumbent-based range restriction corrections in lieu of applicant-based corrections. They also cast doubt on recent evidence for differential validity of predictors of job performance. (PsycINFO Database Record

Fit and frustration as drivers of targeted counterproductive work behaviors: A multifoci perspective.

In this article, the authors integrate the theory of work adjustment (Dawis, England, & Lofquist, 1964) and the stressor emotion model of counterproductive work behaviors (CWBs; Spector & Fox, 2005) to examine workplace frustration as an intervening mechanism that mediates relations between person-environment (P-E) fit and CWBs. Moreover, we adopt a multifoci perspective to estimate effects for multiple fit, frustration, and CWB foci. We examine the nature of relations between fit, frustration, and CWB for like foci (target similar effects), as well as cross-foci effects. Study 1 examines proposed effects in a sample of 447 employee-coworker dyads. Study 2 uses a 3-wave survey design and tests effects in a sample of 669 employees. Results from both studies suggest that (a) frustration mediates the effects of P-E fit on CWBs and (b) the most consistent effects were observed among the variables with matching foci. Implications for research and practice are discussed. (PsycINFO Database Record

Taking it to another level: do personality-based human capital resources matter to firm performance?

Drawing on the attraction-selection-attrition perspective, strategic human resource management (SHRM) scholarship, and recent human capital research, this study explores organization-level emergence of personality (i.e., personality-based human capital resources) and its direct, interactive, and (conditional) indirect effects on organization-level outcomes based on data from 6,709 managers across 71 firms. Results indicate that organization-level mean emotional stability, extraversion, and conscientiousness are positively related to organization-level managerial job satisfaction and labor productivity but not to financial performance. Furthermore, organization-level mean and variance in emotional stability interact to predict all three organization-level outcomes, and organization-level mean and variance in extraversion interact to predict firm financial performance. Specifically, the positive effects of organization-level mean emotional stability and extraversion are stronger when organization-level variance in these traits is lower. Finally, organization-level mean emotional stability, extraversion, and conscientiousness are all positively related to firm financial performance indirectly via labor productivity, and the indirect effects are more positive when organization-level variance in those personality traits is lower. Overall, the findings suggest that personality-based human capital resources demonstrate tangible effects on organization-level outcomes. Theoretical and practical implications of these findings are discussed along with study limitations and future research directions.

Understanding organizational commitment: A meta-analytic examination of the roles of the five-factor model of personality and culture.

We examined the relationships between the Five-Factor Model (FFM) of personality traits and three forms of organizational commitment (affective, normative, and continuance commitment) and their variability across individualistic and collectivistic cultures. Meta-analytic results based on 55 independent samples from 50 studies (N = 18,262) revealed that (a) all FFM traits had positive relationships with affective commitment; (b) all FFM traits had positive relationships with normative commitment; and (c) Emotional Stability, Extraversion, and Openness to Experience had negative relationships with continuance commitment. In particular, Agreeableness was found to be the trait most strongly related to both affective and normative commitment. The results also showed that Agreeableness had stronger relationships with affective and normative commitment in collectivistic cultures than in individualistic cultures. We provide theoretical and practical implications of these findings for personality, job attitudes, and employee selection and retention.

A meta-analysis of the relationship between general mental ability and nontask performance.

Although one of the most well-established research findings in industrial-organizational psychology is that general mental ability (GMA) is a strong and generalizable predictor of job performance, this meta-analytically derived conclusion is based largely on measures of task or overall performance. The primary purpose of this study is to address a void in the research literature by conducting a meta-analysis to determine the direction and magnitude of the correlation of GMA with 2 dimensions of nontask performance: counterproductive work behaviors (CWB) and organizational citizenship behaviors (OCB). Overall, the results show that the true-score correlation between GMA and CWB is essentially 0 (-.02, k = 35), although rating source of CWB moderates this relationship. The true-score correlation between GMA and OCB is positive but modest in magnitude (.23, k = 43). The 2nd purpose of this study is to conduct meta-analytic relative weight analyses to determine the relative importance of GMA and the five-factor model (FFM) of personality traits in predicting nontask and task performance criteria. Results indicate that, collectively, the FFM traits are substantially more important for CWB than GMA, that the FFM traits are roughly equal in importance to GMA for OCB, and that GMA is substantially more important for task and overall job performance than the FFM traits. Implications of these findings for the development of optimal selection systems and the development of comprehensive theories of job performance are discussed along with study limitation and future research directions. (PsycINFO Database Record (c) 2014 APA, all rights reserved).

Developing leaders' strategic thinking through global work experience: the moderating role of cultural distance.

To respond to the challenge of how organizations can develop leaders who can think strategically, we investigate the relation of leaders' global work experiences--that is, those experiences that require the role incumbent to transcend national boundaries--to their competency in strategic thinking. We further examine whether leaders' exposure to a country whose culture is quite distinct from the culture of their own country (i.e., one that is culturally distant) moderates these relationships. Our analyses of 231 upper level leaders reveals that the time they have spent in global work experiences positively relates to their strategic thinking competency, particularly for leaders who have had exposure to a more culturally distant country. We discuss these findings in light of the research on international work experiences and leader development.

Differential validity for cognitive ability tests in employment and educational settings: not much more than range restriction?

The concept of differential validity suggests that cognitive ability tests are associated with varying levels of validity across ethnic groups, such that validity is lower in certain ethnic subgroups than in others. A recent meta-analysis has revived the viability of this concept. Unfortunately, data were not available in this meta-analysis to correct for range restriction within ethnic groups. We reviewed the differential validity literature and conducted 4 studies. In Study 1, we empirically demonstrated that using a cognitive ability test with a common cutoff decreases variance in test scores of Black subgroup samples more than in White samples. In Study 2, we developed a simulation that examined the effects of range restriction on estimates of differential validity. Results demonstrated that different levels of range restriction for subgroups can explain the apparent observed differential validity results in employment and educational settings (but not military settings) when no differential validity exists in the population. In Study 3, we conducted a simulation in which we examined how one corrects for range restriction affects the accuracy of these corrections. Results suggest that the correction approach using a common range restriction ratio for various subgroups may create or perpetuate the illusion of differential validity and that corrections are most accurate when done within each subgroup. Finally, in Study 4, we conducted a simulation in which we assumed differential validity in the population. We found that range restriction artificially increased the size of observed differential validity estimates when the validity of cognitive ability tests was assumed to be higher among Whites. Overall, we suggest that the concept of differential validity may be largely artifactual and current data are not definitive enough to suggest such effects exist.

The five-factor model of personality traits and organizational citizenship behaviors: a meta-analysis.

Using meta-analytic tests based on 87 statistically independent samples, we investigated the relationships between the five-factor model (FFM) of personality traits and organizational citizenship behaviors in both the aggregate and specific forms, including individual-directed, organization-directed, and change-oriented citizenship. We found that Emotional Stability, Extraversion, and Openness/Intellect have incremental validity for citizenship over and above Conscientiousness and Agreeableness, 2 well-established FFM predictors of citizenship. In addition, FFM personality traits predict citizenship over and above job satisfaction. Finally, we compared the effect sizes obtained in the current meta-analysis with the comparable effect sizes predicting task performance from previous meta-analyses. As a result, we found that Conscientiousness, Emotional Stability, and Extraversion have similar magnitudes of relationships with citizenship and task performance, whereas Openness and Agreeableness have stronger relationships with citizenship than with task performance. This lends some support to the idea that personality traits are (slightly) more important determinants of citizenship than of task performance. We conclude with proposed directions for future research on the relationships between FFM personality traits and specific forms of citizenship, based on the current findings.

Too much of a good thing: curvilinear relationships between personality traits and job performance.

The relationships between personality traits and performance are often assumed to be linear. This assumption has been challenged conceptually and empirically, but results to date have been inconclusive. In the current study, we took a theory-driven approach in systematically addressing this issue. Results based on two different samples generally supported our expectations of the curvilinear relationships between personality traits, including Conscientiousness and Emotional Stability, and job performance dimensions, including task performance, organizational citizenship behavior, and counterproductive work behaviors. We also hypothesized and found that job complexity moderated the curvilinear personality­performance relationships such that the inflection points after which the relationships disappear were lower for low-complexity jobs than they were for high-complexity jobs. This finding suggests that high levels of the two personality traits examined are more beneficial for performance in high- than low-complexity jobs. We conclude by discussing the implications of these findings for the use of personality in personnel selection.

Validity of observer ratings of the five-factor model of personality traits: a meta-analysis.

Conclusions reached in previous research about the magnitude and nature of personality-performance linkages have been based almost exclusively on self-report measures of personality. The purpose of this study is to address this void in the literature by conducting a meta-analysis of the relationship between observer ratings of the five-factor model (FFM) personality traits and overall job performance. Our results show that the operational validities of FFM traits based on observer ratings are higher than those based on self-report ratings. In addition, the results show that when based on observer ratings, all FFM traits are significant predictors of overall performance. Further, observer ratings of FFM traits show meaningful incremental validity over self-reports of corresponding FFM traits in predicting overall performance, but the reverse is not true. We conclude that the validity of FFM traits in predicting overall performance is higher than previously believed, and our results underscore the importance of disentangling the validity of personality traits from the method of measurement of the traits.

The five-factor model of personality and managerial performance: validity gains through the use of 360 degree performance ratings.

This study investigated the usefulness of the five-factor model (FFM) of personality in predicting two aspects of managerial performance (task vs. contextual) assessed by utilizing the 360 degree performance rating system. The authors speculated that one reason for the low validity of the FFM might be the failure of single-source (e.g., supervisor) ratings to comprehensively capture the construct of managerial performance. The operational validity of personality was found to increase substantially (50%-74%) across all of the FFM personality traits when both peer and subordinate ratings were added to supervisor ratings according to the multitrait-multimethod approach. Furthermore, the authors responded to the recent calls to validate tests via a multivariate (e.g., multitrait-multimethod) approach by decomposing overall managerial performance into task and contextual performance criteria and by using multiple rating perspectives (sources). Overall, this study contributes to the evidence that personality may be even more useful in predicting managerial performance if the performance criteria are less deficient.

Intervention effects on college performance and retention as mediated by motivational, emotional, and social control factors: integrated meta-analytic path analyses.

Using both organizational and educational perspectives, the authors proposed and tested theoretical models on the mediating roles that psychosocial factors (PSFs; motivational, emotional, and social control factors) play between college interventions (academic skill, self-management, socialization, and First-Year-Experience interventions) and college outcomes (academic performance and retention). They first determined through meta-analysis of 404 data points the effects of college interventions on college outcomes and on PSFs. These meta-analytic findings were then combined with results from S. B. Robbins et al.'s (2004) meta-analysis to test the proposed models. Integrated meta-analytic path analyses showed the direct and indirect effects (via PSFs) of intervention strategies on both performance and retention outcomes. The authors highlight the importance of both academic skill and self-management-based interventions; they also note the salience of motivational and emotional control mediators across both performance and retention outcomes. Implications from organizational and educational perspectives, limitations, and future directions are addressed.

Fixed- versus random-effects models in meta-analysis: model properties and an empirical comparison of differences in results.

Today most conclusions about cumulative knowledge in psychology are based on meta-analysis. We first present an examination of the important statistical differences between fixed-effects (FE) and random-effects (RE) models in meta-analysis and between two different RE procedures, due to Hedges and Vevea, and to Hunter and Schmidt. The implications of these differences for the appropriate interpretation of published meta-analyses are explored by applying the two RE procedures to 68 meta-analyses from five large meta-analytic studies previously published in Psychological Bulletin. Under the assumption that the goal of research is generalizable knowledge, results indicated that the published FE confidence intervals (CIs) around mean effect sizes were on average 52% narrower than their actual width, with similar results being produced by the two RE procedures. These nominal 95% FE CIs were found to be on average 56% CIs. Because most meta-analyses in the literature use FE models, these findings suggest that the precision of meta-analysis findings in the literature has often been substantially overstated, with important consequences for research and practice.