Auditing the AI auditors: A framework for evaluating fairness and bias in high stakes AI predictive models.

Researchers, governments, ethics watchdogs, and the public are increasingly voicing concerns about unfairness and bias in artificial intelligence (AI)-based decision tools. Psychology's more-than-a-century of research on the measurement of psychological traits and the prediction of human behavior can benefit such conversations, yet psychological researchers often find themselves excluded due to mismatches in terminology, values, and goals across disciplines. In the present paper, we begin to build a shared interdisciplinary understanding of AI fairness and bias by first presenting three major lenses, which vary in focus and prototypicality by discipline, from which to consider relevant issues: (a) individual attitudes, (b) legality, ethicality, and morality, and (c) embedded meanings within technical domains. Using these lenses, we next present psychological audits as a standardized approach for evaluating the fairness and bias of AI systems that make predictions about humans across disciplinary perspectives. We present 12 crucial components to audits across three categories: (a) components related to AI models in terms of their source data, design, development, features, processes, and outputs, (b) components related to how information about models and their applications are presented, discussed, and understood from the perspectives of those employing the algorithm, those affected by decisions made using its predictions, and third-party observers, and (c) meta-components that must be considered across all other auditing components, including cultural context, respect for persons, and the integrity of individual research designs used to support all model developer claims. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Social learning dynamics influence performance and career self-efficacy in career-oriented educational virtual environments.

Educational virtual environments (EVEs) are defined by their features of immersion (degree of sensory engagement) and fidelity (degree of realism). Increasingly, EVEs are being used for career development and training purposes, which we refer to as career-oriented EVEs. However, little research has examined the effects of immersion and fidelity on career-related outcomes, like self-efficacy and interests, and the learning dynamics that may influence these outcomes. We address these research needs across two studies using an inductive approach. Study 1 compares welding career exploration in EVEs to traditional career exploration and finds that individuals using EVEs report more positive career self-efficacy. Study 2 examines the influence of social learning dynamics, or how individuals learn from each other through behavioral modeling, on performance and career-related self-efficacy and interest. Groups were assigned to use either a high or low immersion and fidelity EVE. Findings indicate strong social learning dynamics in both EVEs, but the effects were stronger for groups using the higher immersion and fidelity EVE. Specifically, groups converged on two performance measures, and the performance of individuals who were situated as behavioral models significantly predicted the performance of other group members. Performance at the individual level, in turn, predicted career self-efficacy and interest for men but not women, and only for those using the higher immersion and fidelity EVE. Based on these findings, we conclude with practical recommendations for and implications of implementing career-oriented EVEs for career exploration and skills training.

Stereotypical descriptions of computer science career interests are not representative of many computer scientists.

Using responses from a large respondent-initiated online survey, we find that the career interests of many current and aspiring computer scientists in the United States diverge from a popular and official depiction of computer scientists' interests used for career and workforce development worldwide. Distinct profiles of career interests emerged from the data. These profiles suggest that many women in the field value social and artistic expression in a way not currently recognized by established depictions of computer scientists' interests. Better capturing the diversity of interests in computer science might help to boost women's, and men's, engagement in this STEM field.

Pay for performance, satisfaction and retention in longitudinal crowdsourced research.

In the social and cognitive sciences, crowdsourcing provides up to half of all research participants. Despite this popularity, researchers typically do not conceptualize participants accurately, as gig-economy worker-participants. Applying theories of employee motivation and the psychological contract between employees and employers, we hypothesized that pay and pay raises would drive worker-participant satisfaction, performance, and retention in a longitudinal study. In an experiment hiring 359 Amazon Mechanical Turk Workers, we found that initial pay, relative increase of pay over time, and overall pay did not have substantial influence on subsequent performance. However, pay significantly predicted participants' perceived choice, justice perceptions, and attrition. Given this, we conclude that worker-participants are particularly vulnerable to exploitation, having relatively low power to negotiate pay. Results of this study suggest that researchers wishing to crowdsource research participants using MTurk might not face practical dangers such as decreased performance as a result of lower pay, but they must recognize an ethical obligation to treat Workers fairly.

Moderating effects of person and job characteristics on digital monitoring outcomes.

Many individuals perceive digital monitoring to be an inherently negative practice that invades privacy, but recent research suggests that it has positive effects for workers under certain circumstances. This review expands upon existing digital monitoring frameworks by adopting a psychological perspective to explain individual and contextual variation in monitoring reactions. To do so, we identify person characteristics (e.g. trait reactance, self-efficacy, ethical orientation, goal orientation) and job characteristics (e.g. manual versus nonmanual labor, autonomy, task significance) that moderate workers' reactions and performance outcomes while being digitally monitored. Future research on moderators such as these will remain important as organizations continue to collect big data using digital monitoring.

When big brother is watching: goal orientation shapes reactions to electronic monitoring during online training.

Web-based training is frequently used by organizations as a convenient and low-cost way to teach employees new knowledge and skills. As web-based training is typically unproctored, employees may be held accountable to the organization by computer software that monitors their behaviors. The current study examines how the introduction of electronic performance monitoring may provoke negative emotional reactions and decrease learning among certain types of e-learners. Through motivated action theory and trait activation theory, we examine the role of performance goal orientation when e-learners are exposed to asynchronous and synchronous monitoring. We show that some e-learners are more susceptible than others to evaluation apprehension when they perceive their activities are being monitored electronically. Specifically, e-learners higher in avoid performance goal orientation exhibited increased evaluation apprehension if they believed asynchronous monitoring was present, and they showed decreased skill attainment as a result. E-learners higher on prove performance goal orientation showed greater evaluation apprehension if they believed real-time monitoring was occurring, resulting in decreased skill attainment.

The viability of crowdsourcing for survey research.

Online contract labor portals (i.e., crowdsourcing) have recently emerged as attractive alternatives to university participant pools for the purposes of collecting survey data for behavioral research. However, prior research has not provided a thorough examination of crowdsourced data for organizational psychology research. We found that, as compared with a traditional university participant pool, crowdsourcing respondents were older, were more ethnically diverse, and had more work experience. Additionally, the reliability of the data from the crowdsourcing sample was as good as or better than the corresponding university sample. Moreover, measurement invariance generally held across these groups. We conclude that the use of these labor portals is an efficient and appropriate alternative to a university participant pool, despite small differences in personality and socially desirable responding across the samples. The risks and advantages of crowdsourcing are outlined, and an overview of practical and ethical guidelines is provided.