Using artificial intelligence to assess personal qualities in college admissions.

Personal qualities like prosocial purpose and leadership predict important life outcomes, including college success. Unfortunately, the holistic assessment of personal qualities in college admissions is opaque and resource intensive. Can artificial intelligence (AI) advance the goals of holistic admissions? While cost-effective, AI has been criticized as a "black box" that may inadvertently penalize already disadvantaged subgroups when used in high-stakes settings. Here, we consider an AI approach to assessing personal qualities that aims to overcome these limitations. Research assistants and admissions officers first identified the presence/absence of seven personal qualities in n = 3131 applicant essays describing extracurricular and work experiences. Next, we fine-tuned pretrained language models with these ratings, which successfully reproduced human codes across demographic subgroups. Last, in a national sample (N = 309,594), computer-generated scores collectively demonstrated incremental validity for predicting 6-year college graduation. We discuss challenges and opportunities of AI for assessing personal qualities.

Automated video interview personality assessments: Reliability, validity, and generalizability investigations.

Organizations are increasingly adopting automated video interviews (AVIs) to screen job applicants despite a paucity of research on their reliability, validity, and generalizability. In this study, we address this gap by developing AVIs that use verbal, paraverbal, and nonverbal behaviors extracted from video interviews to assess Big Five personality traits. We developed and validated machine learning models within (using nested cross-validation) and across three separate samples of mock video interviews (total N = 1,073). Also, we examined their test-retest reliability in a fourth sample (N = 99). In general, we found that the AVI personality assessments exhibited stronger evidence of validity when they were trained on interviewer-reports rather than self-reports. When cross-validated in the other samples, AVI personality assessments trained on interviewer-reports had mixed evidence of reliability, exhibited consistent convergent and discriminant relations, used predictors that appear to be conceptually relevant to the focal traits, and predicted academic outcomes. On the other hand, there was little evidence of reliability or validity for the AVIs trained on self-reports. We discuss the implications for future work on AVIs and personality theory, and provide practical recommendations for the vendors marketing such approaches and organizations considering adopting them. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

A Social Media Study on Demographic Differences in Perceived Job Satisfaction.

Effective ways to measure employee job satisfaction are fraught with problems of scale, misrepresentation, and timeliness. Current methodologies are limited in capturing subjective differences in expectations, needs, and values at work, and they do not lay emphasis on demographic differences, which may impact people's perceptions of job satisfaction. This study proposes an approach to assess job satisfaction by leveraging large-scale social media data. Starting with an initial Twitter dataset of 1.5M posts, we examine two facets of job satisfaction, pay and supervision. By adopting a theory-driven approach, we first build machine learning classifiers to assess perceived job satisfaction with an average AUC of 0.84. We then study demographic differences in perceived job satisfaction by geography, sex, and race in the U.S. For geography, we find that job satisfaction on Twitter exhibits insightful relationships with macroeconomic indicators such as financial wellbeing and unemployment rates. For sex and race, we find that females express greater pay satisfaction but lower supervision satisfaction than males, whereas Whites express the least pay and supervision satisfaction. Unpacking linguistic differences, we find contrasts in different groups' underlying priorities and concerns, e.g., under-represented groups saliently express about basic livelihood, whereas the majority groups saliently express about self-actualization. We discuss the role of frame of reference and the "job satisfaction paradox", conceptualized by organizational psychologists, in explaining our observed differences. We conclude with theoretical and sociotechnical implications of our work for understanding and improving worker wellbeing.