ABSTRACT
Simulations are increasingly popular in employee selection and training. While face valid and engaging, the attributes being assessed are often poorly understood. This study evaluated the extent to which a multitasking assessment based on concurrent memorization, math, visual monitoring, and listening tasks predicted simulated unmanned aircraft vehicle (UAV) mission performance in a military trainee sample (N = 368). Performance was based on accuracy of mission planning, information recall during "Lost Link" conditions, and success in rescuing stranded allies while monitoring the aircraft's resources. Although scores on the multitasking assessment were only weakly related to performance of pre-flight mission planning tasks completed under static conditions, multitasking was strongly related to overall simulated UAV mission performance, including execution of tasks requiring attending to multiple, dynamic sources of information and shifting attention among concurrent processes and demands. Further, multi-tasking demonstrated substantial incremental validity beyond the traditional measures of cognitive ability that have been used for decades within the US military. Implications, limitations, and recommendations for selection and classification and future research are discussed.
ABSTRACT
The predictive validity of the Tailored Adaptive Personality Assessment System (TAPAS), the U.S. Army's first computer-adaptive personality test incorporating multidimensional pairwise preference items, has been demonstrated for training performance in both the Army and Air Force. While the unique TAPAS format has been described as more resistant to applicant faking than traditional self-report personality measures, evidence regarding the magnitude of applicant score distortion on TAPAS, and how such distortion (if present) may affect reliability and validity, has been limited. To address this gap, the present study compared operational TAPAS scores of Air Force enlisted recruits (administered pre-accession to applicants) to their post-accession retest scores under honest and directed faking ("fake good") conditions (based on re-administration of TAPAS during Basic Military Training). Data are presented on the relationship of applicant pre-accession scores to their retest scores under honest conditions (a form of test-retest reliability) and the magnitude of mean score differences in applicant, honest, and directed faking conditions is documented. Further, the validity of the TAPAS as an indicator for counterproductive work behaviors (CWB) was evaluated. Results indicate that TAPAS scores are relatively stable over time and the TAPAS methodology appears to reduce score distortion. In addition, the results suggest that the validities of the TAPAS scores as CWB correlates are comparable across honest and directed faking testing conditions and generally in line with those found for traditional Likert-type self-report Big Five measures.
