An examination of the comparative reliability, validity, and accuracy of performance ratings made using computerized adaptive rating scales.

This laboratory research compared the reliability, validity, and accuracy of a computerized adaptive rating scale (CARS) format and 2 relatively common and representative rating formats. The CARS is a paired-comparison rating task that uses adaptive testing principles to present pairs of scaled behavioral statements to the rater to iteratively estimate a ratee's effectiveness on 3 dimensions of contextual performance. Videotaped vignettes of 6 office workers were prepared, depicting prescripted levels of contextual performance, and 112 subjects rated these vignettes using the CARS format and one or the other competing format. Results showed 23%-37% lower standard errors of measurement for the CARS format. In addition, validity was significantly higher for the CARS format (d = .18), and Cronbach's accuracy coefficients showed significantly higher accuracy, with a median effect size of .08. The discussion focuses on possible reasons for the results.

Selecting pilots with crew resource management skills.

For years, pilot selection has focused primarily on the identification of individuals with superior flying skills and abilities. More recently, the aviation community has become increasingly aware that successful completion of a flight or mission requires not only flying skills but the ability to work well in a crew situation. This project involved development and validation of a crew resource management (CRM) skills test for Air Force transport pilots. A significant relation was found between the CRM skills test and behavior-based ratings of aircraft commander CRM performance, and the implications of these findings for CRM-based selection and training are discussed.

Personnel selection.

This chapter reviews literature from approximately mid-1993 through early 1996 in the areas of performance and criteria, validity, statistical and equal opportunity issues, selection for work groups, person-organization fit, applicant reactions to selection procedures, and research on predictors, including ability, personality, assessment centers, interviews, and biodata. The review revolves around three themes: (a) attention toward criteria and models of performance, (b) interest in personality measures as predictors of job performance, and (c) work on the person-organization fit selection model. In our judgment, these themes merge when it is recognized that development of performance models that differentiate criterion constructs reveal highly interpretable relationships between the predictor domain (i.e. ability, personality, and job knowledge) and the criterion domain (i.e. technical proficiency, extra-technical proficiency constructs such as prosocial organizational behavior, and overall job performance). These and related developments are advancing the science of personnel selection and should enhance selection practices in the future.