Do Tele-Operators Learn to Better Judge Whether a Robot Can Pass Through an Aperture?

OBJECTIVE: This experiment examined whether tele-operators learn to better judge a robot's ability to pass through an aperture, hereafter referred to as pass-ability judgments, and detailed the nature of such learning. BACKGROUND: Jones, Johnson, and Schmidlin reported that tele-operators' pass-ability judgments did not improve over the course of their experiment, which was surprising. METHOD: In each of seven blocks, tele-operators made pass-ability judgments about 10 apertures whose width varied. During each trial, participants drove the robot toward the aperture, answered yes or no to whether it could pass through that aperture, and then attempted to drive the robot through the aperture. Pass-ability judgments were analyzed in terms of percentage correct and absolute thresholds; the latter mimicked how Jones et al. analyzed their data. RESULTS: Learning was revealed when judgments were analyzed in terms of percentage correct and not when analyzed in terms of absolute thresholds. Further analyses revealed that tele-operators only improved their pass-ability judgments for impassable apertures, and tele-operators' perceptual sensitivity and response bias changed over the course of the experiment. CONCLUSION: The percentage correct-based analyses revealed that tele-operators learned to make better pass-ability judgments. Jones et al.'s decision to analyze their data in terms of absolute thresholds obscured learning. APPLICATION: The present results suggested that researchers should employ percentage correct when studying learning in this domain, training protocols should focus on improving tele-operators' abilities to judge the pass-ability of impassable apertures, and tele-operators truly learned to better discriminate passable and impassable apertures.

An investigation of the prevalence of replication research in human factors.

OBJECTIVE: The present studies investigated the nature of replication research within the human factors literature. BACKGROUND: Many claim that researchers in certain fields do not replicate prior research. This is troubling because replications allow science to self-correct. A successful replication corroborates the original finding, whereas an unsuccessful replication falsifies it. To date, no one has assessed whether this issue affects the field of human factors. METHOD: In the first study, eight articles (parent articles) were selected from the 1991 issues of the journal Human Factors. Each article that had referenced one of the eight parent articles between 1991 and September 2006 (child articles) were also retrieved. Two investigators coded and compared each child article against its 1991 parent article to determine whether the child article replicated its parent article. The second study replicated these procedures. RESULTS: Half or more of the parent articles in Study I and Study 2 (75% and 50%, respectively) were replicated at least once. Furthermore, human factors researchers conducted replications of their own work as well as the work of others. However, many researchers did not state that they replicated previous research. CONCLUSION: Replications seem to be common in the human factors literature. However, readers may not realize that a study replicated prior research. Thus, they may incorrectly assess the evidence concerning a given finding. APPLICATION: Human factors professionals should be taught how to identify replications and to be cautious of research that has not been replicated.