Decision criteria do not shift: commentary on Mueller and Weidemann (2008).

The effects of base rates and payoffs on the shapes of rating receiver operating characteristic curves are inconsistent with the basic assumptions of signal detection theory (SDT), in particular the notion of a shifting decision criterion. Mueller and Weidemann (2008) propose that these unexpected phenomena are not due to problems with the decision-criterion construct but are instead due to two compounded effects: instability of the decision criterion across trials, and even greater instability in the flanking criteria that determine which confidence rating will be reported. There are several problems with the authors' decision-noise hypothesis. First, even if their hypothesis about decision noise were taken for granted, the key feature of the ratings data that rejects the SDT model would remain a mystery. Second, the same violations of SDT that are exhibited in the ratings paradigm are also exhibited in the yes-no detection task when response time is substituted for confidence as a basis for analysis. Finally, the decision-noise hypothesis predicts that sensitivity will increase when one source of this variation-the response on a previous trial-is controlled for. This prediction was consistently violated in both the yes-no and ratings conditions of Mueller and Weidemann's experiment. In an Addendum, we respond to Weidemann and Mueller's (2008) reply to this Comment.

Is the area measure a historical anomaly?

Green's well-known area theorem establishes an equivalence between the area under the yes-no ROC curve and the percent correct of an unbiased observer in a two-alternative forced-choice (2AFC) task with equivalent stimuli. In this article, we show that this conversion from yes-no detection data to hypothetical performance in a 2AFC task is unnecessary: The same yes-no detection data that are used to compute the area statistic can always be used to compute the percent correct of an unbiased observer in the yes-no detection task itself. We also show that the ROC curve may not be the ideal graphical device for many investigators. A more natural representation of the difficulty of a discrimination task is obtained by plotting the distribution of the posterior betting odds under equal base rates, which can be estimated from their distributions under unequal base rates. Finally, unlike the area measure and other traditional detection theory statistics, both the yes-no percent correct measure and the odds distributions generalize in an obvious and direct way to classification paradigms with more than two responses (e.g., identification).

Intelligibility of speech in a virtual 3-D environment.

In a simulated air traffic control task, improvement in the detection of auditory warnings when using virtual 3-D audio depended on the spatial configuration of the sounds. Performance improved substantially when two of four sources were placed to the left and the remaining two were placed to the right of the participant. Surprisingly, little or no benefits were observed for configurations involving the elevation or transverse (front/back) dimensions of virtual space, suggesting that position on the interaural (left/right) axis is the crucial factor to consider in auditory display design. The relative importance of interaural spacing effects was corroborated in a second, free-field (real space) experiment. Two additional experiments showed that (a) positioning signals to the side of the listener is superior to placing them in front even when two sounds are presented in the same location, and (b) the optimal distance on the interaural axis varies with the amplitude of the sounds. These results are well predicted by the behavior of an ideal observer under the different display conditions. This suggests that guidelines for auditory display design that allow for effective perception of speech information can be developed from an analysis of the physical sound patterns.

Decision processes in discrimination: fundamental misrepresentations of signal detection theory.

A new approach to studying decision making in discrimination tasks is described that does not depend on the technical assumptions of signal detection theory (e.g., normality of the encoding distributions). In 3 different experiments, results of these new distribution-free tests converge on a single, surprising conclusion: response biases had substantial effects on the encoding distributions but no effect on the decision rule, which was uniformly unbiased in equal and unequal base rate conditions and in symmetric and asymmetric payoff conditions. This seemingly paradoxical result is fundamentally inconsistent with the entire family of signal detection theory models, raising some important questions about the significance of many published results in the human performance literature.

Measures and interpretations of vigilance performance: evidence against the detection criterion.

Operators' performance in a vigilance task is often assumed to depend on their choice of a detection criterion. When the signal rate is low this criterion is set high, causing the hit and false alarm rates to be low. With increasing time on task the criterion presumably tends to increase even further, thereby further decreasing the hit and false alarm rates. Virtually all of the empirical evidence for this simple interpretation is based on estimates of the bias measure beta from signal detection theory. In this article, I describe a new approach to studying decision making that does not require the technical assumptions of signal detection theory. The results of this new analysis suggest that the detection criterion is never biased toward either response, even when the signal rate is low and the time on task is long. Two modifications of the signal detection theory framework are considered to account for this seemingly paradoxical result. The first assumes that the signal rate affects the relative sizes of the variances of the information distributions; the second assumes that the signal rate affects the logic of the operator's stopping rule. Actual or potential applications of this research include the improved training and performance assessment of operators in areas such as product quality control, air traffic control, and medical and clinical diagnosis.

Form and objective of the decision rule in absolute identification.

In several conditions of a line length identification experiment, the subjects' decision making strategies were systematically biased against the responses on the edges of the stimulus range. When the range and number of the stimuli were small, the bias caused the percentage of correct responses to be highest in the center and lowest on the extremes of the range. Two general classes of decision rules that would explain these results are considered. The first class assumes that subjects intend to adopt an optimal decision rule, but systematically misrepresent one or more parameters of the decision making context. The second class assumes that subjects use a different measure of performance than the one assumed by the experimenter: instead of maximizing the chances of a correct response, the subject attempts to minimize the expected size of the response error (a "fidelity criterion"). In a second experiment, extended experience and feedback did not diminish the bias effect, but explicitly penalizing all response errors equally, regardless of their size, did reduce or eliminate it in some subjects. Both results favor the fidelity criterion over the optimal rule.

Testing models of decision making using confidence ratings in classification.

Classification implies decision making (or response selection) of some kind. Studying the decision process using a traditional signal detection theory analysis is difficult for two reasons: (a) The model makes a strong assumption about the encoding process (normal noise), and (b) the two most popular decision models, optimal and distance-from-criterion models, can mimic each other's predictions about performance level. In this article, the authors show that by analyzing certain distributional properties of confidence ratings, a researcher can determine whether the decision process is optimal, without knowing the form of the encoding distributions. Empirical results are reported for three types of experiments: recognition memory, perceptual discrimination, and perceptual categorization. In each case, the data strongly favored the distance-from-criterion model over the optimal model.

Subitizing: magical numbers or mere superstition?

It is widely believed that humans are endowed with a specialized numerical process, called subitizing, which enables them to apprehend rapidly and accurately the numerosity of small sets of objects. A major part of the evidence for this process is a purported discontinuity in the mean response time (RT) versus numerosity curves at about 4 elements, when subjects enumerate up to 7 or more elements in a visual display. In this article, RT data collected in a speeded enumeration experiment are subjected to a variety of statistical analyses, including several tests on the RT distributions. None of these tests reveals a significant discontinuity as numerosity increases. The data do suggest a strong stochastic dominance in RT by display numerosity, indicating that the mental effort required to enumerate does increase with each additional element in the display, both within and beyond the putative subitizing range.

Is subitizing a unique numerical ability?

Two models that predict the relation between mean enumeration time and numerosity in a speeded enumeration experiment are tested. The first is a bilinear two-process model, and the second is a log-linear single-process model. Previously, support for the bilinear model has provided evidence for the existence of a unique numerical ability called "subitizing." Both models are shown to yield close approximations to the empirical data, but at the same time to consistently violate the robust shape of these data. Two fundamental discrepancies exist: (1) Enumeration of single-element displays is unpredictably fast, both in the data reported here and elsewhere, and (2) the response-time function for multiple elements is continuously convex upward, with a significant log-quadratic component. The findings support the contention that enumeration is a capacity-limited process, but not statistically reliable change in processing character, that is, from subitizing to some other process, is evident in enumeration of displays of up to six elements.

Length distortion of temporally extended visual displays: similarity to haptic spatial perception.

Three experiments were designed to investigate length distortion--the tendency to inflate estimates of the inferred distance between two points, as the length of a circuitous pathway between them increases. This phenomenon, previously demonstrated with haptic and locomotor exploration (Lederman, Klatzky, Collins, & Wardell, 1987), was extended to vision through the presentation of pathways as a temporal sequence of illuminated points. The magnitude of the visual effect was less than that previously found in haptics, and conditions promoting the effect included a relatively slow presentation rate and moderately complex pathways. Commonalities in the pattern, if not the magnitude, of length distortion in vision and haptics suggest that similar processes of spatial encoding may underlie the phenomenon in both domains.