ABSTRACT
Across a broad set of applications, system outcomes may be summarized as probabilities in confusion or contingency tables. In settings with more than two outcomes (e.g., stages of cancer), these outcomes represent multinomial experiments. Measures to summarize system performance have been presented as linear combinations of the resulting multinomial probabilities. Statistical inference on the linear combination of multinomial probabilities has been focused on large-sample and parametric settings and not small-sample settings. Such inference is valuable, however, especially in settings such as those resulting from pilot or low-cost studies. To address this gap, we leverage the fiducial approach to derive confidence intervals around the linear combination of multinomial parameters with desirable frequentist properties. One of the original arguments against the fiducial approach was its inability to extend to multiparameter settings. Therefore, the great novelty of this work is both the derived interval and the logical framework for applying the fiducial approach in multiparameter settings. Through simulation, we demonstrate that the proposed method maintains a minimum coverage of 1 - α $1 - \alpha$ , unlike the bootstrap and large-sample methods, at comparable interval lengths. Finally, we illustrate its use in a medical problem of selecting classifiers for diagnosing chronic allograph nephropathy in postkidney transplant patients.
