On meta-analyses of imaging data and the mixture of records.

Neumann et al. (2010) aim to find directed graphical representations of the independence and dependence relations among activities in brain regions by applying a search procedure to merged fMRI activity records from a large number of contrasts obtained under a variety of conditions. To that end, Neumann et al., obtain three graphical models, justifying their search procedure with simulations that find that merging the data sampled from probability distributions characterized by two distinct Bayes net graphs results in a graphical object that combines the edges in the individual graphs. We argue that the graphical objects they obtain cannot be interpreted as representations of conditional independence and dependence relations among localized neural activities; specifically, directed edges and directed pathways in their graphical results may be artifacts of the manner in which separate studies are combined in the meta-analytic procedure. With a larger simulation study, we argue that their simulation results with combined data sets are an artifact of their choice of examples. We provide sufficient conditions and necessary conditions for the merger of two or more probability distributions, each characterized by the Markov equivalence class of a directed acyclic graph, to be describable by a Markov equivalence class whose edges are a union of those for the individual distributions. Contrary to Neumann et al., we argue that the scientific value of searches for network representations from imaging data lies in attempting to characterize large scaled neural mechanisms, and we suggest several alternative strategies for combining data from multiple experiments.

The TETRAD Project: Constraint Based Aids to Causal Model Specification.

The statistical community has brought logical rigor and mathematical precision to the problem of using data to make inferences about a model's parameter values. The TETRAD project, and related work in computer science and statistics, aims to apply those standards to the problem of using data and background knowledge to make inferences about a model's specification. We begin by drawing the analogy between parameter estimation and model specification search. We then describe how the specification of a structural equation model entails familiar constraints on the covariance matrix for all admissible values of its parameters; we survey results on the equivalence of structural equation models, and we discuss search strategies for model specification. We end by presenting several algorithms that are implemented in the TETRAD I1 program.

Reply to Comments.

We will respond to our commentators individually, but the order of our responses follows naturally from the issues they bring up. Judea Pearl describes SEM's unfortunate retreat from the clear causal semantics articulated by Sewall Wright (1921) and later by Haavelmo (1943) to the algebraic interpretation preferred more recently by econometricians. We agree with Pearl about the history and also the problem, namely that the algebraic interpretation is suitable for estimation but expressively too weak to even distinguish among competing causal claims. Here we try to elaborate on the distinction between the semantics of a causal SEM and the epistemological connections between statistical data, background knowledge, and causal structure. We argue that many modern critics of SEM make their hay by conflating this distinction. Having tried to make it clear, we then turn to the assumptions that give the epistemological issues their structure, namely the Causal Independence and Faithfulness assumptions. Jim Woodward questions these assumptions at length, especially the Causal Independence assumption, and we spend the second part of our response defending it. Phil Wood seems to accept the fundamental assumptions upon which TETRAD rests, and even the utility of tools like it, but he brings out a wide array of subtle difficulties that we have not had time to discuss, some of which we now cover. Kwok-fai Ting questions the utility of any specification search done by computer, and we attempt to address his concerns last.

An evaluation of machine-learning methods for predicting pneumonia mortality.

This paper describes the application of eight statistical and machine-learning methods to derive computer models for predicting mortality of hospital patients with pneumonia from their findings at initial presentation. The eight models were each constructed based on 9847 patient cases and they were each evaluated on 4352 additional cases. The primary evaluation metric was the error in predicted survival as a function of the fraction of patients predicted to survive. This metric is useful in assessing a model's potential to assist a clinician in deciding whether to treat a given patient in the hospital or at home. We examined the error rates of the models when predicting that a given fraction of patients will survive. We examined survival fractions between 0.1 and 0.6. Over this range, each model's predictive error rate was within 1% of the error rate of every other model. When predicting that approximately 30% of the patients will survive, all the models have an error rate of less than 1.5%. The models are distinguished more by the number of variables and parameters that they contain than by their error rates; these differences suggest which models may be the most amenable to future implementation as paper-based guidelines.