Analytic reproducibility in articles receiving open data badges at the journal Psychological Science: an observational study.

For any scientific report, repeating the original analyses upon the original data should yield the original outcomes. We evaluated analytic reproducibility in 25 Psychological Science articles awarded open data badges between 2014 and 2015. Initially, 16 (64%, 95% confidence interval [43,81]) articles contained at least one 'major numerical discrepancy' (>10% difference) prompting us to request input from original authors. Ultimately, target values were reproducible without author involvement for 9 (36% [20,59]) articles; reproducible with author involvement for 6 (24% [8,47]) articles; not fully reproducible with no substantive author response for 3 (12% [0,35]) articles; and not fully reproducible despite author involvement for 7 (28% [12,51]) articles. Overall, 37 major numerical discrepancies remained out of 789 checked values (5% [3,6]), but original conclusions did not appear affected. Non-reproducibility was primarily caused by unclear reporting of analytic procedures. These results highlight that open data alone is not sufficient to ensure analytic reproducibility.

The Interactions of Rational, Pragmatic Agents Lead to Efficient Language Structure and Use.

Despite their diversity, languages around the world share a consistent set of properties and distributional regularities. For example, the distribution of word frequencies, the distribution of syntactic dependency lengths, and the presence of ambiguity are all remarkably consistent across languages. We discuss a framework for studying how these system-level properties emerge from local, in-the-moment interactions of rational, pragmatic speakers and listeners. To do so, we derive a novel objective function for measuring the communicative efficiency of linguistic systems in terms of the interactions of speakers and listeners. We examine the behavior of this objective in a series of simulations focusing on the communicative function of ambiguity in language. These simulations suggest that rational pragmatic agents will produce communicatively efficient systems and that interactions between such agents provide a framework for examining efficient properties of language structure and use more broadly.

A Bayesian decision-making framework for replication.

Replication is the cornerstone of science - but when and why? Not all studies need replication, especially when resources are limited. We propose that a decision-making framework based on Bayesian philosophy of science provides a basis for choosing which studies to replicate.