Consensus-based guidance for conducting and reporting multi-analyst studies.

Any large dataset can be analyzed in a number of ways, and it is possible that the use of different analysis strategies will lead to different results and conclusions. One way to assess whether the results obtained depend on the analysis strategy chosen is to employ multiple analysts and leave each of them free to follow their own approach. Here, we present consensus-based guidance for conducting and reporting such multi-analyst studies, and we discuss how broader adoption of the multi-analyst approach has the potential to strengthen the robustness of results and conclusions obtained from analyses of datasets in basic and applied research.

Putting the Self in Self-Correction: Findings From the Loss-of-Confidence Project.

Science is often perceived to be a self-correcting enterprise. In principle, the assessment of scientific claims is supposed to proceed in a cumulative fashion, with the reigning theories of the day progressively approximating truth more accurately over time. In practice, however, cumulative self-correction tends to proceed less efficiently than one might naively suppose. Far from evaluating new evidence dispassionately and infallibly, individual scientists often cling stubbornly to prior findings. Here we explore the dynamics of scientific self-correction at an individual rather than collective level. In 13 written statements, researchers from diverse branches of psychology share why and how they have lost confidence in one of their own published findings. We qualitatively characterize these disclosures and explore their implications. A cross-disciplinary survey suggests that such loss-of-confidence sentiments are surprisingly common among members of the broader scientific population yet rarely become part of the public record. We argue that removing barriers to self-correction at the individual level is imperative if the scientific community as a whole is to achieve the ideal of efficient self-correction.

Scientific Utopia III: Crowdsourcing Science.

Most scientific research is conducted by small teams of investigators who together formulate hypotheses, collect data, conduct analyses, and report novel findings. These teams operate independently as vertically integrated silos. Here we argue that scientific research that is horizontally distributed can provide substantial complementary value, aiming to maximize available resources, promote inclusiveness and transparency, and increase rigor and reliability. This alternative approach enables researchers to tackle ambitious projects that would not be possible under the standard model. Crowdsourced scientific initiatives vary in the degree of communication between project members from largely independent work curated by a coordination team to crowd collaboration on shared activities. The potential benefits and challenges of large-scale collaboration span the entire research process: ideation, study design, data collection, data analysis, reporting, and peer review. Complementing traditional small science with crowdsourced approaches can accelerate the progress of science and improve the quality of scientific research.

Conformity under uncertainty: reliance on gender stereotypes in online hiring decisions.

We apply Bentley et al.'s theoretical framework to better understand gender discrimination in online labor markets. Although such settings are designed to encourage employer behavior in the northwest corner of Homo economicus, actual online hiring decisions tend to drift southeast into a "confirmation bias plus weak feedback loops" pattern of discrimination based on inaccurate social stereotypes.

It pays to be Herr Kaiser: Germans with noble-sounding surnames more often work as managers than as employees.

In the field study reported here (N = 222,924), we found that Germans with noble-sounding surnames, such as Kaiser ("emperor"), König ("king"), and Fürst ("prince"), more frequently hold managerial positions than Germans with last names that either refer to common everyday occupations, such as Koch ("cook"), Bauer ("farmer"), and Becker/Bäcker ("baker"), or do not refer to any social role. This phenomenon occurs despite the fact that noble-sounding surnames never indicated that the person actually held a noble title. Because of basic properties of associative cognition, the status linked to a name may spill over to its bearer and influence his or her occupational outcomes.