Open Science 2.0: Towards a truly collaborative research ecosystem.

Conversations about open science have reached the mainstream, yet many open science practices such as data sharing remain uncommon. Our efforts towards openness therefore need to increase in scale and aim for a more ambitious target. We need an ecosystem not only where research outputs are openly shared but also in which transparency permeates the research process from the start and lends itself to more rigorous and collaborative research. To support this vision, this Essay provides an overview of a selection of open science initiatives from the past 2 decades, focusing on methods transparency, scholarly communication, team science, and research culture, and speculates about what the future of open science could look like. It then draws on these examples to provide recommendations for how funders, institutions, journals, regulators, and other stakeholders can create an environment that is ripe for improvement.

Are most published research findings false in a continuous universe?

Diagnostic screening models for the interpretation of null hypothesis significance test (NHST) results have been influential in highlighting the effect of selective publication on the reproducibility of the published literature, leading to John Ioannidis' much-cited claim that most published research findings are false. These models, however, are typically based on the assumption that hypotheses are dichotomously true or false, without considering that effect sizes for different hypotheses are not the same. To address this limitation, we develop a simulation model that overcomes this by modeling effect sizes explicitly using different continuous distributions, while retaining other aspects of previous models such as publication bias and the pursuit of statistical significance. Our results show that the combination of selective publication, bias, low statistical power and unlikely hypotheses consistently leads to high proportions of false positives, irrespective of the effect size distribution assumed. Using continuous effect sizes also allows us to evaluate the degree of effect size overestimation and prevalence of estimates with the wrong sign in the literature, showing that the same factors that drive false-positive results also lead to errors in estimating effect size direction and magnitude. Nevertheless, the relative influence of these factors on different metrics varies depending on the distribution assumed for effect sizes. The model is made available as an R ShinyApp interface, allowing one to explore features of the literature in various scenarios.

Memory destabilization during reconsolidation: a consequence of homeostatic plasticity?

Remembering is not a static process: When retrieved, a memory can be destabilized and become prone to modifications. This phenomenon has been demonstrated in a number of brain regions, but the neuronal mechanisms that rule memory destabilization and its boundary conditions remain elusive. Using two distinct computational models that combine Hebbian plasticity and synaptic downscaling, we show that homeostatic plasticity can function as a destabilization mechanism, accounting for behavioral results of protein synthesis inhibition upon reactivation with different re-exposure times. Furthermore, by performing systematic reviews, we identify a series of overlapping molecular mechanisms between memory destabilization and synaptic downscaling, although direct experimental links between both phenomena remain scarce. In light of these results, we propose a theoretical framework where memory destabilization can emerge as an epiphenomenon of homeostatic adaptations prompted by memory retrieval.

Comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literature.

BACKGROUND: Preprint usage is growing rapidly in the life sciences; however, questions remain on the relative quality of preprints when compared to published articles. An objective dimension of quality that is readily measurable is completeness of reporting, as transparency can improve the reader's ability to independently interpret data and reproduce findings. METHODS: In this observational study, we initially compared independent samples of articles published in bioRxiv and in PubMed-indexed journals in 2016 using a quality of reporting questionnaire. After that, we performed paired comparisons between preprints from bioRxiv to their own peer-reviewed versions in journals. RESULTS: Peer-reviewed articles had, on average, higher quality of reporting than preprints, although the difference was small, with absolute differences of 5.0% [95% CI 1.4, 8.6] and 4.7% [95% CI 2.4, 7.0] of reported items in the independent samples and paired sample comparison, respectively. There were larger differences favoring peer-reviewed articles in subjective ratings of how clearly titles and abstracts presented the main findings and how easy it was to locate relevant reporting information. Changes in reporting from preprints to peer-reviewed versions did not correlate with the impact factor of the publication venue or with the time lag from bioRxiv to journal publication. CONCLUSIONS: Our results suggest that, on average, publication in a peer-reviewed journal is associated with improvement in quality of reporting. They also show that quality of reporting in preprints in the life sciences is within a similar range as that of peer-reviewed articles, albeit slightly lower on average, supporting the idea that preprints should be considered valid scientific contributions.

Two years into the Brazilian Reproducibility Initiative: reflections on conducting a large-scale replication of Brazilian biomedical science.

Scientists have increasingly recognised that low methodological and analytical rigour combined with publish-or-perish incentives can make the published scientific literature unreliable. As a response to this, large-scale systematic replications of the literature have emerged as a way to assess the problem empirically. The Brazilian Reproducibility Initiative is one such effort, aimed at estimating the reproducibility of Brazilian biomedical research. Its goal is to perform multicentre replications of a quasi-random sample of at least 60 experiments from Brazilian articles published over a 20-year period, using a set of common laboratory methods. In this article, we describe the challenges of managing a multicentre project with collaborating teams across the country, as well as its successes and failures over the first two years. We end with a brief discussion of the Initiative's current status and its possible future contributions after the project is concluded in 2021.

Using collaboration networks to identify authorship dependence in meta-analysis results.

Meta-analytic methods are powerful resources to summarize the existing evidence concerning a given research question and are widely used in many academic fields. Meta-analyzes can also be used to study sources of heterogeneity and bias among results, which should be considered to avoid inaccuracies. Many of these sources can be related to study authorship, as both methodological heterogeneity and researcher bias may lead to deviations in results between different research groups. In this work, we describe a method to objectively attribute study authorship within a given meta-analysis to different research groups by using graph cluster analysis of collaboration networks. We then provide empirical examples of how the research group of origin can impact effect size in distinct types of meta-analyzes, demonstrating how non-independence between within-group results can bias effect size estimates if uncorrected. Finally, we show that multilevel random-effects models using research group as a level of analysis can be a simple tool for correcting for authorship dependence in results.

Addressing selective reporting of experiments through predefined exclusion criteria.

The pressure for every research article to tell a clear story often leads researchers in the life sciences to exclude experiments that 'did not work' when they write up their results. However, this practice can lead to reporting bias if the decisions about which experiments to exclude are taken after data have been collected and analyzed. Here we discuss how to balance clarity and thoroughness when reporting the results of research, and suggest that predefining the criteria for excluding experiments might help researchers to achieve this balance.

Shifting from fear to safety through deconditioning-update.

Aversive memories are at the heart of psychiatric disorders such as phobias and post-traumatic stress disorder (PTSD). Here, we present a new behavioral approach in rats that robustly attenuates aversive memories. This method consists of 'deconditioning' animals previously trained to associate a tone with a strong footshock by replacing it with a much weaker one during memory retrieval. Our results indicate that deconditioning-update is more effective than traditional extinction in reducing fear responses; moreover, such effects are long lasting and resistant to renewal and spontaneous recovery. Remarkably, this strategy overcame important boundary conditions for memory updating, such as remote or very strong traumatic memories. The same beneficial effect was found in other types of fear-related memories. Deconditioning was mediated by L-type voltage-gated calcium channels and is consistent with computational accounts of mismatch-induced memory updating. Our results suggest that shifting from fear to safety through deconditioning-update is a promising approach to attenuate traumatic memories.

Two years into the Brazilian Reproducibility Initiative: reflections on conducting a large-scale replication of Brazilian biomedical science

Scientists have increasingly recognised that low methodological and analytical rigour combined with publish-or-perish incentives can make the published scientific literature unreliable. As a response to this, large-scale systematic replications of the literature have emerged as a way to assess the problem empirically. The Brazilian Reproducibility Initiative is one such effort, aimed at estimating the reproducibility of Brazilian biomedical research. Its goal is to perform multicentre replications of a quasi-random sample of at least 60 experiments from Brazilian articles published over a 20-year period, using a set of common laboratory methods. In this article, we describe the challenges of managing a multicentre project with collaborating teams across the country, as well as its successes and failures over the first two years. We end with a brief discussion of the Initiative's current status and its possible future contributions after the project is concluded in 2021.

A Freely Available, Self-Calibrating Software for Automatic Measurement of Freezing Behavior.

Freezing behavior is commonly used as a measure of associative fear memory. It can be measured by a trained observer, but this task is time-consuming and subject to variation. Commercially available software packages can also be used to quantify freezing; however, they can be expensive and usually require various parameters to be adjusted by the researcher, leading to additional work and variability in results. With this in mind, we developed Phobos, a freely available, self-calibrating software that measures freezing in a set of videos using a brief manual quantification performed by the user to automatically adjust parameters. To optimize the software, we used four different video sets with different features in order to determine the most relevant parameters, the amount of videos needed for calibration and the minimum criteria to consider it reliable. The results of four different users were compared in order to test intra- and interobserver variability in manual and automated freezing scores. Our results suggest that Phobos can be an inexpensive, simple and reliable tool for measurement of fear-related behavior, with intra- and interuser variability similar to that obtained with manual scoring.

Chronic in vivo optogenetic stimulation modulates neuronal excitability, spine morphology, and Hebbian plasticity in the mouse hippocampus.

Prolonged increases in excitation can trigger cell-wide homeostatic responses in neurons, altering membrane channels, promoting morphological changes, and ultimately reducing synaptic weights. However, how synaptic downscaling interacts with classical forms of Hebbian plasticity is still unclear. In this study, we investigated whether chronic optogenetic stimulation of hippocampus CA1 pyramidal neurons in freely moving mice could (a) cause morphological changes reminiscent of homeostatic scaling, (b) modulate synaptic currents that might compensate for chronic excitation, and (c) lead to alterations in Hebbian plasticity. After 24 hr of stimulation with 15-ms blue light pulses every 90 s, dendritic spine density and area were reduced in the CA1 region of mice expressing channelrhodopsin-2 (ChR2) when compared to controls. This protocol also reduced the amplitude of mEPSCs for both the AMPA and NMDA components in ex vivo slices obtained from ChR2-expressing mice immediately after the end of stimulation. Finally, chronic stimulation impaired the induction of LTP and facilitated that of LTD in these slices. Our results indicate that neuronal responses to prolonged network excitation can modulate subsequent Hebbian plasticity in the hippocampus.

The Brazilian Reproducibility Initiative.

Most efforts to estimate the reproducibility of published findings have focused on specific areas of research, even though science is usually assessed and funded on a regional or national basis. Here we describe a project to assess the reproducibility of findings in biomedical science published by researchers based in Brazil. The Brazilian Reproducibility Initiative is a systematic, multicenter effort to repeat between 60 and 100 experiments: the project will focus on a set of common methods, repeating each experiment in three different laboratories from a countrywide network. The results, due in 2021, will allow us to estimate the level of reproducibility of biomedical science in Brazil, and to investigate what aspects of the published literature might help to predict whether a finding is reproducible.

Different temporal windows for CB1 receptor involvement in contextual fear memory destabilisation in the amygdala and hippocampus.

Reconsolidation is a process in which re-exposure to a reminder causes a previously acquired memory to undergo a process of destabilisation followed by subsequent restabilisation. Different molecular mechanisms have been postulated for destabilisation in the amygdala and hippocampus, including CB1 receptor activation, protein degradation and AMPA receptor exchange; however, most of the amygdala studies have used pre-reexposure interventions, while those in the hippocampus have usually performed them after reexposure. To test whether the temporal window for destabilisation is similar across both structures, we trained Lister Hooded rats in a contextual fear conditioning task, and 1 day later performed memory reexposure followed by injection of either the NMDA antagonist MK-801 (0.1 mg/kg) or saline in order to block reconsolidation. In parallel, we also performed local injections of either the CB1 antagonist SR141716A or its vehicle in the hippocampus or in the amygdala, either immediately before or immediately after reactivation. Infusion of SR141716A in the hippocampus prevented the reconsolidation-blocking effect of MK-801 when performed after reexposure, but not before it. In the amygdala, meanwhile, pre-reexposure infusions of SR141716A impaired reconsolidation blockade by MK-801, although the time-dependency of this effect was not as clear as in the hippocampus. Our results suggest the temporal windows for CB1-receptor-mediated memory destabilisation during reconsolidation vary between brain structures. Whether this reflects different time windows for engagement of these structures or different roles played by CB1 receptors in destabilisation across structures remains an open question for future studies.

Effect size and statistical power in the rodent fear conditioning literature - A systematic review.

Proposals to increase research reproducibility frequently call for focusing on effect sizes instead of p values, as well as for increasing the statistical power of experiments. However, it is unclear to what extent these two concepts are indeed taken into account in basic biomedical science. To study this in a real-case scenario, we performed a systematic review of effect sizes and statistical power in studies on learning of rodent fear conditioning, a widely used behavioral task to evaluate memory. Our search criteria yielded 410 experiments comparing control and treated groups in 122 articles. Interventions had a mean effect size of 29.5%, and amnesia caused by memory-impairing interventions was nearly always partial. Mean statistical power to detect the average effect size observed in well-powered experiments with significant differences (37.2%) was 65%, and was lower among studies with non-significant results. Only one article reported a sample size calculation, and our estimated sample size to achieve 80% power considering typical effect sizes and variances (15 animals per group) was reached in only 12.2% of experiments. Actual effect sizes correlated with effect size inferences made by readers on the basis of textual descriptions of results only when findings were non-significant, and neither effect size nor power correlated with study quality indicators, number of citations or impact factor of the publishing journal. In summary, effect sizes and statistical power have a wide distribution in the rodent fear conditioning literature, but do not seem to have a large influence on how results are described or cited. Failure to take these concepts into consideration might limit attempts to improve reproducibility in this field of science.

On the transdiagnostic nature of peripheral biomarkers in major psychiatric disorders: A systematic review.

The search for biomarkers has been a leading endeavor in biological psychiatry. To analyze its evolution over the years, we performed a systematic review to evaluate (a) the most studied peripheral molecular markers in major psychiatric disorders, (b) the main features of studies proposing them as biomarkers and (c) whether their patterns of variation are similar across disorders. Of the six molecules most commonly studied as plasmatic markers of schizophrenia, major depressive disorder or bipolar disorder, five (BDNF, TNF-alpha, IL-6, C-reactive protein and cortisol) were the same across diagnoses. An analysis of this literature showed that, while 66% of studies compared patients and controls, only 34% were longitudinal, and only 10% presented a measure of diagnostic or prognostic efficacy. Meta-analyses showed variation in the levels of these molecules to be robust across studies, but similar among disorders, suggesting them to reflect transdiagnostic systemic consequences of psychiatric illness. Based on this, we discuss how current publication practices have led to research fragmentation across diagnoses, and suggest approaches to face this issue.

On the transition from reconsolidation to extinction of contextual fear memories.

Retrieval of an associative memory can lead to different phenomena. Brief reexposure sessions tend to trigger reconsolidation, whereas more extended ones trigger extinction. In appetitive and fear cued Pavlovian memories, an intermediate "null point" period has been observed where neither process seems to be engaged. Here we investigated whether this phenomenon extends to contextual fear memory. Adult rats were subjected to a contextual fear conditioning paradigm, reexposed to the context 2 d later for 3, 5, 10, 20, or 30 min, with immediate injections of MK-801 or saline following reexposure, and tested on the following day. We observed a significant effect of MK-801 with the 3- and 30-min sessions, impairing reconsolidation and extinction, respectively. However, it did not have significant effects with 5-, 10-, or 20-min sessions, even though freezing decreased from reexposure to test. Further analyses indicated that this is not likely to be due to a variable transition point at the population level. In conclusion, the results show that in contextual fear memories there is a genuine "null point" between the parameters that induce reconsolidation and extinction, as defined by the effects of MK-801, although NMDA receptor-independent decreases in freezing can still occur in these conditions.

Protocol for a Systematic Review of Effect Sizes and Statistical Power in the Rodent Fear Conditioning Literature.

The concepts of effect size and statistical power are often disregarded in basic neuroscience, and most articles in the field draw their conclusions solely based on the arbitrary significance thresholds of statistical inference tests. Moreover, studies are often underpowered, making conclusions from significance tests less reliable. With this in mind, we present the protocol of a systematic review to study the distribution of effect sizes and statistical power in the rodent fear conditioning literature, and to analyze how these factors influence the description and publication of results. To do this we will conduct a search in PubMed for "fear conditioning" AND ("mouse" OR "mice" OR "rat" OR "rats") and obtain all articles published online in 2013. Experiments will be included if they: (a) describe the effect(s) of a single intervention on fear conditioning acquisition or consolidation; (b) have a control group to which the experimental group is compared; (c) use freezing as a measure of conditioned fear; and (d) have available data on mean freezing, standard deviation and sample size of each group and on the statistical significance of the comparison. We will use the extracted data to calculate the distribution of effect sizes in these experiments, as well as the distribution of statistical power curves for detecting a range of differences at a threshold of α=0.05. We will assess correlations between these variables and (a) the chances of a result being statistically significant, (b) the way the result is described in the article text, (c) measures to reduce risk of bias in the article and (d) the impact factor of the journal and the number of citations of the article. We will also perform analyses to see whether effect sizes vary systematically across species, gender, conditioning protocols or intervention types.