Auto-STEED: A data mining tool for automated extraction of experimental parameters and risk of bias items from in vivo publications.

Background: Systematic reviews, i.e., research summaries that address focused questions in a structured and reproducible manner, are a cornerstone of evidence-based medicine and research. However, certain systematic review steps such as data extraction are labour-intensive which hampers their applicability, not least with the rapidly expanding body of biomedical literature. Objective: To bridge this gap, we aimed at developing a data mining tool in the R programming environment to automate data extraction from neuroscience in vivo publications. The function was trained on a literature corpus (n=45 publications) of animal motor neuron disease studies and tested in two validation corpora (motor neuron diseases, n=31 publications; multiple sclerosis, n=244 publications). Results: Our data mining tool Auto-STEED (Automated and STructured Extraction of Experimental Data) was able to extract key experimental parameters such as animal models and species as well as risk of bias items such as randomization or blinding from in vivo studies. Sensitivity and specificity were over 85 and 80%, respectively, for most items in both validation corpora. Accuracy and F-scores were above 90% and 0.9 for most items in the validation corpora. Time savings were above 99%. Conclusions: Our developed text mining tool Auto-STEED is able to extract key experimental parameters and risk of bias items from the neuroscience in vivo literature. With this, the tool can be deployed to probe a field in a research improvement context or to replace one human reader during data extraction resulting in substantial time-savings and contribute towards automation of systematic reviews. The function is available on Github.

Handling method affects measures of anxiety, but not chronic stress in mice.

Studies in mice have shown that less aversive handling methods (e.g. tunnel or cup handling) can reduce behavioural measures of anxiety in comparison to picking mice up by their tail. Despite such evidence, tail handling continues to be used routinely. Besides resistance to change accustomed procedures, this may also be due to the fact that current evidence in support of less aversive handling is mostly restricted to effects of extensive daily handling, which may not apply to routine husbandry practices. The aim of our study was to assess whether, and to what extent, different handling methods during routine husbandry induce differences in behavioural and physiological measures of stress in laboratory mice. To put the effects of handling method in perspective with chronic stress, we compared handling methods to a validated paradigm of unpredictable chronic mild stress (UCMS). We housed mice of two strains (Balb/c and C57BL/6) and both sexes either under standard laboratory conditions (CTRL) or under UCMS. Half of the animals from each housing condition were tail handled and half were tunnel handled twice per week, once during a cage change and once for a routine health check. We found strain dependent effects of handling method on behavioural measures of anxiety: tunnel handled Balb/c mice interacted with the handler more than tail handled conspecifics, and tunnel handled CTRL mice showed increased open arm exploration in the elevated plus-maze. Mice undergoing UCMS showed increased plasma corticosterone levels and reduced sucrose preference. However, we found no effect of handling method on these stress-associated measures. Our results therefore indicate that routine tail handling can affect behavioural measures of anxiety, but may not be a significant source of chronic husbandry stress. Our results also highlight strain dependent responses to handling methods.

Correction: Do multiple experimenters improve the reproducibility of animal studies?

[This corrects the article DOI: 10.1371/journal.pbio.3001564.].

Reliability of common mouse behavioural tests of anxiety: A systematic review and meta-analysis on the effects of anxiolytics.

The validity of widely used rodent behavioural tests of anxiety has been questioned, as they often fail to produce consistent results across independent replicate studies. In this study, we assessed the sensitivity of common behavioural tests of anxiety in mice to detect anxiolytic effects of drugs prescribed to treat anxiety in humans. We conducted a pre-registered systematic review of 814 studies reporting effects of 25 anxiolytic compounds using common behavioural tests for anxiety. Meta-analyses of effect sizes of treatments showed that only two out of 17 commonly used test measures reliably detected effects of anxiolytic compounds. We report considerable between-study variation in size and even direction of effects of most anxiolytics on most outcome variables. Our findings indicate a general lack of sensitivity of those behavioural tests and cast serious doubt on both construct and predictive validity of most of these tests. In view of scientifically valid and ethically responsible research, we call for a revision of behavioural tests of anxiety in mice and the development of more predictive tests.

The rearing environment persistently modulates mouse phenotypes from the molecular to the behavioural level.

The phenotype of an organism results from its genotype and the influence of the environment throughout development. Even when using animals of the same genotype, independent studies may test animals of different phenotypes, resulting in poor replicability due to genotype-by-environment interactions. Thus, genetically defined strains of mice may respond differently to experimental treatments depending on their rearing environment. However, the extent of such phenotypic plasticity and its implications for the replicability of research findings have remained unknown. Here, we examined the extent to which common environmental differences between animal facilities modulate the phenotype of genetically homogeneous (inbred) mice. We conducted a comprehensive multicentre study, whereby inbred C57BL/6J mice from a single breeding cohort were allocated to and reared in 5 different animal facilities throughout early life and adolescence, before being transported to a single test laboratory. We found persistent effects of the rearing facility on the composition and heterogeneity of the gut microbial community. These effects were paralleled by persistent differences in body weight and in the behavioural phenotype of the mice. Furthermore, we show that environmental variation among animal facilities is strong enough to influence epigenetic patterns in neurons at the level of chromatin organisation. We detected changes in chromatin organisation in the regulatory regions of genes involved in nucleosome assembly, neuronal differentiation, synaptic plasticity, and regulation of behaviour. Our findings demonstrate that common environmental differences between animal facilities may produce facility-specific phenotypes, from the molecular to the behavioural level. Furthermore, they highlight an important limitation of inferences from single-laboratory studies and thus argue that study designs should take environmental background into account to increase the robustness and replicability of findings.

Do multiple experimenters improve the reproducibility of animal studies?

The credibility of scientific research has been seriously questioned by the widely claimed "reproducibility crisis". In light of this crisis, there is a growing awareness that the rigorous standardisation of experimental conditions may contribute to poor reproducibility of animal studies. Instead, systematic heterogenisation has been proposed as a tool to enhance reproducibility, but a real-life test across multiple independent laboratories is still pending. The aim of this study was therefore to test whether heterogenisation of experimental conditions by using multiple experimenters improves the reproducibility of research findings compared to standardised conditions with only one experimenter. To this end, we replicated the same animal experiment in 3 independent laboratories, each employing both a heterogenised and a standardised design. Whereas in the standardised design, all animals were tested by a single experimenter; in the heterogenised design, 3 different experimenters were involved in testing the animals. In contrast to our expectation, the inclusion of multiple experimenters in the heterogenised design did not improve the reproducibility of the results across the 3 laboratories. Interestingly, however, a variance component analysis indicated that the variation introduced by the different experimenters was not as high as the variation introduced by the laboratories, probably explaining why this heterogenisation strategy did not bring the anticipated success. Even more interestingly, for the majority of outcome measures, the remaining residual variation was identified as an important source of variance accounting for 41% (CI95 [34%, 49%]) to 72% (CI95 [58%, 88%]) of the observed total variance. Despite some uncertainty surrounding the estimated numbers, these findings argue for systematically including biological variation rather than eliminating it in animal studies and call for future research on effective improvement strategies.

Effects of weaning age and housing conditions on phenotypic differences in mice.

Poor reproducibility is considered a serious problem in laboratory animal research, with important scientific, economic, and ethical implications. One possible source of conflicting findings in laboratory animal research are environmental differences between animal facilities combined with rigorous environmental standardization within studies. Due to phenotypic plasticity, study-specific differences in environmental conditions during development can induce differences in the animals' responsiveness to experimental treatments, thereby contributing to poor reproducibility of experimental results. Here, we studied how variation in weaning age (14-30 days) and housing conditions (single versus group housing) affects the phenotype of SWISS mice as measured by a range of behavioral and physiological outcome variables. Weaning age, housing conditions, and their interaction had little effect on the development of stereotypies, as well as on body weight, glucocorticoid metabolite concentrations, and behavior in the elevated plus-maze and open field test. These results are surprising and partly in conflict with previously published findings, especially with respect to the effects of early weaning. Our results thus question the external validity of previous findings and call for further research to identify the sources of variation between replicate studies and study designs that produce robust and reproducible experimental results.