A Systematic Online Living Evidence Summary of experimental Alzheimer's disease research.

BACKGROUND: Despite extensive investment, the development of effective treatments for Alzheimer's disease (AD) has been largely unsuccessful. To improve translation, it is crucial to ensure the quality and reproducibility of foundational evidence generated from laboratory models. Systematic reviews play a key role in providing an unbiased overview of the evidence, assessing rigour and reporting, and identifying factors that influence reproducibility. However, the sheer pace of evidence generation is prohibitive to evidence synthesis and assessment. NEW METHOD: To address these challenges, we have developed AD-SOLES, an integrated workflow of automated tools that collect, curate, and visualise the totality of evidence from in vivo experiments. RESULTS: AD-SOLES is a publicly accessible interactive dashboard aiming to surface and expose data from in vivo experiments. It summarises the latest evidence, tracks reporting quality and transparency, and allows research users to easily locate evidence relevant to their specific research question. COMPARISON WITH EXISTING METHODS: Using automated screening methodologies within AD-SOLES, systematic reviews can begin at an accelerated starting point compared to traditional approaches. Furthermore, through text-mining approaches within the full-text of publications, users can identify research of interest using specific models, outcomes, or interventions without relying on details in the title and/or abstract. CONCLUSIONS: By automating the collection, curation, and visualisation of evidence from in vivo experiments, AD-SOLES addresses the challenges posed by the rapid pace of evidence generation. AD-SOLES aims to offer guidance for research improvement, reduce research waste, highlight knowledge gaps, and support informed decision making for researchers, funders, patients, and the public.

Trace amine-associated receptor 1 (TAAR1) agonism for psychosis: a living systematic review and meta-analysis of human and non-human data.

Background: Trace amine-associated receptor 1 (TAAR1) agonism shows promise for treating psychosis, prompting us to synthesise data from human and non-human studies. Methods: We co-produced a living systematic review of controlled studies examining TAAR1 agonists in individuals (with or without psychosis/schizophrenia) and relevant animal models. Two independent reviewers identified studies in multiple electronic databases (until 17.11.2023), extracted data, and assessed risk of bias. Primary outcomes were standardised mean differences (SMD) for overall symptoms in human studies and hyperlocomotion in animal models. We also examined adverse events and neurotransmitter signalling. We synthesised data with random-effects meta-analyses. Results: Nine randomised trials provided data for two TAAR1 agonists (ulotaront and ralmitaront), and 15 animal studies for 10 TAAR1 agonists. Ulotaront and ralmitaront demonstrated few differences compared to placebo in improving overall symptoms in adults with acute schizophrenia (N=4 studies, n=1291 participants; SMD=0.15, 95%CI: -0.05, 0.34), and ralmitaront was less efficacious than risperidone (N=1, n=156, SMD=-0.53, 95%CI: -0.86, -0.20). Large placebo response was observed in ulotaront phase-III trials. Limited evidence suggested a relatively benign side-effect profile for TAAR1 agonists, although nausea and sedation were common after a single dose of ulotaront. In animal studies, TAAR1 agonists improved hyperlocomotion compared to control (N=13 studies, k=41 experiments, SMD=1.01, 95%CI: 0.74, 1.27), but seemed less efficacious compared to dopamine D 2 receptor antagonists (N=4, k=7, SMD=-0.62, 95%CI: -1.32, 0.08). Limited human and animal data indicated that TAAR1 agonists may regulate presynaptic dopaminergic signalling. Conclusions: TAAR1 agonists may be less efficacious than dopamine D 2 receptor antagonists already licensed for schizophrenia. The results are preliminary due to the limited number of drugs examined, lack of longer-term data, publication bias, and assay sensitivity concerns in trials associated with large placebo response. Considering their unique mechanism of action, relatively benign side-effect profile and ongoing drug development, further research is warranted. Registration: PROSPERO-ID: CRD42023451628.

There is a need for more effective treatments for psychosis, including schizophrenia. Psychosis is a collection of mental health symptoms, such as hearing voices, that can cause distress and impair functioning. These symptoms are thought to be caused by changes in a chemical messenger system in the brain called dopamine. Currently used antipsychotic medications target brain receptors that respond to dopamine. They are not effective in some people and can cause uncomfortable adverse events, such as weight gain and movement disorders, especially with long-term use. A new type of drug is the trace amine-associated receptor 1 (TAAR1) agonists. These drugs act on different brain receptors that can affect the activity of the dopamine system, but do not directly bind to dopamine receptors. We aimed to understand if TAAR1 agonists can reduce symptoms of psychosis, what adverse events they might have, and how they work. We did this by reviewing and collating all available evidence until November 2023. This is a "living" systematic review, so it will be regularly updated in the future. We looked at both human and animal studies investigating TAAR1 agonists. Human studies suggested that two TAAR1 agonists (namely, ulotaront or ralmitaront) might have little to no effect on reducing symptoms of psychosis compared to placebo in people with schizophrenia. They seemed to cause fewer adverse events than current antipsychotics. Data from animal studies suggested that TAAR1 agonists had some positive effects but potentially smaller than other antipsychotics. There were little to no data from both human and animal studies about how TAAR1 agonists actually work. From the current evidence we are uncertain about these results. With the ongoing development of new TAAR1 agonists, more evidence is needed to understand their potential role in the treatment of psychosis.

Analysis of animal-to-human translation shows that only 5% of animal-tested therapeutic interventions obtain regulatory approval for human applications.

There is an ongoing debate about the value of animal experiments to inform medical practice, yet there are limited data on how well therapies developed in animal studies translate to humans. We aimed to assess 2 measures of translation across various biomedical fields: (1) The proportion of therapies which transition from animal studies to human application, including involved timeframes; and (2) the consistency between animal and human study results. Thus, we conducted an umbrella review, including English systematic reviews that evaluated the translation of therapies from animals to humans. Medline, Embase, and Web of Science Core Collection were searched from inception until August 1, 2023. We assessed the proportion of therapeutic interventions advancing to any human study, a randomized controlled trial (RCT), and regulatory approval. We meta-analyzed the concordance between animal and human studies. The risk of bias was probed using a 10-item checklist for systematic reviews. We included 122 articles, describing 54 distinct human diseases and 367 therapeutic interventions. Neurological diseases were the focus of 32% of reviews. The overall proportion of therapies progressing from animal studies was 50% to human studies, 40% to RCTs, and 5% to regulatory approval. Notably, our meta-analysis showed an 86% concordance between positive results in animal and clinical studies. The median transition times from animal studies were 5, 7, and 10 years to reach any human study, an RCT, and regulatory approval, respectively. We conclude that, contrary to widespread assertions, the rate of successful animal-to-human translation may be higher than previously reported. Nonetheless, the low rate of final approval indicates potential deficiencies in the design of both animal studies and early clinical trials. To ameliorate the efficacy of translating therapies from bench to bedside, we advocate for enhanced study design robustness and the reinforcement of generalizability.

Prevention of infections and fever to improve outcome in older patients with acute stroke (PRECIOUS): a randomised, open, phase III, multifactorial, clinical trial with blinded outcome assessment.

Background: Infections and fever after stroke are associated with poor functional outcome or death. We assessed whether prophylactic treatment with anti-emetic, antibiotic, or antipyretic medication would improve functional outcome in older patients with acute stroke. Methods: We conducted an international, 2∗2∗2-factorial, randomised, controlled, open-label trial with blinded outcome assessment in patients aged 66 years or older with acute ischaemic stroke or intracerebral haemorrhage and a score on the National Institutes of Health Stroke Scale ≥ 6. Patients were randomly allocated (1:1) to metoclopramide (oral, rectal, or intravenous; 10 mg thrice daily) vs. no metoclopramide, ceftriaxone (intravenous; 2000 mg once daily) vs. no ceftriaxone, and paracetamol (oral, rectal, or intravenous; 1000 mg four times daily) vs. no paracetamol, started within 24 h after symptom onset and continued for four days. All participants received standard of care. The target sample size was 3800 patients. The primary outcome was the score on the modified Rankin Scale (mRS) at 90 days analysed with ordinal logistic regression and reported as an adjusted common odds ratio (an acOR < 1 suggests benefit and an acOR > 1 harm). This trial is registered (ISRCTN82217627). Findings: From April 2016 through June 2022, 1493 patients from 67 European sites were randomised to metoclopramide (n = 704) or no metoclopramide (n = 709), ceftriaxone (n = 594) or no ceftriaxone (n = 482), and paracetamol (n = 706) or no paracetamol (n = 739), of whom 1471 were included in the intention-to-treat analysis. Prophylactic use of study medication did not significantly alter the primary outcome at 90 days: metoclopramide vs. no metoclopramide (adjusted common odds ratio [acOR], 1.01; 95% CI 0.81-1.25), ceftriaxone vs. no ceftriaxone (acOR 0.99; 95% CI 0.77-1.27), paracetamol vs. no paracetamol (acOR 1.19; 95% CI 0.96-1.47). The study drugs were safe and not associated with an increased incidence of serious adverse events. Interpretation: We observed no sign of benefit of prophylactic use of metoclopramide, ceftriaxone, or paracetamol during four days in older patients with a moderately severe to severe acute stroke. Funding: This project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No: 634809.

Designing, conducting, and reporting reproducible animal experiments.

In biomedicine and many other fields, there are growing concerns around the reproducibility of research findings, with many researchers being unable to replicate their own or others' results. This raises important questions as to the validity and usefulness of much published research. In this review, we aim to engage researchers in the issue of research reproducibility and equip them with the necessary tools to increase the reproducibility of their research. We first highlight the causes and potential impact of non-reproducible research and emphasise the benefits of working reproducibly for the researcher and broader research community. We address specific targets for improvement and steps that individual researchers can take to increase the reproducibility of their work. We next provide recommendations for improving the design and conduct of experiments, focusing on in vivo animal experiments. We describe common sources of poor internal validity of experiments and offer practical guidance for limiting these potential sources of bias at different experimental stages, as well as discussing other important considerations during experimental design. We provide a list of key resources available to researchers to improve experimental design, conduct, and reporting. We then discuss the importance of open research practices such as study preregistration and the use of preprints and describe recommendations around data management and sharing. Our review emphasises the importance of reproducible work and aims to empower every individual researcher to contribute to the reproducibility of research in their field.

Trace amine-associated receptor 1 (TAAR1) agonists for psychosis: protocol for a living systematic review and meta-analysis of human and non-human studies.

BACKGROUND: There is an urgent need to develop more effective and safer antipsychotics beyond dopamine 2 receptor antagonists. An emerging and promising approach is TAAR1 agonism. Therefore, we will conduct a living systematic review and meta-analysis to synthesize and triangulate the evidence from preclinical animal experiments and clinical studies on the efficacy, safety, and underlying mechanism of action of TAAR1 agonism for psychosis. METHODS: Independent searches will be conducted in multiple electronic databases to identify clinical and animal experimental studies comparing TAAR1 agonists with licensed antipsychotics or other control conditions in individuals with psychosis or animal models for psychosis, respectively. The primary outcomes will be overall psychotic symptoms and their behavioural proxies in animals. Secondary outcomes will include side effects and neurobiological measures. Two independent reviewers will conduct study selection, data extraction using predefined forms, and risk of bias assessment using suitable tools based on the study design. Ontologies will be developed to facilitate study identification and data extraction. Data from clinical and animal studies will be synthesized separately using random-effects meta-analysis if appropriate, or synthesis without meta-analysis. Study characteristics will be investigated as potential sources of heterogeneity. Confidence in the evidence for each outcome and source of evidence will be evaluated, considering the summary of the association, potential concerns regarding internal and external validity, and reporting biases. When multiple sources of evidence are available for an outcome, an overall conclusion will be drawn in a triangulation meeting involving a multidisciplinary team of experts. We plan trimonthly updates of the review, and any modifications in the protocol will be documented. The review will be co-produced by multiple stakeholders aiming to produce impactful and relevant results and bridge the gap between preclinical and clinical research on psychosis. PROTOCOL REGISTRATION: PROSPERO-ID: CRD42023451628.

Motor Neuron Disease Systematic Multi-Arm Adaptive Randomised Trial (MND-SMART): a multi-arm, multi-stage, adaptive, platform, phase III randomised, double-blind, placebo-controlled trial of repurposed drugs in motor neuron disease.

INTRODUCTION: Motor neuron disease (MND) is a rapidly fatal neurodegenerative disease. Despite decades of research and clinical trials there remains no cure and only one globally approved drug, riluzole, which prolongs survival by 2-3 months. Recent improved mechanistic understanding of MND heralds a new translational era with many potential targets being identified that are ripe for clinical trials. Motor Neuron Disease Systematic Multi-Arm Adaptive Randomised Trial (MND-SMART) aims to evaluate the efficacy of drugs efficiently and definitively in a multi-arm, multi-stage, adaptive trial. The first two drugs selected for evaluation in MND-SMART are trazodone and memantine. METHODS AND ANALYSIS: Initially, up to 531 participants (177/arm) will be randomised 1:1:1 to oral liquid trazodone, memantine and placebo. The coprimary outcome measures are the Amyotrophic Lateral Sclerosis Functional Rating Scale Revised (ALSFRS-R) and survival. Comparisons will be conducted in four stages. The decision to continue randomising to arms after each stage will be made by the Trial Steering Committee who receive recommendations from the Independent Data Monitoring Committee. The primary analysis of ALSFRS-R will be conducted when 150 participants/arm, excluding long survivors, have completed 18 months of treatment; if positive the survival effect will be inferentially analysed when 113 deaths have been observed in the placebo group. The trial design ensures that other promising drugs can be added for evaluation in planned trial adaptations. Using this novel trial design reduces time, cost and number of participants required to definitively (phase III) evaluate drugs and reduces exposure of participants to potentially ineffective treatments. ETHICS AND DISSEMINATION: MND-SMART was approved by the West of Scotland Research Ethics Committee on 2 October 2019. (REC reference: 19/WS/0123) Results of the study will be submitted for publication in a peer-reviewed journal and a summary provided to participants. TRIAL REGISTRATION NUMBERS: European Clinical Trials Registry (2019-000099-41); NCT04302870.

Using median survival in meta-analysis of experimental time-to-event data.

BACKGROUND: Time-to-event data is frequently reported in both clinical and preclinical research spheres. Systematic review and meta-analysis is a tool that can help to identify pitfalls in preclinical research conduct and reporting that can help to improve translational efficacy. However, pooling of studies using hazard ratios (HRs) is cumbersome especially in preclinical meta-analyses including large numbers of small studies. Median survival is a much simpler metric although because of some limitations, which may not apply to preclinical data, it is generally not used in survival meta-analysis. We aimed to appraise its performance when compared with hazard ratio-based meta-analysis when pooling large numbers of small, imprecise studies. METHODS: We simulated a survival dataset with features representative of a typical preclinical survival meta-analysis, including with influence of a treatment and a number of covariates. We calculated individual patient data-based hazard ratios and median survival ratios (MSRs), comparing the summary statistics directly and their performance at random-effects meta-analysis. Finally, we compared their sensitivity to detect associations between treatment and influential covariates at meta-regression. RESULTS: There was an imperfect correlation between MSR and HR, although the opposing direction of treatment effects between summary statistics appeared not to be a major issue. Precision was more conservative for HR than MSR, meaning that estimates of heterogeneity were lower. There was a slight sensitivity advantage for MSR at meta-analysis and meta-regression, although power was low in all circumstances. CONCLUSIONS: We believe we have validated MSR as a summary statistic for use in a meta-analysis of small, imprecise experimental survival studies-helping to increase confidence and efficiency in future reviews in this area. While assessment of study precision and therefore weighting is less reliable, MSR appears to perform favourably during meta-analysis. Sensitivity of meta-regression was low for this set of parameters, so pooling of treatments to increase sample size may be required to ensure confidence in preclinical survival meta-regressions.

Regulatory delays in a multinational clinical stroke trial.

INTRODUCTION: The initiation and conduct of randomised clinical trials are complicated by multiple barriers, including delays in obtaining regulatory approvals. Quantitative data on the extent of the delays due to national or local review in randomised clinical trials is scarce. MATERIALS AND METHODS: We assessed the times needed to obtain regulatory approval and to initiate a trial site for an academic, EU-funded, phase III, randomised clinical trial of pharmacological prevention of complications in patients with acute stroke in over 80 sites in nine European countries. The primary outcome was the time from the first submission to a regulatory authority to initiation of a trial site. Secondary outcomes included time needed to complete each individual preparatory requirement and the number of patients recruited by each site in the first 6 and 12 months. RESULTS: The median time from the first submission to a regulatory authority to initiation of a trial site was 784 days (IQR: 586-1102). The single most time-consuming step was the conclusion of a clinical trial agreement between the national coordinator and the trial site, which took a median of 194 days (IQR: 93-293). A longer time to site initiation was associated with a lower patient recruitment rate in the first six months after initiation (B = -0.002; p = 0.02). CONCLUSION: In this EU-funded clinical trial, approximately 26 months were needed to initiate a trial site for patient recruitment. The conclusion of a contract with a trial site was the most time-consuming activity. To simplify and speed up the process, we suggest that the level of detail of contracts for academic trials should be proportional to the risks and commercial interests of these trials.

Red blood cell transfusion in animal models of acute brain injuries: a systematic review protocol.

BACKGROUND: Anemia is common in neurocritically ill patients. Considering the limited clinical evidence in this population, preclinical data may provide some understanding of the potential impact of anemia and of red blood cell transfusion in these patients. We aim to estimate the association between different transfusion strategies and neurobehavioral outcome in animal models. METHODS: We will conduct a systematic review of comparative studies of red blood cell transfusion strategies using animal models of traumatic brain injury, ischemic stroke or cerebral hemorrhage. We will search MEDLINE, EMBASE, and Web of Science databases for eligible studies from inception onwards. Two independent reviewers will perform study selection and data extraction. We will report our results in a descriptive synthesis focusing on characteristics of included studies, reported outcomes, risk of bias, and construct validity. Our primary outcome is the neurological function (neurobehavioral performance) and our secondary outcomes include mortality, infarct size, intracranial pressure, cerebral perfusion pressure, cerebral blood flow, and brain tissue oxygen tension. If appropriate, we will also perform a quantitative synthesis and pool results using random-effect models. Heterogeneity will be expressed with I2 statistics. Subgroup analyses are planned according to animal model characteristics, co-interventions, and risks of bias. DISCUSSION: Our study is aligned with the efforts to better understand the level of evidence on the impact of red blood cell transfusion strategies from preclinical studies in animal models of acute brain injury and the potential translation of information from the preclinical to the clinical research field. SYSTEMATIC REVIEW REGISTRATION: PROSPERO CRD42018086662 .

Meta-analysis of variation suggests that embracing variability improves both replicability and generalizability in preclinical research.

The replicability of research results has been a cause of increasing concern to the scientific community. The long-held belief that experimental standardization begets replicability has also been recently challenged, with the observation that the reduction of variability within studies can lead to idiosyncratic, lab-specific results that cannot be replicated. An alternative approach is to, instead, deliberately introduce heterogeneity, known as "heterogenization" of experimental design. Here, we explore a novel perspective in the heterogenization program in a meta-analysis of variability in observed phenotypic outcomes in both control and experimental animal models of ischemic stroke. First, by quantifying interindividual variability across control groups, we illustrate that the amount of heterogeneity in disease state (infarct volume) differs according to methodological approach, for example, in disease induction methods and disease models. We argue that such methods may improve replicability by creating diverse and representative distribution of baseline disease state in the reference group, against which treatment efficacy is assessed. Second, we illustrate how meta-analysis can be used to simultaneously assess efficacy and stability (i.e., mean effect and among-individual variability). We identify treatments that have efficacy and are generalizable to the population level (i.e., low interindividual variability), as well as those where there is high interindividual variability in response; for these, latter treatments translation to a clinical setting may require nuance. We argue that by embracing rather than seeking to minimize variability in phenotypic outcomes, we can motivate the shift toward heterogenization and improve both the replicability and generalizability of preclinical research.

Introduction to the EQIPD quality system.

While high risk of failure is an inherent part of developing innovative therapies, it can be reduced by adherence to evidence-based rigorous research practices. Supported through the European Union's Innovative Medicines Initiative, the EQIPD consortium has developed a novel preclinical research quality system that can be applied in both public and private sectors and is free for anyone to use. The EQIPD Quality System was designed to be suited to boost innovation by ensuring the generation of robust and reliable preclinical data while being lean, effective and not becoming a burden that could negatively impact the freedom to explore scientific questions. EQIPD defines research quality as the extent to which research data are fit for their intended use. Fitness, in this context, is defined by the stakeholders, who are the scientists directly involved in the research, but also their funders, sponsors, publishers, research tool manufacturers, and collaboration partners such as peers in a multi-site research project. The essence of the EQIPD Quality System is the set of 18 core requirements that can be addressed flexibly, according to user-specific needs and following a user-defined trajectory. The EQIPD Quality System proposes guidance on expectations for quality-related measures, defines criteria for adequate processes (i.e. performance standards) and provides examples of how such measures can be developed and implemented. However, it does not prescribe any pre-determined solutions. EQIPD has also developed tools (for optional use) to support users in implementing the system and assessment services for those research units that successfully implement the quality system and seek formal accreditation. Building upon the feedback from users and continuous improvement, a sustainable EQIPD Quality System will ultimately serve the entire community of scientists conducting non-regulated preclinical research, by helping them generate reliable data that are fit for their intended use.

Comparison of commonly used methods in random effects meta-analysis: application to preclinical data in drug discovery research.

BACKGROUND: Meta-analysis of preclinical data is used to evaluate the consistency of findings and to inform the design and conduct of future studies. Unlike clinical meta-analysis, preclinical data often involve many heterogeneous studies reporting outcomes from a small number of animals. Here, we review the methodological challenges in preclinical meta-analysis in estimating and explaining heterogeneity in treatment effects. METHODS: Assuming aggregate-level data, we focus on two topics: (1) estimation of heterogeneity using commonly used methods in preclinical meta-analysis: method of moments (DerSimonian and Laird; DL), maximum likelihood (restricted maximum likelihood; REML) and Bayesian approach; (2) comparison of univariate versus multivariable meta-regression for adjusting estimated treatment effects for heterogeneity. Using data from a systematic review on the efficacy of interleukin-1 receptor antagonist in animals with stroke, we compare these methods, and explore the impact of multiple covariates on the treatment effects. RESULTS: We observed that the three methods for estimating heterogeneity yielded similar estimates for the overall effect, but different estimates for between-study variability. The proportion of heterogeneity explained by a covariate is estimated larger using REML and the Bayesian method as compared with DL. Multivariable meta-regression explains more heterogeneity than univariate meta-regression. CONCLUSIONS: Our findings highlight the importance of careful selection of the estimation method and the use of multivariable meta-regression to explain heterogeneity. There was no difference between REML and the Bayesian method and both methods are recommended over DL. Multiple meta-regression is worthwhile to explain heterogeneity by more than one variable, reducing more variability than any univariate models and increasing the explained proportion of heterogeneity.

A Systematic Review and Meta-Analysis of Hypothermia in Experimental Traumatic Brain Injury: Why Have Promising Animal Studies Not Been Replicated in Pragmatic Clinical Trials?

Therapeutic hypothermia was a mainstay of severe traumatic brain injury (TBI) management for half a century. Recent trials have suggested that its effect on long-term functional outcome is neutral or negative, despite apparently promising pre-clinical data. Systematic review and meta-analysis is a useful tool to collate experimental data and investigate the basis of its conclusions. We searched three online databases to identify studies testing systemic hypothermia as monotherapy for treatment of animals subjected to a TBI. Data pertaining to TBI paradigm, animal subjects, and hypothermia management were extracted as well as those relating to risk of bias. We pooled outcome data where sufficient numbers allowed and investigated heterogeneity in neurobehavioral outcomes using multi-variate meta-regression. We identified 90 publications reporting 272 experiments testing hypothermia in animals subject to TBI. The subjects were mostly small animals, with well-established models predominating. Target temperature was comparable to clinical trial data but treatment was initiated very early. Study quality was low and there was some evidence of publication bias. Delay to treatment, comorbidity, and blinded outcome assessment appeared to predict neurobehavioral outcome on multi-variate meta-regression. Therapeutic hypothermia appears to be an efficacious treatment in experimental TBI, which differs from the clinical evidence. The pre-clinical literature showed limitations in quality and design and these both appeared to affect neurobehavioral experiment outcome. These should be acknowledged when designing and interpreting pre-clinical TBI studies in the future.

Framework for advancing rigorous research.

There is a pressing need to increase the rigor of research in the life and biomedical sciences. To address this issue, we propose that communities of 'rigor champions' be established to campaign for reforms of the research culture that has led to shortcomings in rigor. These communities of rigor champions would also assist in the development and adoption of a comprehensive educational platform that would teach the principles of rigorous science to researchers at all career stages.

Research Culture and Reproducibility.

There is ongoing debate regarding the robustness and credibility of published scientific research. We argue that these issues stem from two broad causal mechanisms: the cognitive biases of researchers and the incentive structures within which researchers operate. The UK Reproducibility Network (UKRN) is working with researchers, institutions, funders, publishers, and other stakeholders to address these issues.

Effects of dietary fat manipulation on cognition in mice and rats: protocol for a systematic review and meta-analysis.

INTRODUCTION AND OBJECTIVE: The Western diet that comprises high levels of long-chain saturated fats and sugar is associated not only with metabolic disorders such as obesity and type 2 diabetes but also has been recently linked to brain changes and cognitive dysfunction. However, in animal studies, reported effects are variable, and the mechanisms underlying these effects are unclear. In the proposed review, we aim to summarise the diverse evidence of the effects of so-called 'high-fat' and ketogenic diets on behavioural measures of cognition in postweaning mice and rats, relative to animals on standard diets and to determine potential underlying mechanisms of high-fat diet-induced effects. SEARCH STRATEGY: A comprehensive search strategy was designed to retrieve studies reporting use of a high-fat or ketogenic diet in postweaning mice and rats that included cognitive assessments. Three databases (Medline, SCOPUS and Web of Science) were searched and 4487 unique references were retrieved. SCREENING AND ANNOTATION: Studies were screened for inclusion by two independent reviewers, with 330 studies retained for analysis. Characteristics of disease model choice, experimental design, intervention use and outcome assessment are to be extracted using the Systematic Review Facility (http://syrf.org.uk/) tool. Studies will be assessed for study quality and risk of bias and confidence of mechanistic involvement. DATA MANAGEMENT AND REPORTING: For cognitive outcomes, effect sizes will be calculated using normalised mean difference and summarised using a random effects model. The contribution of potential sources of heterogeneity to the observed effects of diet on cognition will be assessed using multivariable meta-regression, with partitioning of heterogeneity as a sensitivity analysis. A preliminary version of this protocol was published on 9 April 2019 on the Collaborative Approach to Meta-Analysis and Review of Animal Data from Experimental Studies website (http://www.dcn.ed.ac.uk/camarades/research.html%23protocols). ETHICS AND DISSEMINATION: No ethical approval is required as there are no subjects in the proposed study.

Design of Meta-Analysis Studies.

Any given research claim can be made with a degree of confidence that a phenomenon is present, with an estimate of the precision of the observed effects and a prediction of the extent to which the findings might hold true under different experimental or real-world conditions. In some situations, the certainty and precision obtained from a single study are sufficient reliably to inform future research decisions. However, in other situations greater certainty is required. This might be the case where a substantial research investment is planned, a pivotal claim is to be made or the launch of a clinical trial programme is being considered. Under these circumstances, some form of summary of findings across studies may be helpful.Summary estimates can describe findings from exploratory (observational) or hypothesis testing experiments, but importantly, the creation of such summaries is, in itself, observational rather than experimental research. The process is therefore particularly at risk from selective identification of literature to be included, and this can be addressed using systematic search strategies and pre-specified criteria for inclusion and exclusion against which possible contributing data will be assessed. This characterises a systematic review (in contrast to nonsystematic or narrative reviews). In meta-analysis, there is an attempt to provide a quantitative summary of such research findings.