Measurement challenges and causes of incomplete results reporting of biomedical animal studies: Results from an interview study.

BACKGROUND: Existing evidence indicates that a significant amount of biomedical research involving animals remains unpublished. At the same time, we lack standards for measuring the extent of results reporting in animal research. Publication rates may vary significantly depending on the level of measurement such as an entire animal study, individual experiments within a study, or the number of animals used. METHODS: Drawing on semi-structured interviews with 18 experts and qualitative content analysis, we investigated challenges and opportunities for the measurement of incomplete reporting of biomedical animal research with specific reference to the German situation. We further investigate causes of incomplete reporting. RESULTS: The in-depth expert interviews revealed several reasons for why incomplete reporting in animal research is difficult to measure at all levels under the current circumstances. While precise quantification based on regulatory approval documentation is feasible at the level of entire studies, measuring incomplete reporting at the more individual experiment and animal levels presents formidable challenges. Expert-interviews further identified six drivers of incomplete reporting of results in animal research. Four of these are well documented in other fields of research: a lack of incentives to report non-positive results, pressures to 'deliver' positive results, perceptions that some data do not add value, and commercial pressures. The fifth driver, reputational concerns, appears to be far more salient in animal research than in human clinical trials. The final driver, socio-political pressures, may be unique to the field. DISCUSSION: Stakeholders in animal research should collaborate to develop a clear conceptualisation of complete reporting in animal research, facilitate valid measurements of the phenomenon, and develop incentives and rewards to overcome the causes for incomplete reporting.

Specific T cells targeting Staphylococcus aureus fibronectin-binding protein 1 induce a type 2/type 1 inflammatory response in sensitized atopic dermatitis patients.

BACKGROUND: Atopic dermatitis (AD) is one of the most common inflammatory skin diseases worldwide and Staphylococcus aureus colonization and secondary infections occur in the majority of AD patients. Allergic sensitizations against microbial antigens have been discussed as possible trigger factors of AD. Recently, we reported IgE sensitization against fibronectin-binding protein 1 (FBP1), an essential virulence component in S. aureus, in a subgroup of patients suffering from AD. To expand these findings by investigating delayed-type immune reactions, the objective of this study was to detect and phenotypically characterize FBP1-specific T cells as possible trigger factors in AD. METHODS: Immunodominant T-cell epitopes were mapped by proliferation testing of patient-derived FBP1-specific T-cell lines after stimulation with single 15mer peptides, which were derived from different functional domains of the FBP1 sequence. Major histocompatibility complex class II tetramers carrying immunodominant epitopes successfully stained T helper cells in 8 out of 8 HLA-matched, IgE-sensitized AD patients. RESULTS: Cytokine profiling of multimer-sorted cells revealed that predominantly the type 2 cytokines IL-13 and IL-4 were secreted by these cells. In contrast, IL-17, the marker cytokine for response to extracellular pathogens, was scarcely detectable. CONCLUSIONS: We demonstrate that FBP1 contains immunodominant peptides that induce a specific pro-inflammatory T helper cell response with increased Th2 levels that can drive an allergic inflammation in sensitized AD patients.

Results dissemination from completed clinical trials conducted at German university medical centers remained delayed and incomplete. The 2014 -2017 cohort.

OBJECTIVE: Timely publication of clinical trial results is central for evidence-based medicine. In this follow-up study we benchmark the performance of German university medical centers (UMCs) regarding timely dissemination of clinical trial results in recent years. METHODS: Following the same search and tracking methods used in our previous study for the years 2009 - 2013, we identified trials led by German UMCs completed between 2014 and 2017 and tracked results dissemination for the identified trials. RESULTS: We identified 1,658 trials in the 2014 -2017 cohort. Of these trials, 43% published results as either journal publication or summary results within 24 months after completion date, which is an improvement of 3.8% percentage points compared to the previous study. At the UMC level, the proportion published after 24 months ranged from 14% to 71%. Five years after completion, 30% of the trials still remained unpublished. CONCLUSION: Despite minor improvements compared to the previously investigated cohort, the proportion of timely reported trials led by German UMCs remains low. German UMCs should take further steps to improve the proportion of timely reported trials.

Web-based survey among animal researchers on publication practices and incentives for increasing publication rates.

OBJECTIVES: Publication bias, non-publication, and selective reporting of animal studies limit progress toward the 3Rs (Replacement, Reduction, and Refinement) that guide ethical animal testing, waste public resources, and result in redundant research, which collectively undermine the public's trust in scientific reliability. In this study, we aimed to 1) validate findings from a previous follow-up study by our team that examined the publication rates of animal studies from protocol to publication and 2) identify incentives for improving publication rates in animal research. METHODS: The researchers responsible for the animal proposals (n = 210) from our previous study were contacted as participants for a Web-based survey between October 2019 and April 2020. Question types varied between free text questions, answer options based on a 5-point Likert scale and closed yes/no questions. RESULTS: In total, 78 researchers responsible for 101 of 210 animal study proposals participated, yielding a response rate of 48.1%. Results showed that the publication rate increased from 67% in our follow-up study to 70%. According to a 5-point Likert scale (from 1 = "not relevant" to 5 = "extremely relevant"), the most widely accepted suggestions for increasing publication rates were "Publication costs for open access journals are fully covered by funders or universities" (mean 4.02, SD 1.01), "Performance-based allocation of intramural funds for results reporting of animal research not supporting the initial hypothesis (including preprints and repositories)" (mean 3.37, SD 1.05), and "Researchers receive more information from scientific journals that also publish non-significant results" (mean 3.30, SD 1.02). CONCLUSION: While the extent of publication and publication practices have been thoroughly investigated for clinical trials, less data is available for animal research to date. Therefore, the study contributes in complementing the picture of publication practice in animal research. Suggestions from our survey may help improve the publication rates of animal studies.

Investigator brochures for phase I/II trials lack information on the robustness of preclinical safety studies.

AIM: Meaningful and ethical phase I/II trials can only be conducted with supportive prospective risk-benefit assessment. This relies largely on preclinical animal studies addressing the safety and efficacy of treatments. These studies are reported in an Investigator's Brochure (IB) to inform ethics review boards and regulatory authorities. Our study investigated the extent, reporting quality and accessibility of preclinical safety studies (PCSSs) compiled in IBs. METHODS: We analysed a sample of 46 IBs for phase I/II trials approved at a leading German university medical centre from 2010 to 2016. We extracted all PCSSs presented in the 46 IBs and assessed them for reporting on methodological measures to reduce validity threats. RESULTS: The 46 IBs included 777 PCSSs. Blinded outcome assessment, randomization and sample size calculation were reported for fewer than 1% of studies. Only 5% of the PCSSs provided a reference to published data. Compliance with Good Laboratory Practice (GLP) guidance was reported for 52% of PCSSs, but the GLP document itself does not include any relevant methodological requirements for the reduction of validity threats. CONCLUSION: Scarce reporting in IBs and the very limited publicly available data on PCSSs make it almost impossible for investigators to critically evaluate the robustness of preclinical evidence of drug safety. Combined with recent findings on the presentation of preclinical efficacy studies in IBs, we conclude that the current reporting patterns in IBs strongly limit the independent review of evidential support for early human trials. Regulatory authorities and IRBs should require better reporting in IBs.

Results dissemination of registered clinical trials across Polish academic institutions: a cross-sectional analysis.

OBJECTIVES: To establish the rates of publication and reporting of results for interventional clinical trials across Polish academic medical centres (AMCs) completed between 2009 and 2013. We aim also to compare the publication and reporting success between adult and paediatric trials. DESIGN: Cross-sectional study. SETTING: AMCs in Poland. PARTICIPANTS: AMCs with interventional trials registered on ClinicalTrials.gov. MAIN OUTCOME MEASURE: Results reporting on ClinicalTrials.gov and publishing via journal publication. RESULTS: We identified 305 interventional clinical trials registered on ClinicalTrials.gov, completed between 2009 and 2013 and affiliated with at least one AMC. Overall, 243 of the 305 trials (79.7%) had been published as articles or posted their summary results on ClinicalTrials.gov. Results were posted within a year of study completion and/or published within 2 years of study completion for 131 trials (43.0%). Dissemination by both posting and publishing results in a timely manner was achieved by four trials (1.3%). CONCLUSIONS: Our cross-sectional analysis revealed that Polish AMCs fail to meet the expectation for timely disseminating the findings of all interventional clinical trials. Delayed dissemination and non-dissemination of trial results negatively affects decisions in healthcare.

Attitudes towards animal study registries and their characteristics: An online survey of three cohorts of animal researchers.

OBJECTIVES: Prospective registration of animal studies has been suggested as a new measure to increase value and reduce waste in biomedical research. We sought to further explore and quantify animal researchers' attitudes and preferences regarding animal study registries (ASRs). DESIGN: Cross-sectional online survey. SETTING AND PARTICIPANTS: We conducted a survey with three different samples representing animal researchers: i) corresponding authors from journals with high Eigenfactor, ii) a random Pubmed sample and iii) members of the CAMARADES network. MAIN OUTCOME MEASURES: Perceived level of importance of different aspects of publication bias, the effect of ASRs on different aspects of research as well as the importance of different research types for being registered. RESULTS: The survey yielded responses from 413 animal researchers (response rate 7%). The respondents indicated, that some aspects of ASRs can increase administrative burden but could be outweighed by other aspects decreasing this burden. Animal researchers found it more important to register studies that involved animal species with higher levels of cognitive capabilities. The time frame for making registry entries publicly available revealed a strong heterogeneity among respondents, with the largest proportion voting for "access only after consent by the principal investigator" and the second largest proportion voting for "access immediately after registration". CONCLUSIONS: The fact that the more senior and experienced animal researchers participating in this survey clearly indicated the practical importance of publication bias and the importance of ASRs underscores the problem awareness across animal researchers and the willingness to actively engage in study registration if effective safeguards for the potential weaknesses of ASRs are put into place. To overcome the first-mover dilemma international consensus statements on how to deal with prospective registration of animal studies might be necessary for all relevant stakeholder groups including animal researchers, academic institutions, private companies, funders, regulatory agencies, and journals.

Publication rates in animal research. Extent and characteristics of published and non-published animal studies followed up at two German university medical centres.

Non-publication and publication bias in animal research is a core topic in current debates on the "reproducibility crisis" and "failure rates in clinical research". To date, however, we lack reliable evidence on the extent of non-publication in animal research. We collected a random and stratified sample (n = 210) from all archived animal study protocols of two major German UMCs (university medical centres) and tracked their results publication. The overall publication rate was 67%. Excluding doctoral theses as results publications, the publication rate decreased to 58%. We did not find substantial differences in publication rates with regard to i) the year of animal study approval, ii) the two UMCs, iii) the animal type (rodents vs. non-rodents), iv) the scope of research (basic vs. preclinical), or v) the discipline of the applicant. Via the most reliable assessment strategy currently available, our study confirms that the non-publication of results from animal studies conducted at UMCs is relatively common. The non-publication of 33% of all animal studies is problematic for the following reasons: A) the primary legitimation of animal research, which is the intended knowledge gain for the wider scientific community, B) the waste of public resources, C) the unnecessary repetition of animal studies, and D) incomplete and potentially biased preclinical evidence for decision making on launching early human trials. Results dissemination should become a professional standard for animal research. Academic institutions and research funders should develop effective policies in this regard.

Result dissemination from clinical trials conducted at German university medical centers was delayed and incomplete.

OBJECTIVES: Timely and comprehensive reporting of clinical trial results builds the backbone of evidence-based medicine and responsible research. The proportion of timely disseminated trial results can inform alternative national and international benchmarking of university medical centers (UMCs). STUDY DESIGN AND SETTING: For all German UMCs, we tracked all registered trials completed between 2009 and 2013. The results and an interactive website benchmark German UMCs regarding their performance in result dissemination. RESULTS: We identified and tracked 2,132 clinical trials. For 1,509 trials, one of the German UMCs took the academic lead. Of these 1,509 "lead trials," 39% published their results (mostly via journal publications) in a timely manner (<24 months after completion). More than 6 years after study completion, 26% of all eligible lead trials still had not disseminated results. CONCLUSION: Despite substantial attention from many stakeholders to the topic, there is still a strong delay or even absence of result dissemination for many trials. German UMCs have several opportunities to improve this situation. Further research should evaluate whether and how a transparent benchmarking of UMC performance in result dissemination helps to increase value and reduce waste in medical research.

Preclinical efficacy in therapeutic area guidelines from the U.S. Food and Drug Administration and the European Medicines Agency: a cross-sectional study.

BACKGROUND AND PURPOSE: Therapeutic area guidelines (TAGs) published by the EMA and the FDA offer guidance in planning the launch of a trial in a certain indication. We assessed and compared the guidance on preclinical efficacy of all available TAGs from EMA and FDA. EXPERIMENTAL APPROACH: EMA and FDA websites and databases were searched for all TAGs. A mixed deductive and inductive approach was applied to analyse and cluster content for preclinical efficacy. KEY RESULTS: A total of 114 EMA and 120 FDA TAGs were identified, covering 126 indications. Our core finding is that 75% of EMA TAGs and 58% from the FDA TAGs do not offer any guidance on preclinical efficacy. TAGs varied widely on the extent, nature and detail of guidance. CONCLUSIONS AND IMPLICATIONS: Guidance on preclinical efficacy in a consistent, comprehensive and explicit way that still allows for justified deviations is an important but neglected aspect of transparency for drug development. This transparency would help sponsors in designing preclinical studies and in negotiating more efficiently with regulators.

Preclinical efficacy studies in investigator brochures: Do they enable risk-benefit assessment?

Human protection policies require favorable risk-benefit judgments prior to launch of clinical trials. For phase I and II trials, evidence for such judgment often stems from preclinical efficacy studies (PCESs). We undertook a systematic investigation of application materials (investigator brochures [IBs]) presented for ethics review for phase I and II trials to assess the content and properties of PCESs contained in them. Using a sample of 109 IBs most recently approved at 3 institutional review boards based at German Medical Faculties between the years 2010-2016, we identified 708 unique PCESs. We then rated all identified PCESs for their reporting on study elements that help to address validity threats, whether they referenced published reports, and the direction of their results. Altogether, the 109 IBs reported on 708 PCESs. Less than 5% of all PCESs described elements essential for reducing validity threats such as randomization, sample size calculation, and blinded outcome assessment. For most PCESs (89%), no reference to a published report was provided. Only 6% of all PCESs reported an outcome demonstrating no effect. For the majority of IBs (82%), all PCESs were described as reporting positive findings. Our results show that most IBs for phase I/II studies did not allow evaluators to systematically appraise the strength of the supporting preclinical findings. The very rare reporting of PCESs that demonstrated no effect raises concerns about potential design or reporting biases. Poor PCES design and reporting thwart risk-benefit evaluation during ethical review of phase I/II studies.

The spectrum of ethical issues in a Learning Health Care System: a systematic qualitative review.

PURPOSE: To determine systematically the spectrum of ethical issues that is raised for stakeholders in a 'Learning Health Care System' (LHCS). DATA SOURCES: The systematic review was conducted in PubMed and Google Books between the years 2007 and 2015. STUDY SELECTION: The literature search retrieved 1258 publications. Each publication was independently screened by two reviewers for eligibility for inclusion. Ethical issues were defined as arising when a relevant normative principle is not adequately considered or two principles come into conflict. DATA EXTRACTION: A total of 65 publications were included in the final analysis and were analysed using an adapted version of qualitative content analysis. A coding frame was developed inductively from the data, only the highest-level categories were generated deductively for a life-cycle perspective. RESULTS OF DATA SYNTHESIS: A total of 67 distinct ethical issues could be categorized under different phases of the LHCS life-cycle. An overarching theme that was repeatedly raised was the conflict between the current regulatory system and learning health care. CONCLUSION: The implementation of a LHCS can help realize the ethical imperative to continuously improve the quality of health care. However, the implementation of a LHCS can also raise a number of important ethical issues itself. This review highlights the importance for health care leaders and policy makers to balance the need to protect and respect individual participants involved in learning health care activities with the social value of improving health care.

Tracking the timely dissemination of clinical studies. Characteristics and impact of 10 tracking variables.

Background: Several meta-research studies and benchmarking activities have assessed how comprehensively and timely, academic institutions and private companies publish their clinical studies. These current "clinical trial tracking" activities differ substantially in how they sample relevant studies, and how they follow up on their publication. Methods: To allow informed policy and decision making on future publication assessment and benchmarking of institutions and companies, this paper outlines and discusses 10 variables that influence the tracking of timely publications. Tracking variables were initially selected by experts and by the authors through discussion. To validate the completeness of our set of variables, we conducted i) an explorative review of tracking studies and ii) an explorative tracking of registered clinical trials of three leading German university medical centres. Results: We identified the following 10 relevant variables impacting the tracking of clinical studies: 1) responsibility for clinical studies, 2) type and characteristics of clinical studies, 3) status of clinical studies, 4) source for sampling, 5) timing of registration, 6) determination of completion date, 7) timeliness of dissemination, 8) format of dissemination, 9) source for tracking, and 10) inter-rater reliability. Based on the description of these tracking variables and their influence, we discuss which variables could serve in what ways as a standard assessment of "timely publication". Conclusions: To facilitate the tracking and consequent benchmarking of how often and how timely academic institutions and private companies publish clinical study results, we have two core recommendations. First, the improvement in the link between registration and publication, for example via institutional policies for academic institutions and private companies. Second, the comprehensive and transparent reporting of tracking studies according to the 10 variables presented in this paper.

Animal Study Registries: Results from a Stakeholder Analysis on Potential Strengths, Weaknesses, Facilitators, and Barriers.

Publication bias in animal research, its extent, its predictors, and its potential countermeasures are increasingly discussed. Recent reports and conferences highlight the potential strengths of animal study registries (ASRs) in this regard. Others have warned that prospective registration of animal studies could diminish creativity, add administrative burdens, and complicate intellectual property issues in translational research. A literature review and 21 international key-informant interviews were conducted and thematically analyzed to develop a comprehensive matrix of main- and subcategories for potential ASR-related strengths, weaknesses, facilitators, and barriers (SWFBs). We identified 130 potential SWFBs. All stakeholder groups agreed that ASRs could in various ways improve the quality and refinement of animal studies while allowing their number to be reduced, as well as supporting meta-research on animal studies. However, all stakeholder groups also highlighted the potential for theft of ideas, higher administrative burdens, and reduced creativity and serendipity in animal studies. Much more detailed reasoning was captured in the interviews than is currently found in the literature, providing a comprehensive account of the issues and arguments around ASRs. All stakeholder groups highlighted compelling potential strengths of ASRs. Although substantial weaknesses and implementation barriers were highlighted as well, different governance measures might help to minimize or even eliminate their impact. Such measures might include confidentiality time frames for accessing prospectively registered protocols, harmonized reporting requirements across ASRs, ethics reviews, lab notebooks, and journal submissions. The comprehensive information gathered in this study could help to guide a more evidence-based debate and to design pilot tests for ASRs.

Ethics Reporting in Biospecimen and Genetic Research: Current Practice and Suggestions for Changes.

Modern approaches for research with human biospecimens employ a variety of substantially different types of ethics approval and informed consent. In most cases, standard ethics reporting such as "consent and approval was obtained" is no longer meaningful. A structured analysis of 120 biospecimen studies recently published in top journals revealed that more than 85% reported on consent and approval, but in more than 90% of cases, this reporting was insufficient and thus potentially misleading. Editorial policies, reporting guidelines, and material transfer agreements should include recommendations for meaningful ethics reporting in biospecimen research. Meaningful ethics reporting is possible without higher word counts and could support public trust as well as networked research.

α-NAC-Specific Autoreactive CD8+ T Cells in Atopic Dermatitis Are of an Effector Memory Type and Secrete IL-4 and IFN-γ.

Autoreactivity may play a critical role in the chronification of atopic dermatitis (AD). Several studies showed that AD patients produce IgE Abs specific for autoantigens, and we described Th as well as CD8(+) T cells specific for the autoallergen Hom s 2, the α-chain of the nascent polypeptide-associated complex (α-NAC). This study aimed to investigate the frequency and inflammatory phenotype of autoallergen-specific CD8(+) T cells. CD8(+) T cell immunodominant epitopes of α-NAC were mapped by applying prediction softwares, and binding affinity was confirmed by stabilization of empty MHC complexes. MHC class I tetramers were assembled and binding cells were analyzed directly ex vivo by flow cytometry and in terms of single-cell assessment by ChipCytometry. We report significantly elevated numbers of α-NAC-specific peripheral T cells in sensitized patients compared with nonatopic controls. These cells secrete IL-4 and IFN-γ, and surface markers revealed significantly elevated frequencies of circulating terminally differentiated α-NAC-specific CD8(+) T cells in patients with AD compared with nonatopic donors. The observed phenotype of α-NAC-specific CD8(+) T cells indicates a role in the pathogenesis of AD.