Replication of null results: Absence of evidence or evidence of absence?

In several large-scale replication projects, statistically non-significant results in both the original and the replication study have been interpreted as a 'replication success.' Here, we discuss the logical problems with this approach: Non-significance in both studies does not ensure that the studies provide evidence for the absence of an effect and 'replication success' can virtually always be achieved if the sample sizes are small enough. In addition, the relevant error rates are not controlled. We show how methods, such as equivalence testing and Bayes factors, can be used to adequately quantify the evidence for the absence of an effect and how they can be applied in the replication setting. Using data from the Reproducibility Project: Cancer Biology, the Experimental Philosophy Replicability Project, and the Reproducibility Project: Psychology we illustrate that many original and replication studies with 'null results' are in fact inconclusive. We conclude that it is important to also replicate studies with statistically non-significant results, but that they should be designed, analyzed, and interpreted appropriately.

Combining evidence from clinical trials in conditional or accelerated approval.

Conditional (European Medicines Agency) or accelerated (U.S. Food and Drug Administration) approval of drugs allows earlier access to promising new treatments that address unmet medical needs. Certain post-marketing requirements must typically be met in order to obtain full approval, such as conducting a new post-market clinical trial. We study the applicability of the recently developed harmonic mean χ 2 -test to this conditional or accelerated approval framework. The proposed approach can be used both to support the design of the post-market trial and the analysis of the combined evidence provided by both trials. Other methods considered are the two-trials rule, Fisher's criterion and Stouffer's method. In contrast to some of the traditional methods, the harmonic mean χ 2 -test always requires a post-market clinical trial. If the p -value from the pre-market clinical trial is ≪ 0.025 , a smaller sample size for the post-market clinical trial is needed than with the two-trials rule. For illustration, we apply the harmonic mean χ 2 -test to a drug which received conditional (and later full) market licensing by the EMA. A simulation study is conducted to study the operating characteristics of the harmonic mean χ 2 -test and two-trials rule in more detail. We finally investigate the applicability of these two methods to compute the power at interim of an ongoing post-market trial. These results are expected to aid in the design and assessment of the required post-market studies in terms of the level of evidence required for full approval.

Pathology of Echinococcosis: A Morphologic and Immunohistochemical Study on 138 Specimens With Focus on the Differential Diagnosis Between Cystic and Alveolar Echinococcosis.

Infection of humans by the larval stage of the tapeworms Echinococcus granulosus sensu lato or Echinococcus multilocularis causes the life-threatening zoonoses cystic echinococcosis (CE) and alveolar echinococcosis (AE). Although cystic liver lesions are a hallmark of both diseases, course, prognosis, and patients' management decisively differ between the two. The wide and overlapping spectrum of morphologies and the limited availability of ancillary tools are challenges for pathologists to reliably diagnose and subtype echinococcosis. Here, we systematically and quantitatively recorded the pathologic spectrum in a clinically and molecularly defined echinococcosis cohort (138 specimens from 112 patients). Immunohistochemistry using a novel monoclonal antibody (mAbEmG3) was implemented, including its combined application with the mAbEm2G11. Six morphologic criteria sufficiently discriminated between CE and AE: size of smallest (CE/AE: >2/≤2 mm) and largest cyst (CE/AE: >25/≤25 mm), thickness of laminated layer (CE/AE: >0.15/≤0.15 mm) and pericystic fibrosis (CE/AE: >0.6/≤0.6 mm), striation of laminated layer (CE/AE: moderate-strong/weak), and number of cysts (CE/AE: ≤9/>9). Combined immunohistochemistry with mAbEm2G11 (E. multilocularis specific) and mAbEmG3 (reactive in AE and CE) was equally specific as and occasionally more sensitive than polymerase chain reaction. On the basis of these findings, we developed a diagnostic algorithm for the differential diagnosis of echinococcosis. In summary, we have not only identified the means to diagnose echinococcosis with greater certainty, but also defined morphologic criteria, which robustly discriminate between CE and AE. We expect our findings to improve echinococcosis diagnostics, especially of challenging cases, beneficially impacting the management of echinococcosis patients.