Assessing environmentally significant effects: a better strength-of-evidence than a single P value?

Interpreting a P value from a traditional nil hypothesis test as a strength-of-evidence for the existence of an environmentally important difference between two populations of continuous variables (e.g. a chemical concentration) has become commonplace. Yet, there is substantial literature, in many disciplines, that faults this practice. In particular, the hypothesis tested is virtually guaranteed to be false, with the result that P depends far too heavily on the number of samples collected (the 'sample size'). The end result is a swinging burden-of-proof (permissive at low sample size but precautionary at large sample size). We propose that these tests be reinterpreted as direction detectors (as has been proposed by others, starting from 1960) and that the test's procedure be performed simultaneously with two types of equivalence tests (one testing that the difference that does exist is contained within an interval of indifference, the other testing that it is beyond that interval-also known as bioequivalence testing). This gives rise to a strength-of-evidence procedure that lends itself to a simple confidence interval interpretation. It is accompanied by a strength-of-evidence matrix that has many desirable features: not only a strong/moderate/dubious/weak categorisation of the results, but also recommendations about the desirability of collecting further data to strengthen findings.

Responses of Tuatara (Sphenodon punctatus) to removal of introduced Pacific rats from islands.

Invasive mammalian predators such as rats are now widespread on islands, but hypotheses about their effects have rarely been tested. Circumstantial evidence from New Zealand indicates that, when introduced to islands, Pacific rats (Rattus exulans) have negative effects on endemic plants, invertebrates, birds, and reptiles, including the tuatara (Sphenodon punctatus). We tested the effects of Pacific rats on tuatara by comparing the demographic structure and body condition of tuatara populations on three islands before and after removal of rats and on a fourth island where rats remained. In the presence of rats, juvenile tuatara constituted on average 0-5% of the sample tuatara populations. When Pacific rats were removed after at least 200 years' occupancy, the proportion of juvenile tuatara increased 3.5- to 17-fold and body condition of adult males and females also improved (sometimes dramatically). We predict that, unless Pacific rats are removed from Taranga Island, the tuatara population will collapse because of low population density and the lack of juvenile recruitment. Our results demonstrate that when invasive species exert subtle effects on recruitment and body condition, the effects on populations of long-lived endemic species may only become apparent long after the invasion.