Killeen's (2005) p rep coefficient: logical and mathematical problems.

In his article, "An alternative to null-hypothesis significance tests," Killeen (2005) urged the discipline to abandon the practice of p obs-based null hypothesis testing and to quantify the signal-to-noise characteristics of experimental outcomes with replication probabilities. He described the coefficient that he invented, prep, as the probability of obtaining "an effect of the same sign as that found in an original experiment" (Killeen, 2005, p. 346). The journal Psychological Science quickly came to encourage researchers to employ prep, rather than p obs, in the reporting of their experimental findings. In the current article, we (a) establish that Killeen's derivation of prep contains an error, the result of which is that prep is not, in fact, the probability that Killeen set out to derive; (b) establish that prep is not a replication probability of any kind but, rather, is a quasi-power coefficient; and (c) suggest that Killeen has mischaracterized both the relationship between replication probabilities and statistical inference, and the kinds of claims that are licensed by knowledge of the value assumed by the replication probability that he attempted to derive.