P(rep) misestimates the probability of replication.

The probability of "replication," P(rep), has been proposed as a means of identifying replicable and reliable effects in the psychological sciences. We conduct a basic test of P(rep) that reveals that it misestimates the true probability of replication, especially for small effects. We show how these general problems with P(rep) play out in practice, when it is applied to predict the replicability of observed effects over a series of experiments. Our results show that, over any plausible series of experiments, the true probabilities of replication will be very different from those predicted by P(rep). We discuss some basic problems in the formulation of P(rep) that are responsible for its poor performance, and conclude that P(rep) is not a useful statistic for psychological science.