Calculating the probability of rare events: why settle for an approximation?

OBJECTIVE

Health services researchers often need to compute the probability of observing a certain number of events when only a few such events are expected. Our objective is to show that the standard approaches (Poisson, binomial, and normal approximations) are inappropriate in such instances, and to suggest an alternative.

DATA SOURCES

Patients undergoing cholecystectomy (34,234) in 465 California hospitals in 1983 are used to demonstrate the biases arising from various methods of calculating the probability of observing a given number of deaths in each hospital. Similar data from other procedures and diagnoses with lower and higher mortality rates are also used for illustration.

STUDY DESIGN

The computational methods to derive probabilities using the Poisson, normal, simulation, and exact probabilities are discussed. Using a previously developed risk factor model, the probability of observing the actual number of deaths (or more) is calculated given the expectation of death for each patient in each hospital. Results for the four methods are compared, showing the types of random and systematic errors in the Poisson, normal, and simulation approaches.

DATA COLLECTION

Routinely collected hospital discharge abstract data were provided by the California Office of Statewide Planning and Development.

PRINCIPAL FINDINGS

The Poisson and normal approximations are often biased substantially in calculating upper-tail p-values, especially when the expected number of adverse outcomes is less than five. Simulations allow unbiased calculations, and the degree of random error can be made arbitrarily small given enough trials. Exact calculations using a simple recursive algorithm can be done very efficiently on either a mainframe or personal computer. For example, the whole set of cholecystectomy patients can be assessed in less than 90 seconds on a Macintosh.

CONCLUSIONS

Calculating the probability of observing a small number of events using standard approaches may result in substantial errors. The availability of a simple and inexpensive method of calculating these probabilities exactly can avoid these errors.