Validation of probabilistic predictions.

Current advances in high-speed computing and increased availability of statistical software have led to widespread use of statistical methods for the development of computerized protocols predictive of binary health outcomes. If these predictive algorithms are to be used in settings other than those for which they were developed, e.g., applied in a different geographic setting or extrapolated for use in a slightly different population, then they should be carefully validated to ensure appropriate application. Miller et al. (Stat Med. 1991) provided a comprehensive methodology for external validation of logistic prediction models, and applied these methods in a temporal validation setting. In this article, the authors emphasize how these methods can be applied to general forms of probabilistic predictions and provide several SAS macros for computation of the desired statistics.