Some statistical implications of rounding in the College of American Pathologists Survey.

Quantitative measurements of constituent concentrations of Survey specimens are generally recorded to the nearest integer or to the nearest tenth. Questions have arisen as to whether this degree of rounding of measurements is appropriate when available laboratory equipment can determine results more precisely. Statisticians have studied the effects of rounding off on an variety of standard statistical procedures. In this report biases that may result for estimates of the mean and variance from several typical data sets obtained in the 1975 College of American Pathologists (CAP) Survey are examined. The study indicates that the amount of bias in the sample mean and variance is generally minimal.