BACKGROUND AND OBJECTIVE

Response bias may affect the result of surveys with <100% response rate. We applied methods commonly used in meta-analysis to ascertain the extent to which response bias affects multiwave survey results.

METHODS

To test hypotheses of between-wave similarity, we used the Cochran-Armitage test for trends and the Q-test of heterogeneity across waves in a survey of 2,127 North American clinicians using six e-mail waves and one fax wave and achieving a response rate of 22%. We used the I2 statistic To quantify the extent of inconsistency in survey outcomes across waves not due to within-wave random error (i.e., inconsistency due to response bias).

RESULTS

With this survey, tests of heterogeneity and trend were not significant and I2 equaled 0%. These results suggest that the underlying responses did not differ across waves and thus strengthened the inference that response bias was not affecting the interpretation of the survey.

CONCLUSION

Researchers can use procedures that assess inconsistency in meta-analyses to evaluate the validity of a multiwave survey with a less than optimal response rate.