Use of design effects and sample weights in complex health survey data: a review of published articles using data from 3 commonly used adolescent health surveys.

OBJECTIVES

We assessed how frequently researchers reported the use of statistical techniques that take into account the complex sampling structure of survey data and sample weights in published peer-reviewed articles using data from 3 commonly used adolescent health surveys.

METHODS

We performed a systematic review of 1003 published empirical research articles from 1995 to 2010 that used data from the National Longitudinal Study of Adolescent Health (n=765), Monitoring the Future (n=146), or Youth Risk Behavior Surveillance System (n=92) indexed in ERIC, PsycINFO, PubMed, and Web of Science.

RESULTS

Across the data sources, 60% of articles reported accounting for design effects and 61% reported using sample weights. However, the frequency and clarity of reporting varied across databases, publication year, author affiliation with the data, and journal.

CONCLUSIONS

Given the statistical bias that occurs when design effects of complex data are not incorporated or sample weights are omitted, this study calls for improvement in the dissemination of research findings based on complex sample data. Authors, editors, and reviewers need to work together to improve the transparency of published findings using complex sample data.