School of Education, University of North Carolina at Chapel Hill, NC, USA.
Department of Mental Health, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA.
Eval Rev. 2020 Feb;44(1):84-108. doi: 10.1177/0193841X20938497.
Many studies in psychological and educational research aim to estimate population average treatment effects (PATE) using data from large complex survey samples, and many of these studies use propensity score methods. Recent advances have investigated how to incorporate survey weights with propensity score methods. However, to this point, that work had not been well summarized, and it was not clear how much difference the different PATE estimation methods would make empirically.
The purpose of this study is to systematically summarize the appropriate use of survey weights in propensity score analysis of complex survey data and use a case study to empirically compare the PATE estimates using multiple analysis methods that include ordinary least squares regression, weighted least squares regression, and various propensity score applications.
We first summarize various propensity score methods that handle survey weights. We then demonstrate the performance of various analysis methods using a nationally representative data set, the Early Childhood Longitudinal Study-Kindergarten to estimate the effects of preschool on children's academic achievement. The correspondence of the results was evaluated using multiple criteria.
It is important for researchers to think carefully about their estimand of interest and use methods appropriate for that estimand. If interest is in drawing inferences to the survey target population, it is important to take the survey weights into account, particularly in the outcome analysis stage for estimating the PATE. The case study shows, however, not much difference among various analysis methods in one applied example.
许多心理和教育研究旨在使用大型复杂调查样本中的数据来估计总体平均治疗效果（PATE），其中许多研究使用倾向评分方法。最近的研究探讨了如何将调查权重与倾向评分方法结合使用。然而，到目前为止，这项工作尚未得到很好的总结，也不清楚不同的 PATE 估计方法在实践中会有多大差异。
本研究旨在系统总结在复杂调查数据的倾向评分分析中使用调查权重的适当方法，并通过一个案例研究，使用包括普通最小二乘法回归、加权最小二乘法回归和各种倾向评分应用在内的多种分析方法，从实证上比较 PATE 估计值。
我们首先总结了各种处理调查权重的倾向评分方法。然后，我们使用一个全国代表性数据集——早期儿童纵向研究-幼儿园，来演示各种分析方法的性能，以估计学前教育对儿童学业成绩的影响。使用多种标准评估结果的一致性。
研究人员应仔细考虑其感兴趣的估计量，并使用适合该估计量的方法。如果研究目的是对调查目标人群进行推断，那么在进行估计 PATE 的结果分析阶段时，考虑调查权重尤为重要。然而，在一个应用实例中，各种分析方法之间并没有太大的差异。
