Tipton Elizabeth, Peck Laura R
1 Department of Human Development, Teachers College, Columbia University, New York, NY, USA.
2 Social and Economic Policy, Abt Associates Inc., Bethesda, MD, USA.
Eval Rev. 2017 Aug;41(4):326-356. doi: 10.1177/0193841X16655656. Epub 2016 Jul 29.
Large-scale randomized experiments are important for determining how policy interventions change average outcomes. Researchers have begun developing methods to improve the external validity of these experiments. One new approach is a balanced sampling method for site selection, which does not require random sampling and takes into account the practicalities of site recruitment including high nonresponse.
The goal of balanced sampling is to develop a strategic sample selection plan that results in a sample that is compositionally similar to a well-defined inference population. To do so, a population frame is created and then divided into strata, which "focuses" recruiters on specific subpopulations. Units within these strata are then ranked, thus identifying "replacements" similar to sites that can be recruited when the ideal site refuses to participate in the experiment.
In this article, we consider how a balanced sample strategic site selection method might be implemented in a welfare policy evaluation.
We find that simply developing a population frame can be challenging, with three possible and reasonable options arising in the welfare policy arena. Using relevant study-specific contextual variables, we craft a recruitment plan that considers nonresponse.
大规模随机试验对于确定政策干预如何改变平均结果至关重要。研究人员已开始开发方法以提高这些试验的外部效度。一种新方法是用于地点选择的平衡抽样方法，该方法不需要随机抽样，并考虑到包括高无应答率在内的地点招募实际情况。
平衡抽样的目标是制定一个战略样本选择计划，该计划能产生一个在构成上与明确界定的推断总体相似的样本。为此，要创建一个总体框架，然后将其划分为不同层次，这使招募人员“聚焦”于特定亚群体。然后对这些层次内的单位进行排名，从而识别出与理想地点拒绝参与试验时可招募的地点相似的“替代地点”。
在本文中，我们考虑如何在福利政策评估中实施平衡样本战略地点选择方法。
我们发现，仅仅创建一个总体框架可能具有挑战性，在福利政策领域会出现三种可能且合理的选择。利用相关的特定研究背景变量，我们制定了一个考虑无应答情况的招募计划。
