West Brady T, Li Dan
Survey Methodology Program, Survey Research Center, Institute for Social Research, University of Michigan-Ann Arbor, Ann Arbor, MI, USA.
ALG, A TRUECar Company, Los Angeles, CA, USA.
Sociol Methods Res. 2019 Aug;48(3):485-533. doi: 10.1177/0049124117729698. Epub 2017 Sep 25.
In face-to-face surveys, interviewer observations are a cost-effective source of paradata for nonresponse adjustment of survey estimates and responsive survey designs. Unfortunately, recent studies have suggested that the accuracy of these observations can vary substantially among interviewers, even after controlling for household-, area-, and interviewer-level characteristics, limiting their utility. No study has identified sources of this unexplained variance in observation accuracy. Motivated by theoretical expectations from the observer bias literature, this study analyzed more than 45,000 open-ended justifications provided by interviewers in the U.S. National Survey of Family Growth (NSFG) for their observations on two key features of all sampled NSFG households: presence of children and expected probability of household response. The study finds that variability among interviewers in the cues used to record these observations (evident from the open-ended justifications) explains much of the previously unexplained variance in observation accuracy.
在面对面调查中，访员观察是一种具有成本效益的辅助数据来源，可用于对调查估计值进行无回答调整以及设计响应式调查。遗憾的是，最近的研究表明，即使在控制了家庭、地区和访员层面的特征之后，这些观察的准确性在访员之间仍可能存在很大差异，从而限制了它们的效用。尚无研究确定这种观察准确性中无法解释的差异来源。受观察者偏差文献的理论预期启发，本研究分析了美国全国家庭成长调查（NSFG）中访员提供的超过45000条开放式理由，这些理由涉及他们对NSFG所有抽样家庭的两个关键特征的观察：是否有孩子以及家庭回应的预期概率。研究发现，访员在记录这些观察时所使用线索的差异（从开放式理由中可见）解释了此前观察准确性中许多无法解释的差异。
