Wagner James, Centeno Lena, Dulaney Richard, Edwards Brad, Suzer-Gurtekin Z Tuba, Coffey Stephanie
Research Professor in the University of Michigan's Survey Research Center (UM SRC), 4053 ISR, 426 Thompson St., Ann Arbor, MI 48104, USA.
Senior Study Director, with Westat, 1600 Research Blvd, Rockville, MD 20850, USA.
J Surv Stat Methodol. 2023 Aug 30;12(4):932-960. doi: 10.1093/jssam/smad028. eCollection 2024 Sep.
Survey design decisions are-by their very nature-tradeoffs between costs and errors. However, measuring costs is often difficult. Furthermore, surveys are growing more complex. Many surveys require that cost information be available to make decisions during data collection. These complexities create new challenges for monitoring and understanding survey costs. Often, survey cost information lags behind reporting of paradata. Furthermore, in some situations, the measurement of costs at the case level is difficult. Given the time lag in reporting cost information and the difficulty of assigning costs directly to cases, survey designers and managers have frequently turned to proxy indicators for cost. These proxy measures are often based upon level-of-effort paradata. An example of such a proxy cost indicator is the number of attempts per interview. Unfortunately, little is known about how accurately these proxy indicators actually mirror the true costs of the survey. In this article, we examine a set of these proxy indicators across several surveys with different designs, including different modes of interview. We examine the strength of correlation between these indicators and two different measures of costs-the total project cost and total interviewer hours. This article provides some initial evidence about the quality of these proxies as surrogates for the true costs using data from several different surveys with interviewer-administered modes (telephone, face to face) across three organizations (University of Michigan's Survey Research Center, Westat, US Census Bureau). We find that some indicators (total attempts, total contacts, total completes, sample size) are correlated (average correlation ∼0.60) with total costs across several surveys. These same indicators are strongly correlated (average correlation ∼0.82) with total interviewer hours. For survey components, three indicators (total attempts, sample size, and total miles) are strongly correlated with both total costs (average correlation ∼0.77) and with total interviewer hours (average correlation ∼0.86).
调查设计决策就其本质而言是成本与误差之间的权衡。然而，衡量成本往往很困难。此外，调查正变得越来越复杂。许多调查要求在数据收集过程中能够获取成本信息以做出决策。这些复杂性给监测和理解调查成本带来了新的挑战。通常，调查成本信息滞后于辅助数据的报告。此外，在某些情况下，在个案层面衡量成本很困难。鉴于成本信息报告存在时间滞后以及直接将成本分配到个案的困难，调查设计者和管理者经常求助于成本的替代指标。这些替代指标通常基于工作量辅助数据。这样一个替代成本指标的例子是每次访谈的尝试次数。不幸的是，对于这些替代指标实际能多准确地反映调查的真实成本知之甚少。在本文中，我们在几项具有不同设计（包括不同访谈方式）的调查中研究了一组这样的替代指标。我们研究了这些指标与两种不同成本衡量指标——项目总成本和总访员工时之间的相关强度。本文利用来自三个组织（密歇根大学调查研究中心、韦斯塔特公司、美国人口普查局）采用访员管理方式（电话、面对面）进行的几项不同调查的数据，提供了一些关于这些替代指标作为真实成本替代物质量的初步证据。我们发现，一些指标（总尝试次数、总接触次数、总完成次数、样本量）在几项调查中与总成本相关（平均相关性约为0.60）。这些相同的指标与总访员工时高度相关（平均相关性约为0.82）。对于调查组成部分，三个指标（总尝试次数、样本量和总里程）与总成本（平均相关性约为0.77）以及总访员工时（平均相关性约为0.86）都高度相关。
