Ong Ai Rene, Schultz Robert, Sinozich Sofi, West Brady T, Wagner James, Sinibaldi Jennifer, Finamore John
Institute for Social Research, University of Michigan, 426 Thompson St, Ann Arbor, MI 48104.
University of Maryland, College Park, MD 20742.
J Off Stat. 2022 Dec;38(4):1235-1251. doi: 10.2478/jos-2022-0052. Epub 2022 Dec 1.
Large-scale, nationally representative surveys serve many vital functions, but these surveys are often long and burdensome for respondents. Cutting survey length can help to reduce respondent burden and may improve data quality but removing items from these surveys is not a trivial matter. We propose a method to empirically assess item importance and associated burden in national surveys and guide this decision-making process using different research products produced from such surveys. This method is demonstrated using the Survey of Doctorate Recipients (SDR), a biennial survey administered to individuals with a Science, Engineering, and Health doctorate. We used three main sources of information on the SDR variables: 1) a bibliography of documents using the SDR data, 2) the SDR website that allows users to download summary data, and 3) web timing paradata and break-off rates. The bibliography was coded for SDR variable usage and citation counts. Putting this information together, we identified 35 unused items (17% of the survey) by any of these sources and found that the most burdensome items are highly important. We conclude with general recommendations for those hoping to employ similar methodologies in the future.
大规模、具有全国代表性的调查发挥着许多至关重要的作用，但这些调查对受访者来说往往冗长且繁重。缩短调查长度有助于减轻受访者负担，可能还会提高数据质量，但从这些调查中剔除项目并非易事。我们提出一种方法，用于实证评估全国性调查中项目的重要性及相关负担，并利用此类调查产生的不同研究成果来指导这一决策过程。使用对拥有科学、工程和健康领域博士学位者进行的两年一次的博士学位获得者调查（SDR）来演示该方法。我们使用了关于SDR变量的三个主要信息来源：1）使用SDR数据的文献目录，2）允许用户下载汇总数据的SDR网站，以及3）网络时间辅助数据和中断率。对文献目录进行SDR变量使用情况和引用次数编码。综合这些信息，我们通过这些来源中的任何一个确定了35个未使用的项目（占调查的17%），并发现最繁重的项目非常重要。最后，我们为那些希望在未来采用类似方法的人提供了一般性建议。
