Survey and Qualitative Methods Core, Division of Population Sciences, Dana-Farber Cancer Institute, Boston, MA, United States.
Westat, Rockville, MD, United States.
J Med Internet Res. 2021 Dec 20;23(12):e31545. doi: 10.2196/31545.
Young adults' early adoption of new cell phone technologies have created challenges to survey recruitment but offer opportunities to combine random digit dialing (RDD) sampling with web mode data collection. The National Young Adult Health Survey was designed to test the feasibility of this methodology.
In this study, we compared response rates across the telephone mode and web mode, assessed sample representativeness, examined design effects (DEFFs), and compared cigarette smoking prevalence to a gold standard national survey.
We conducted a survey experiment where the sampling frame was randomized to single-mode telephone interviews, telephone-to-web sequential mixed mode, and single-mode web survey. A total of 831 respondents aged 18 to 34 years were recruited via RDD at baseline. A soft launch was conducted prior to main launch. We compared the web mode to the telephone modes (ie, single-mode and mixed mode) at wave 1 based on the American Association for Public Opinion Research response rate 3 for screening and extended surveys. Base-weighted demographic distributions were compared to the American Community Survey. The sample was calibrated to the US Census Bureau's American Community Survey to calculate DEFFs and to compare cigarette smoking prevalence to the National Health Interview Survey. Prevalence estimates are estimated with sampling weights and are presented with unweighted sample sizes. Consistency of estimates was judged by 95% CI.
The American Association for Public Opinion Research response rate 3 was higher in the telephone mode than in the web mode (24% and 30% vs 6.1% and 12.5%, for soft launch and main launch, respectively), which was reflected in response rate 3 for screening and extended surveys. During the soft launch, the extended survey and eligibility rate were low for respondents pushed to the web mode. To boost productivity and survey completes for the web condition, the main launch used cell phone numbers from the sampling frame where the sample vendor matched the number to auxiliary data, which suggested that the number likely belonged to an adult in the target age range. This increased the eligibility rate, but the screener response rate was lower. Compared to population distribution from the US Census Bureau, the telephone mode overrepresented men (57.1% [unweighted n=412] vs 50.9%) and those enrolled in college (40.3% [unweighted n=269] vs 23.8%); it also underrepresented those with a Bachelor of Arts or Science (34.4% [unweighted n=239] vs 55%). The web mode overrepresented White, non-Latinos (70.7% [unweighted n=90] vs 54.4%) and those with some college education (30.4% [unweighted n=40] vs 7.6%); it also underrepresented Latinos (13.6% [unweighted n=20] vs 20.7%) and those with a high school or General Education Development diploma (15.3% [unweighted n=20] vs 29.3%). The DEFF measure was 1.28 (subpopulation range 0.96-1.93). The National Young Adult Health Survey cigarette smoking prevalence was consistent with the National Health Interview Survey overall (15%, CI 12.4%-18% [unweighted 149/831] vs 13.5%, CI 12.3%-14.7% [unweighted 823/5552]), with notable deviation among 18- to 24-year-olds (15.6%, CI 11.3%-22.2% [unweighted 51/337] vs 8.7%, CI 7.1%-10.6% [unweighted 167/1647]), and those with education levels lower than Bachelor of Arts or Science (24%, CI 19.3%-29.4% [unweighted 123/524] vs 17.1%, CI 15.6%-18.7% [unweighted 690/3493]).
RDD sampling for a web survey is not feasible for young adults due to its low response rate. However, combining this methodology with RDD telephone surveys may have a great potential for including media and collecting autophotographic data in population surveys.
年轻人对新手机技术的早期采用给调查招募带来了挑战，但也为结合随机数字拨号 (RDD) 抽样和网络模式数据收集创造了机会。国家青年健康调查旨在测试这种方法的可行性。
本研究比较了电话模式和网络模式的应答率，评估了样本代表性，检查了设计效果 (DEFFs)，并将吸烟流行率与黄金标准的全国性调查进行了比较。
我们进行了一项调查实验，抽样框架随机分配给单模式电话访谈、电话到网络顺序混合模式和单模式网络调查。共有 831 名 18 至 34 岁的受访者通过 RDD 在基线时进行了招募。在主启动之前进行了软启动。我们根据美国舆论研究协会的应答率 3，在第 1 波比较了网络模式与电话模式（即单模式和混合模式），以筛查和扩展调查。基于美国社区调查的加权人口分布进行了比较。该样本经过校准，与美国人口普查局的美国社区调查相匹配，以计算 DEFFs，并将吸烟流行率与全国健康访谈调查进行比较。使用抽样权重估计流行率估计值，并呈现未加权样本量。通过 95%CI 判断估计值的一致性。
电话模式的应答率 3 高于网络模式（软启动和主启动时分别为 24%和 30%，而分别为 6.1%和 12.5%），这反映在筛查和扩展调查的应答率 3 中。在软启动期间，推送到网络模式的受访者的扩展调查和合格率较低。为了提高网络条件下的生产力和调查完成率，主启动使用了抽样框架中的手机号码，样本供应商将号码与辅助数据匹配，这表明该号码可能属于目标年龄范围内的成年人。这提高了合格率，但筛检应答率较低。与美国人口普查局的人口分布相比，电话模式过高地代表了男性（57.1%[未加权 n=412] vs 50.9%）和那些上大学的人（40.3%[未加权 n=269] vs 23.8%）；它也过低地代表了那些拥有文科学士或理科学士的人（34.4%[未加权 n=239] vs 55%）。网络模式过高地代表了白人非拉丁裔（70.7%[未加权 n=90] vs 54.4%）和那些有一定大学教育的人（30.4%[未加权 n=40] vs 7.6%）；它也过低地代表了拉丁裔（13.6%[未加权 n=20] vs 20.7%）和那些拥有高中或普通教育发展文凭的人（15.3%[未加权 n=20] vs 29.3%）。DEFF 度量值为 1.28（子群体范围为 0.96-1.93）。国家青年健康调查的吸烟流行率与全国健康访谈调查总体一致（15%，置信区间 12.4%-18%[未加权 149/831] vs 13.5%，置信区间 12.3%-14.7%[未加权 823/5552]），但在 18 至 24 岁的人群中存在显著差异（15.6%，置信区间 11.3%-22.2%[未加权 51/337] vs 8.7%，置信区间 7.1%-10.6%[未加权 167/1647]），以及教育程度低于文科学士或理科学士的人群（24%，置信区间 19.3%-29.4%[未加权 123/524] vs 17.1%，置信区间 15.6%-18.7%[未加权 690/3493]）。
由于年轻人的网络调查应答率较低，因此使用 RDD 抽样进行网络调查是不可行的。但是，将这种方法与 RDD 电话调查相结合，可能有很大的潜力在人群调查中纳入媒体和收集自拍照数据。
