Marketing Systems Group, Horsham, PA, United States.
Truth Initiative Schroeder Institute, Washington, DC, United States.
JMIR Public Health Surveill. 2024 Mar 7;10:e48186. doi: 10.2196/48186.
Increasingly, survey researchers rely on hybrid samples to improve coverage and increase the number of respondents by combining independent samples. For instance, it is possible to combine 2 probability samples with one relying on telephone and another on mail. More commonly, however, researchers are now supplementing probability samples with those from online panels that are less costly. Setting aside ad hoc approaches that are void of rigor, traditionally, the method of composite estimation has been used to blend results from different sample surveys. This means individual point estimates from different surveys are pooled together, 1 estimate at a time. Given that for a typical study many estimates must be produced, this piecemeal approach is computationally burdensome and subject to the inferential limitations of the individual surveys that are used in this process.
In this paper, we will provide a comprehensive review of the traditional method of composite estimation. Subsequently, the method of composite weighting is introduced, which is significantly more efficient, both computationally and inferentially when pooling data from multiple surveys. With the growing interest in hybrid sampling alternatives, we hope to offer an accessible methodology for improving the efficiency of inferences from such sample surveys without sacrificing rigor.
Specifically, we will illustrate why the many ad hoc procedures for blending survey data from multiple surveys are void of scientific integrity and subject to misleading inferences. Moreover, we will demonstrate how the traditional approach of composite estimation fails to offer a pragmatic and scalable solution in practice. By relying on theoretical and empirical justifications, in contrast, we will show how our proposed methodology of composite weighting is both scientifically sound and inferentially and computationally superior to the old method of composite estimation.
Using data from 3 large surveys that have relied on hybrid samples composed of probability-based and supplemental sample components from online panels, we illustrate that our proposed method of composite weighting is superior to the traditional method of composite estimation in 2 distinct ways. Computationally, it is vastly less demanding and hence more accessible for practitioners. Inferentially, it produces more efficient estimates with higher levels of external validity when pooling data from multiple surveys.
The new realities of the digital age have brought about a number of resilient challenges for survey researchers, which in turn have exposed some of the inefficiencies associated with the traditional methods this community has relied upon for decades. The resilience of such challenges suggests that piecemeal approaches that may have limited applicability or restricted accessibility will prove to be inadequate and transient. It is from this perspective that our proposed method of composite weighting has aimed to introduce a durable and accessible solution for hybrid sample surveys.
越来越多的调查研究人员依靠混合样本来提高覆盖范围并通过合并独立样本来增加受访者数量。例如，可以将依赖电话和邮件的两个概率样本组合起来。然而，更常见的是，研究人员现在正在用成本更低的在线面板样本来补充概率样本。除了没有严谨性的特定方法之外，传统上，复合估计方法一直被用于混合来自不同样本调查的结果。这意味着来自不同调查的单个点估计值被汇集在一起，一次一个估计值。鉴于典型研究中必须生成许多估计值，因此这种零碎的方法在计算上是繁琐的，并且受到用于该过程的各个调查的推断限制。
在本文中，我们将全面回顾传统的复合估计方法。随后，引入了复合加权方法，当从多个调查中汇集数据时，该方法在计算和推断上都更加高效。随着对混合抽样替代方案的兴趣日益浓厚，我们希望提供一种易于访问的方法来提高此类抽样调查推断的效率，而不会牺牲严谨性。
具体来说，我们将说明为什么从多个调查中混合调查数据的许多特定程序都缺乏科学完整性并且容易产生误导性推断。此外，我们将演示传统的复合估计方法在实践中如何无法提供实用且可扩展的解决方案。相比之下，我们将通过理论和经验论证，展示我们提出的复合加权方法如何在科学上是合理的，并且在推断和计算上都优于旧的复合估计方法。
使用来自 3 个大型调查的数据，这些调查都依赖于由概率样本和来自在线面板的补充样本组成的混合样本，我们表明，与传统的复合估计方法相比，我们提出的复合加权方法在两个方面具有优势。在计算方面，它的要求大大降低，因此对从业者来说更容易访问。在推断方面，当从多个调查中汇集数据时，它可以生成更有效的估计值，并且具有更高的外部有效性。
数字时代的新现实给调查研究人员带来了一些具有弹性的挑战，这反过来又暴露了该领域几十年来依赖的传统方法存在的一些效率低下的问题。这些挑战的弹性表明，可能具有有限适用性或受限可访问性的零碎方法将被证明是不足和短暂的。正是从这个角度出发，我们提出的复合加权方法旨在为混合样本调查引入一种持久且易于访问的解决方案。
