Department of Computer Science and Information Engineering, National Taiwan University, Taipei, Taiwan.
Institute of Information Science, Academia Sinica, Taipei, Taiwan.
BMC Med Res Methodol. 2023 Jan 16;23(1):15. doi: 10.1186/s12874-023-01834-7.
Surveys are common research tools, and questionnaires revisions are a common occurrence in longitudinal studies. Revisions can, at times, introduce systematic shifts in measures of interest. We formulate that questionnaire revision are a stochastic process with transition matrices. Thus, revision shifts can be reduced by first estimating these transition matrices, which can be utilized in estimation of interested measures.
An ideal survey response model is defined by mapping between the true value of a participant's response to an interval in the grouped data type scale. A population completed surveys multiple times, as modeled with multiple stochastic process. This included stochastic processes related to true values and intervals. While multiple factors contribute to changes in survey responses, here, we explored the method that can mitigate the effects of questionnaire revision. We proposed the Version Alignment Method (VAM), a data preprocessing tool, which can separate the transitions according to revisions from all transitions via solving an optimization problem and using the revision-related transitions to remove the revision effect. To verify VAM, we used simulation data to study the estimation error and a real life MJ dataset containing large amounts of long-term questionnaire responses with several questionnaire revisions to study its feasibility.
We compared the difference of the annual average between consecutive years. Without adjustment, the difference is 0.593 when the revision occurred, while VAM brought it down to 0.115, where difference between years without revision was in the 0.005, 0.125 range. Furthermore, our method rendered the responses to the same set of intervals, thus comparing the relative frequency of items before and after revisions became possible. The average estimation error in L infinity was 0.0044 which occupied the 95% CI which was constructed by bootstrap analysis.
Questionnaire revisions can induce different response bias and information loss, thus causing inconsistencies in the estimated measures. Conventional methods can only partly remedy this issue. Our proposal, VAM, can estimate the aggregate difference of all revision-related systematic errors and can reduce the differences, thus reducing inconsistencies in the final estimations of longitudinal studies.
调查是常见的研究工具，在纵向研究中经常会对问卷进行修订。修订有时会导致感兴趣的测量指标系统地发生变化。我们将问卷修订视为具有转移矩阵的随机过程。因此，可以通过首先估计这些转移矩阵来减少修订带来的偏差，这些矩阵可以用于感兴趣的测量指标的估计。
定义了理想的调查响应模型，该模型通过将参与者对分组数据类型刻度中区间的真实值映射来实现。群体多次完成调查，如多个随机过程建模。这包括与真实值和区间相关的随机过程。虽然有多个因素导致调查响应发生变化，但在这里，我们探索了可以减轻问卷修订影响的方法。我们提出了版本对齐方法（VAM），这是一种数据预处理工具，可以通过解决优化问题并使用与修订相关的转换来分离根据修订的转换和所有转换，从而分离根据修订的转换。为了验证 VAM，我们使用模拟数据研究了估计误差，以及包含大量长期问卷响应的真实生活 MJ 数据集，该数据集进行了多次问卷修订，以研究其可行性。
我们比较了连续两年的年平均差异。在没有调整的情况下，修订发生时的差异为 0.593，而 VAM 将其降低到 0.115，而没有修订的年份之间的差异在 0.005 到 0.125 之间。此外，我们的方法使同一组区间的响应保持一致，因此可以比较修订前后项目的相对频率。L 无穷大的平均估计误差为 0.0044，该值位于通过自举分析构建的 95%置信区间内。
问卷修订会引起不同的响应偏差和信息丢失，从而导致估计指标的不一致。传统方法只能部分解决此问题。我们的提议 VAM 可以估计所有与修订相关的系统误差的总体差异，并减少差异，从而减少纵向研究最终估计中的不一致性。
