Tourangeau Roger, Yan Ting, Sun Hanyu
Westat, MD, USA.
J Surv Stat Methodol. 2020 Nov;8(5):903-931. doi: 10.1093/jssam/smz034. Epub 2019 Oct 3.
Using reinterview data from the PATH Reliability and Validity (PATH-RV) study, we examine the characteristics of questions and respondents that predict the reliability of the answers. In the PATH-RV study, 524 respondents completed an interview twice, five to twenty-four days apart. We coded a number of question characteristics and used them to predict the gross discrepancy rates (GDRs) and kappas for each question. We also investigated respondent characteristics associated with reliability. Finally, we fitted cross-classified models that simultaneously examined a range of respondent and question characteristics. Although the different models yielded somewhat different conclusions, in general factual questions (especially demographic questions), shorter questions, questions that did not use scales, those with fewer response options, and those that asked about a noncentral topic produced more reliable answers than attitudinal questions, longer questions, questions using ordinal scales, those with more response options, and those asking about a central topic. One surprising finding was that items raising potential social desirability concerns yielded more reliable answers than items that did not raise such concerns. The respondent-level models and cross-classified models indicated that five adult respondent characteristics were associated with giving the same answer in both interviews-education, the Big Five trait of conscientiousness, tobacco use, sex, and income. Hispanic youths and non-Hispanic black youths were less likely to give the same answer in both interviews. The cross-classified model also found that more words were associated with less reliable answers. The results are mostly consistent with earlier findings but are nonetheless important because they are much less model-dependent than the earlier work. In addition, this study is the first to incorporate such personality traits as needed for cognition and the Big Five personality factors and to examine the relationships among reliability, item nonresponse, and response latency.
利用“PATH可靠性与有效性（PATH-RV）研究”中的再次访谈数据，我们考察了能够预测答案可靠性的问题及受访者的特征。在PATH-RV研究中，524名受访者进行了两次访谈，两次访谈间隔五至二十四天。我们对若干问题特征进行了编码，并利用这些特征预测每个问题的总差异率（GDR）和卡帕值。我们还调查了与可靠性相关的受访者特征。最后，我们拟合了交叉分类模型，同时考察了一系列受访者和问题特征。尽管不同模型得出的结论略有不同，但总体而言，事实性问题（尤其是人口统计学问题）、较短的问题、未使用量表的问题、答案选项较少的问题以及询问非核心主题的问题，比态度性问题、较长的问题、使用顺序量表的问题、答案选项较多的问题以及询问核心主题的问题，能产生更可靠的答案。一个令人惊讶的发现是，引发潜在社会期望性问题的项目比未引发此类问题的项目能产生更可靠的答案。受访者层面的模型和交叉分类模型表明，有五个成年受访者特征与在两次访谈中给出相同答案相关——教育程度、尽责性这一“大五”人格特质、烟草使用情况、性别和收入。西班牙裔青年和非西班牙裔黑人青年在两次访谈中给出相同答案的可能性较小。交叉分类模型还发现，词汇量越多，答案的可靠性越低。研究结果大多与早期研究结果一致，但仍然很重要，因为它们比早期研究对模型的依赖程度要低得多。此外，本研究首次纳入了认知所需的此类人格特质以及“大五”人格因素，并考察了可靠性、项目无应答和应答潜伏期之间的关系。
