FMP Consulting, 2900 South Quincy Street, Suite 200, Arlington, VA, 22206, USA.
Behav Res Methods. 2024 Feb;56(2):577-599. doi: 10.3758/s13428-023-02074-9. Epub 2023 Feb 3.
It is common to model responses to surveys within latent variable frameworks (e.g., item response theory [IRT], confirmatory factor analysis [CFA]) and use model fit indices to evaluate model-data congruence. Unfortunately, research shows that people occasionally engage in careless responding (CR) when completing online surveys. While CR has the potential to negatively impact model fit, this issue has not been systematically explored. To better understand the CR-fit linkage, two studies were conducted. In study 1, participants' response behaviors were experimentally shaped and used to embed aspects of a comprehensive simulation (study 2) with empirically informed data. For this simulation, 144 unique conditions (which varied the sample size, number of items, CR prevalence, CR severity, and CR type), two latent variable models (IRT, CFA), and six model fit indices (χ, RMSEA, SRMSR [CFA] and M2, RMSEA, SRMSR [IRT]), were examined. The results indicated that CR deteriorates model fit under most circumstances, though these effects are nuanced, variable, and contingent on many factors. These findings can be leveraged by researchers and practitioners to improve survey methods, obtain more accurate survey results, develop more precise theories, and enable more justifiable data-driven decisions.
在潜在变量框架内（例如，项目反应理论 [IRT]、验证性因素分析 [CFA]）对调查反应进行建模并使用模型拟合指数来评估模型与数据的一致性是很常见的。不幸的是，研究表明，人们在完成在线调查时偶尔会出现粗心作答（CR）的情况。虽然 CR 有可能对模型拟合产生负面影响，但这个问题尚未得到系统的探讨。为了更好地理解 CR 与拟合的关系，进行了两项研究。在研究 1 中，参与者的反应行为是通过实验塑造的，并用于嵌入具有实证依据的数据的综合模拟（研究 2）的各个方面。对于这个模拟，有 144 个独特的条件（这些条件改变了样本量、项目数量、CR 发生率、CR 严重程度和 CR 类型）、两个潜在变量模型（IRT、CFA）和六个模型拟合指数（χ、RMSEA、SRMSR [CFA]和 M2、RMSEA、SRMSR [IRT]）进行了检查。结果表明，在大多数情况下，CR 会降低模型拟合度，尽管这些影响是细微的、多变的，并且取决于许多因素。这些发现可以为研究人员和从业者提供帮助，以改进调查方法、获得更准确的调查结果、发展更精确的理论，并做出更合理的数据驱动决策。
