Int-Veen Isabell, Ehlis Ann-Christine, Fallgatter Andreas J, Rosenbaum David
Tübingen Center for Mental Health, University Hospital and Faculty of Medicine, University of Tübingen, Tübingen, Germany.
Front Psychol. 2024 Jun 5;15:1368390. doi: 10.3389/fpsyg.2024.1368390. eCollection 2024.
This study explores the stability of scores on psychometrically validated trait questionnaires over time. We illustrate potential pitfalls through a larger study that used the Ruminative Response Scale (RRS) to categorize participants prior to study inclusion into two groups based on their habitual tendency to ruminate. Surprisingly, when we re-administered the RRS at the start of an experimental session, significant score changes occurred, resulting in participants shifting between the two groups.
To address this, we modified our recruitment process, aiming to reduce careless responses, including an online RRS assessment a week before the lab appointment. We analyzed the different samples prior to and after changing the recruitment procedure, as well as the total sample regarding the psychometric properties of the RRS. We also explored various indices to identify and predict score changes due to careless responding; however, only a subgroup of participants was successfully identified.
Our findings suggest that Mahalanobis distances are effective for identifying substantial score changes, with baseline state rumination emerging as a marginally significant predictor.
We discuss the importance of conducting manipulation checks and offer practical implications for research involving psychometrically validated trait questionnaires.
本研究探讨了经过心理测量验证的特质问卷得分随时间的稳定性。我们通过一项更大规模的研究说明了潜在的陷阱，该研究使用反刍反应量表（RRS）在纳入研究之前根据参与者习惯性的反刍倾向将其分为两组。令人惊讶的是，当我们在实验环节开始时重新施测RRS时，出现了显著的得分变化，导致参与者在两组之间转换。
为了解决这个问题，我们修改了招募流程，旨在减少粗心作答，包括在实验室预约前一周进行在线RRS评估。我们分析了改变招募程序前后的不同样本，以及关于RRS心理测量特性的总样本。我们还探索了各种指标以识别和预测因粗心作答导致的得分变化；然而，仅成功识别出了一部分参与者。
我们的研究结果表明，马氏距离对于识别显著的得分变化是有效的，基线状态反刍成为一个边缘显著的预测因子。
我们讨论了进行操纵检查的重要性，并为涉及经过心理测量验证的特质问卷的研究提供了实际建议。
