Faculty of Humanities, Institute for Logic, Language and Computation, University of Amsterdam, Amsterdam, The Netherlands.
Department of Theory, Methods, and Statistics, Faculty of Psychology, Open Universiteit, Heerlen, The Netherlands.
Br J Soc Psychol. 2024 Apr;63(2):975-1002. doi: 10.1111/bjso.12688. Epub 2023 Nov 2.
The Implicit Association Test (IAT, Greenwald et al., J. Pers. Soc. Psychol., 74, 1998, 1464) is a popular instrument for measuring attitudes and (stereotypical) biases. Greenwald et al. (Behav. Res. Methods, 54, 2021, 1161) proposed a concrete method for validating IAT stimuli: appropriate stimuli should be familiar and easy to classify - translating to rapid (response times <800 ms) and accurate (error < 10%) participant responses. We conducted three analyses to explore the theoretical and practical utility of these proposed validation criteria. We first applied the proposed validation criteria to the data of 15 IATs that were available via Project Implicit. A bootstrap approach with 10,000 'experiments' of 100 participants showed that 5.85% of stimuli were reliably valid (i.e., we are more than 95% confident that a stimulus will also be valid in a new sample of 18- to 25-year-old US participants). Most stimuli (78.44%) could not be reliably validated, indicating a less than 5% certainty in the outcome of stimulus (in)validity for a new sample of participants. We then explored how stimulus validity differs across IATs. Results show that only some stimuli are consistently (in)valid. Most stimuli show between-IAT variances, which indicate that stimulus validity differs across IAT contexts. In the final analysis, we explored the effect of stimulus type (images, nouns, names, adjectives) on stimulus validity. Stimulus type was a significant predictor of stimulus validity. Although images attain the highest stimulus validity, raw data show large differences within stimulus types. Together, the results indicate a need for revised validation criteria. We finish with practical recommendations for stimulus selection and (post-hoc) stimulus validation.
内隐联想测验（IAT，Greenwald 等人，J. Pers. Soc. Psychol.，74，1998，1464）是一种用于测量态度和（刻板）偏见的流行工具。Greenwald 等人（Behav. Res. Methods，54，2021，1161）提出了一种验证 IAT 刺激物的具体方法：适当的刺激物应该是熟悉且易于分类的 - 这转化为参与者快速（反应时间<800ms）和准确（错误<10%）的反应。我们进行了三项分析，以探讨这些拟议验证标准的理论和实际效用。我们首先将提出的验证标准应用于可通过 Project Implicit 获得的 15 个 IAT 的数据。在 10,000 次 100 名参与者的“实验”的自举方法中，有 5.85%的刺激物被可靠地验证为有效（即，我们有超过 95%的信心，即一个刺激物在一个新的 18 至 25 岁的美国参与者样本中也将是有效的）。大多数刺激物（78.44%）无法可靠验证，这表明对于新的参与者样本，对刺激物（有效/无效）结果的确定性不到 5%。然后，我们探索了 IAT 之间刺激物有效性的差异。结果表明，只有一些刺激物是一致有效的。大多数刺激物表现出 IAT 之间的差异，这表明刺激物的有效性在不同的 IAT 环境中有所不同。在最后的分析中，我们探讨了刺激物类型（图像、名词、名称、形容词）对刺激物有效性的影响。刺激物类型是刺激物有效性的一个显著预测因素。尽管图像获得了最高的刺激物有效性，但原始数据显示出刺激物类型内的巨大差异。总而言之，结果表明需要修订验证标准。我们以刺激物选择和（事后）刺激物验证的实际建议结束。
