Günther Fritz, Petilli Marco Alessandro, Vergallito Alessandra, Marelli Marco
University of Tübingen, Tübingen, Germany.
University of Milano-Bicocca, Milan, Italy.
Psychol Res. 2022 Nov;86(8):2512-2532. doi: 10.1007/s00426-020-01429-7.
Theories of grounded cognition assume that conceptual representations are grounded in sensorimotor experience. However, abstract concepts such as jealousy or childhood have no directly associated referents with which such sensorimotor experience can be made; therefore, the grounding of abstract concepts has long been a topic of debate. Here, we propose (a) that systematic relations exist between semantic representations learned from language on the one hand and perceptual experience on the other hand, (b) that these relations can be learned in a bottom-up fashion, and (c) that it is possible to extrapolate from this learning experience to predict expected perceptual representations for words even where direct experience is missing. To test this, we implement a data-driven computational model that is trained to map language-based representations (obtained from text corpora, representing language experience) onto vision-based representations (obtained from an image database, representing perceptual experience), and apply its mapping function onto language-based representations for abstract and concrete words outside the training set. In three experiments, we present participants with these words, accompanied by two images: the image predicted by the model and a random control image. Results show that participants' judgements were in line with model predictions even for the most abstract words. This preference was stronger for more concrete items and decreased for the more abstract ones. Taken together, our findings have substantial implications in support of the grounding of abstract words, suggesting that we can tap into our previous experience to create possible visual representation we don't have.
具身认知理论认为概念表征基于感觉运动经验。然而，诸如嫉妒或童年等抽象概念没有与之直接关联的可产生此类感觉运动经验的指称对象；因此，抽象概念的基础长期以来一直是一个争论的话题。在此，我们提出：（a）一方面从语言中学到的语义表征与另一方面的感知经验之间存在系统关系；（b）这些关系可以自下而上的方式习得；（c）即使在缺乏直接经验的情况下，也有可能从这种学习经验中进行推断，以预测单词的预期感知表征。为了对此进行测试，我们实现了一个数据驱动的计算模型，该模型经过训练，将基于语言的表征（从文本语料库中获得，代表语言经验）映射到基于视觉的表征（从图像数据库中获得，代表感知经验），并将其映射功能应用于训练集之外的抽象和具体单词的基于语言的表征。在三个实验中，我们向参与者呈现这些单词，并伴有两张图像：模型预测的图像和随机对照图像。结果表明，即使对于最抽象的单词，参与者的判断也与模型预测一致。对于更具体的项目，这种偏好更强，而对于更抽象的项目则有所下降。综上所述，我们的研究结果对支持抽象单词的基础具有重大意义，表明我们可以利用以往的经验来创建我们没有的可能视觉表征。
