Faculty of data and decision sciences, Technion-Israel Institute of Technology, Haifa 3200003, Israel.
Department of Psychology and the Neuroscience Institute, Princeton University, Princeton, NJ 08544, USA.
Cereb Cortex. 2023 Jun 8;33(12):7830-7842. doi: 10.1093/cercor/bhad082.
Word embedding representations have been shown to be effective in predicting human neural responses to lingual stimuli. While these representations are sensitive to the textual context, they lack the extratextual sources of context such as prior knowledge, thoughts, and beliefs, all of which constitute the listener's perspective. In this study, we propose conceptualizing the listeners' perspective as a source that induces changes in the embedding space. We relied on functional magnetic resonance imaging data collected by Yeshurun Y, Swanson S, Simony E, Chen J, Lazaridi C, Honey CJ, Hasson U. Same story, different story: the neural representation of interpretive frameworks. Psychol Sci. 2017:28(3):307-319, in which two groups of human listeners (n = 40) were listening to the same story but with different perspectives. Using a dedicated fine-tuning process, we created two modified versions of a word embedding space, corresponding to the two groups of listeners. We found that each transformed space was better fitted with neural responses of the corresponding group, and that the spatial distances between these spaces reflect both interpretational differences between the perspectives and the group-level neural differences. Together, our results demonstrate how aligning a continuous embedding space to a specific context can provide a novel way of modeling listeners' intrinsic perspectives.
词嵌入表示在预测人类对语言刺激的神经反应方面表现出色。虽然这些表示对文本上下文敏感,但它们缺乏语境的额外来源,如先前的知识、思想和信仰,所有这些都构成了听众的视角。在这项研究中,我们将听众的视角概念化为一个诱导嵌入空间变化的来源。我们依赖于 Yeshurun Y、Swanson S、Simony E、Chen J、Lazaridi C、Honey CJ 和 Hasson U 收集的功能磁共振成像数据,这些数据来自于两项研究:Same story, different story: the neural representation of interpretive frameworks. Psychol Sci. 2017:28(3):307-319。在这两项研究中,两组人类听众(n=40)都在听同一个故事,但视角不同。使用专门的微调过程,我们创建了两个对应于两组听众的词嵌入空间的修改版本。我们发现,每个转换后的空间都更适合相应组的神经反应,并且这些空间之间的空间距离反映了视角之间的解释差异和组级神经差异。总的来说,我们的结果表明,如何将连续的嵌入空间与特定的上下文对齐,可以为建模听众内在视角提供一种新方法。