Princeton Neuroscience Institute and Department of Psychology, Princeton University, Princeton, NJ 08544, USA.
Princeton Neuroscience Institute and Department of Psychology, Princeton University, Princeton, NJ 08544, USA; Department of Cognitive and Brain Sciences and Business School, Hebrew University, Jerusalem 9190501, Israel.
Neuron. 2024 Sep 25;112(18):3211-3222.e5. doi: 10.1016/j.neuron.2024.06.025. Epub 2024 Aug 2.
Effective communication hinges on a mutual understanding of word meaning in different contexts. We recorded brain activity using electrocorticography during spontaneous, face-to-face conversations in five pairs of epilepsy patients. We developed a model-based coupling framework that aligns brain activity in both speaker and listener to a shared embedding space from a large language model (LLM). The context-sensitive LLM embeddings allow us to track the exchange of linguistic information, word by word, from one brain to another in natural conversations. Linguistic content emerges in the speaker's brain before word articulation and rapidly re-emerges in the listener's brain after word articulation. The contextual embeddings better capture word-by-word neural alignment between speaker and listener than syntactic and articulatory models. Our findings indicate that the contextual embeddings learned by LLMs can serve as an explicit numerical model of the shared, context-rich meaning space humans use to communicate their thoughts to one another.
有效的沟通取决于对不同语境下单词含义的相互理解。我们在五对癫痫患者的自发性面对面对话中使用脑电图记录大脑活动。我们开发了一种基于模型的耦合框架,该框架将说话者和听话者的大脑活动与来自大型语言模型 (LLM) 的共享嵌入空间对齐。上下文敏感的 LLM 嵌入允许我们从一个大脑到另一个大脑,逐字逐句地跟踪自然对话中的语言信息交换。语言内容在说话者的大脑中先于单词发音出现,并在单词发音后迅速重新出现在听者的大脑中。上下文嵌入比句法和发音模型更好地捕捉说话者和听者之间逐字的神经对齐。我们的发现表明,LLM 学习的上下文嵌入可以作为人类用来相互交流思想的共享、丰富上下文意义空间的显式数字模型。