Tang Mengmeng, Chen Bingfei, Zhao Xiufeng, Zhao Lun
School of Foreign Languages, China University of Petroleum, Beijing, China.
Center of Language & Brain Research, Sichuan International Studies University, Chongqing, China.
Cogn Neurodyn. 2024 Aug;18(4):1743-1752. doi: 10.1007/s11571-023-10037-1. Epub 2023 Dec 13.
The current study investigated the neuro mechanisms of emoji processing as sentence predicate in written context. In the hybrid textuality which is more cognitively engaging, emojis in sentential intermediate positions were designed as either congruent or incongruent to the context. The results showed that incongruent words led to a robust N400 effect, while incongruent emojis only elicited the P600 effect. It implies that semantics and syntax of words can be separated while those of emojis seem to be integrated together. That is, when the meaning of the emoji is violated to the sentential context, its grammatical role cannot be well interpreted, especially when it is used as a key grammatical component in a sentence, such as the predicate. Thus, it shows that even though the meaning of emojis can be interpreted by readers, their syntactic and semantic functions cannot be clearly separated. In comparison with word processing, the larger amplitude with emojis in the time window of 350-500 ms shows more cognitive efforts in emoji semantic processing, possibly arising from the switch of modalities within the visual channel, that is, the multimodal cognitive load.
当前的研究探讨了在书面语境中，表情符号作为句子谓语进行加工的神经机制。在更具认知吸引力的混合文本性中，处于句子中间位置的表情符号被设计为与语境一致或不一致。结果表明，不一致的单词会导致强烈的N400效应，而不一致的表情符号仅引发P600效应。这意味着单词的语义和句法可以分离，而表情符号的语义和句法似乎是整合在一起的。也就是说，当表情符号的含义与句子语境相悖时，其语法作用就无法得到很好的解释，尤其是当它在句子中用作关键语法成分（如谓语）时。因此，这表明尽管读者能够解读表情符号的含义，但其句法和语义功能却无法清晰地分离。与单词加工相比，表情符号在350 - 500毫秒时间窗口内具有更大的波幅，这表明在表情符号语义加工中需要更多的认知努力，这可能源于视觉通道内模态的转换，即多模态认知负荷。
