Suppr超能文献

头部以上的含义:来自漫画和表情符号的组合视觉形态。

Meaning above (and in) the head: Combinatorial visual morphology from comics and emoji.

机构信息

Department of Communication and Cognition, Tilburg School of Humanities and Digital Sciences, Tilburg Center for Cognition and Communication, Tilburg University, P.O. Box 90153, 5000, Tilburg, LE, Netherlands.

Department of Psychology, University of Essex, Colchester, UK.

出版信息

Mem Cognit. 2022 Oct;50(7):1381-1398. doi: 10.3758/s13421-022-01294-2. Epub 2022 Mar 2.

Abstract

Compositionality is a primary feature of language, but graphics can also create combinatorial meaning, like with items above faces (e.g., lightbulbs to mean inspiration). We posit that these "upfixes" (i.e., upwards affixes) involve a productive schema enabling both stored and novel face-upfix dyads. In two experiments, participants viewed either conventional (e.g., lightbulb) or unconventional (e.g., clover-leaves) upfixes with faces which either matched (e.g., lightbulb/smile) or mismatched (e.g., lightbulb/frown). In Experiment 1, matching dyads sponsored higher comprehensibility ratings and faster response times, modulated by conventionality. In Experiment 2, event-related brain potentials (ERPs) revealed conventional upfixes, regardless of matching, evoked larger N250s, indicating perceptual expertise, but mismatching and unconventional dyads elicited larger semantic processing costs (N400) than conventional-matching dyads. Yet mismatches evoked a late negativity, suggesting congruent novel dyads remained construable compared with violations. These results support that combinatorial graphics involve a constrained productive schema, similar to the lexicon of language.

摘要

组合性是语言的主要特征,但图形也可以创造组合意义,例如在面部上方的物品(例如,灯泡表示灵感)。我们假设这些“上缀”(即向上的缀词)涉及一种生产性模式,既能存储又能创造新颖的面向上缀对。在两项实验中,参与者观看了带有面孔的常规(例如,灯泡)或非常规(例如,三叶草叶)上缀,这些面孔要么匹配(例如,灯泡/微笑),要么不匹配(例如,灯泡/皱眉)。在实验 1 中,匹配的对偶对赞助了更高的理解度评分和更快的反应时间,这受到常规性的调节。在实验 2 中,事件相关的脑电位(ERPs)显示,无论匹配与否,常规的上缀都会引起更大的 N250,表明具有感知专业知识,但不匹配和非常规的对偶对比常规匹配的对偶对引起更大的语义处理成本(N400)。然而,不匹配会引起晚期负性,表明与违反相比,一致的新对偶对仍然可以构建。这些结果支持组合图形涉及一种受约束的生产性模式,类似于语言的词汇。

https://cdn.ncbi.nlm.nih.gov/pmc/blobs/500c/9508049/a19cc0aa8ce1/13421_2022_1294_Fig1_HTML.jpg

相似文献

1
Meaning above (and in) the head: Combinatorial visual morphology from comics and emoji.
Mem Cognit. 2022 Oct;50(7):1381-1398. doi: 10.3758/s13421-022-01294-2. Epub 2022 Mar 2.
2
Morphosyntax can modulate the N400 component: event related potentials to gender-marked post-nominal adjectives.
Neuroimage. 2014 May 1;91:262-72. doi: 10.1016/j.neuroimage.2013.09.077. Epub 2014 Jan 23.
4
Distinct neural responses to social and semantic violations: An N400 study.
Int J Psychophysiol. 2019 Mar;137:72-81. doi: 10.1016/j.ijpsycho.2018.12.006. Epub 2018 Dec 14.
5
Immediate integration of novel meanings: N400 support for an embodied view of language comprehension.
Brain Res. 2007 Dec 5;1183:109-23. doi: 10.1016/j.brainres.2007.09.014. Epub 2007 Sep 19.
6
Semantic processing of unattended meaning is modulated by additional task load: evidence from electrophysiology.
Brain Res Cogn Brain Res. 2005 Aug;24(3):500-12. doi: 10.1016/j.cogbrainres.2005.03.001. Epub 2005 Apr 26.
7
Syntactic gender and semantic expectancy: ERPs reveal early autonomy and late interaction.
J Cogn Neurosci. 2000 Jul;12(4):556-68. doi: 10.1162/089892900562336.
8
Brain potentials to inflected adjectives: beyond storage and decomposition.
Brain Res. 2014 Jan 16;1543:223-34. doi: 10.1016/j.brainres.2013.10.038. Epub 2013 Oct 24.
9
Conventionality determines the time course of indirect replies comprehension: An ERP study.
Brain Lang. 2023 Apr;239:105253. doi: 10.1016/j.bandl.2023.105253. Epub 2023 Mar 29.
10
Syntax, concepts, and logic in the temporal dynamics of language comprehension: evidence from event-related potentials.
Neuropsychologia. 2010 May;48(6):1525-42. doi: 10.1016/j.neuropsychologia.2010.01.013. Epub 2010 Feb 4.

引用本文的文献

1
Mark the Unexpected! Animacy Preference and Directed Movement in Visual Language.
Cogn Sci. 2025 May;49(5):e70067. doi: 10.1111/cogs.70067.
2
Semantic processing of verbal narratives compared to semantic processing of visual narratives: an ERP study of school-aged children.
Front Psychol. 2024 Jan 12;14:1253509. doi: 10.3389/fpsyg.2023.1253509. eCollection 2023.

本文引用的文献

1
Visual narrative comprehension: Universal or not?
Psychon Bull Rev. 2020 Apr;27(2):266-285. doi: 10.3758/s13423-019-01670-1.
3
Your Brain on Comics: A Cognitive Model of Visual Narrative Comprehension.
Top Cogn Sci. 2020 Jan;12(1):352-386. doi: 10.1111/tops.12421. Epub 2019 Apr 8.
4
PsychoPy2: Experiments in behavior made easy.
Behav Res Methods. 2019 Feb;51(1):195-203. doi: 10.3758/s13428-018-01193-y.
5
Morphological processing in the brain: The good (inflection), the bad (derivation) and the ugly (compounding).
Cortex. 2019 Jul;116:4-44. doi: 10.1016/j.cortex.2018.08.016. Epub 2018 Sep 1.
6
A strong wink between verbal and emoji-based irony: How the brain processes ironic emojis during language comprehension.
PLoS One. 2018 Aug 15;13(8):e0201727. doi: 10.1371/journal.pone.0201727. eCollection 2018.
8
The effect of category learning on visual attention and visual representation.
Psychophysiology. 2017 Dec;54(12):1855-1871. doi: 10.1111/psyp.12966. Epub 2017 Aug 4.
9
When a hit sounds like a kiss: An electrophysiological exploration of semantic processing in visual narrative.
Brain Lang. 2017 Jun;169:28-38. doi: 10.1016/j.bandl.2017.02.001. Epub 2017 Feb 24.
10
Iconic faces are not real faces: enhanced emotion detection and altered neural processing as faces become more iconic.
Cogn Res Princ Implic. 2016;1(1):19. doi: 10.1186/s41235-016-0021-8. Epub 2016 Dec 12.

文献AI研究员

20分钟写一篇综述,助力文献阅读效率提升50倍。

立即体验

用中文搜PubMed

大模型驱动的PubMed中文搜索引擎

马上搜索

文档翻译

学术文献翻译模型,支持多种主流文档格式。

立即体验