Max Planck Institute for Psycholinguistics, P.O. Box 310, 6500 AH Nijmegen, the Netherlands; Centre for Language Studies, Radboud University, P.O. Box 9103, 6500 HD Nijmegen, the Netherlands.
Tilburg University, P.O. Box 90153, 5000 LE Tilburg, the Netherlands.
Neuropsychologia. 2022 Jul 29;172:108253. doi: 10.1016/j.neuropsychologia.2022.108253. Epub 2022 Apr 30.
Visual narratives make use of various means to convey referential and co-referential meaning, so comprehenders must recognize that different depictions across sequential images represent the same character(s). In this study, we investigated how the order in which different types of panels in visual sequences are presented affects how the unfolding narrative is comprehended. Participants viewed short comic strips while their electroencephalogram (EEG) was recorded. We analyzed evoked and induced EEG activity elicited by both full panels (showing a full character) and refiner panels (showing only a zoom of that full panel), and took into account whether they preceded or followed the panel to which they were co-referentially related (i.e., were cataphoric or anaphoric). We found that full panels elicited both larger N300 amplitude and increased gamma-band power compared to refiner panels. Anaphoric panels elicited a sustained negativity compared to cataphoric panels, which appeared to be sensitive to the referential status of the anaphoric panel. In the time-frequency domain, anaphoric panels elicited reduced 8-12 Hz alpha power and increased 45-65 Hz gamma-band power compared to cataphoric panels. These findings are consistent with models in which the processes involved in visual narrative comprehension partially overlap with those in language comprehension.
视觉叙事利用各种手段来传达指称和共指意义，因此理解者必须认识到，连续图像中的不同描绘代表着相同的人物。在这项研究中，我们调查了视觉序列中不同类型的面板呈现顺序如何影响展开的叙述的理解。参与者在观看短连环漫画时记录他们的脑电图（EEG）。我们分析了由完整面板（显示完整人物）和细化面板（仅显示该完整面板的缩放）引起的诱发和诱导 EEG 活动，并考虑了它们是否先于或后于与它们具有共指关系的面板（即，是前指还是后指）。我们发现，与细化面板相比，完整面板引起的 N300 振幅更大，伽马波段功率增加。与前指相比，后指诱发持续的负性，这似乎对后指的指称状态敏感。在时频域中，与前指相比，后指诱发的 8-12 Hz 阿尔法功率降低和 45-65 Hz 伽马波段功率增加。这些发现与视觉叙事理解中涉及的过程部分与语言理解中涉及的过程重叠的模型一致。
