Coco Moreno I, Keller Frank
Institute for Language, Cognition and Computation, School of Informatics, University of Edinburgh, 10 Crichton Street, Edinburgh, EH8 9AB, UK,
Cogn Process. 2015 May;16(2):131-50. doi: 10.1007/s10339-014-0642-0. Epub 2014 Nov 23.
Situated language production requires the integration of visual attention and linguistic processing. Previous work has not conclusively disentangled the role of perceptual scene information and structural sentence information in guiding visual attention. In this paper, we present an eye-tracking study that demonstrates that three types of guidance, perceptual, conceptual, and structural, interact to control visual attention. In a cued language production experiment, we manipulate perceptual (scene clutter) and conceptual guidance (cue animacy) and measure structural guidance (syntactic complexity of the utterance). Analysis of the time course of language production, before and during speech, reveals that all three forms of guidance affect the complexity of visual responses, quantified in terms of the entropy of attentional landscapes and the turbulence of scan patterns, especially during speech. We find that perceptual and conceptual guidance mediate the distribution of attention in the scene, whereas structural guidance closely relates to scan pattern complexity. Furthermore, the eye-voice span of the cued object and its perceptual competitor are similar; its latency mediated by both perceptual and structural guidance. These results rule out a strict interpretation of structural guidance as the single dominant form of visual guidance in situated language production. Rather, the phase of the task and the associated demands of cross-modal cognitive processing determine the mechanisms that guide attention.
情境语言产生需要视觉注意力和语言处理的整合。以往的研究尚未明确区分感知场景信息和句子结构信息在引导视觉注意力方面的作用。在本文中，我们进行了一项眼动追踪研究，结果表明感知、概念和结构这三种引导方式相互作用，共同控制视觉注意力。在一个线索化语言产生实验中，我们操纵感知引导（场景杂乱程度）和概念引导（线索的生动性），并测量结构引导（话语的句法复杂性）。对语言产生过程中（说话前和说话时）的时间进程进行分析后发现，所有这三种引导形式都会影响视觉反应的复杂性，这种复杂性通过注意力分布的熵和扫描模式的紊乱程度来量化，尤其是在说话过程中。我们发现，感知引导和概念引导调节场景中的注意力分布，而结构引导则与扫描模式的复杂性密切相关。此外，线索化对象及其感知竞争对象的眼声跨度相似；其潜伏期受感知引导和结构引导的共同影响。这些结果排除了将结构引导严格解释为情境语言产生中视觉引导的唯一主导形式的观点。相反，任务阶段以及跨模态认知处理的相关要求决定了引导注意力的机制。
