Kelly Spencer D, Ward Sarah, Creigh Peter, Bartolotti James
Neuroscience Program, Department of Psychology, 13 Oak Dr, Colgate University, Hamitlon, NY 13346, USA.
Brain Lang. 2007 Jun;101(3):222-33. doi: 10.1016/j.bandl.2006.07.008. Epub 2006 Sep 25.
The present study investigates whether knowledge about the intentional relationship between gesture and speech influences controlled processes when integrating the two modalities at comprehension. Thirty-five adults watched short videos of gesture and speech that conveyed semantically congruous and incongruous information. In half of the videos, participants were told that the two modalities were intentionally coupled (i.e., produced by the same communicator), and in the other half, they were told that the two modalities were not intentionally coupled (i.e., produced by different communicators). When participants knew that the same communicator produced the speech and gesture, there was a larger bi-lateral frontal and central N400 effect to words that were semantically incongruous versus congruous with gesture. However, when participants knew that different communicators produced the speech and gesture--that is, when gesture and speech were not intentionally meant to go together--the N400 effect was present only in right-hemisphere frontal regions. The results demonstrate that pragmatic knowledge about the intentional relationship between gesture and speech modulates controlled neural processes during the integration of the two modalities.
本研究调查了关于手势与言语之间意向性关系的知识在理解过程中整合这两种模态时是否会影响受控过程。35名成年人观看了传达语义一致和不一致信息的手势与言语短视频。在一半的视频中，参与者被告知这两种模态是有意配对的（即由同一 communicator 产生），而在另一半视频中，他们被告知这两种模态不是有意配对的（即由不同 communicator 产生）。当参与者知道言语和手势由同一 communicator 产生时，与手势语义不一致的单词相比与手势语义一致的单词会产生更大的双侧额叶和中央N400效应。然而，当参与者知道言语和手势由不同 communicator 产生时，即当手势和言语并非有意配合时，N400效应仅出现在右半球额叶区域。结果表明，关于手势与言语之间意向性关系的语用知识在整合这两种模态的过程中调节受控神经过程。
