Max Planck Institute for Psycholinguistics, 6525 XD, Nijmegen, The Netherlands.
Donders Institute, Radboud University, 6525EN Nijmegen, The Netherlands.
J Neurosci. 2020 Dec 2;40(49):9467-9475. doi: 10.1523/JNEUROSCI.0302-20.2020. Epub 2020 Oct 23.
Neural oscillations track linguistic information during speech comprehension (Ding et al., 2016; Keitel et al., 2018), and are known to be modulated by acoustic landmarks and speech intelligibility (Doelling et al., 2014; Zoefel and VanRullen, 2015). However, studies investigating linguistic tracking have either relied on non-naturalistic isochronous stimuli or failed to fully control for prosody. Therefore, it is still unclear whether low-frequency activity tracks linguistic structure during natural speech, where linguistic structure does not follow such a palpable temporal pattern. Here, we measured electroencephalography (EEG) and manipulated the presence of semantic and syntactic information apart from the timescale of their occurrence, while carefully controlling for the acoustic-prosodic and lexical-semantic information in the signal. EEG was recorded while 29 adult native speakers (22 women, 7 men) listened to naturally spoken Dutch sentences, jabberwocky controls with morphemes and sentential prosody, word lists with lexical content but no phrase structure, and backward acoustically matched controls. Mutual information (MI) analysis revealed sensitivity to linguistic content: MI was highest for sentences at the phrasal (0.8-1.1 Hz) and lexical (1.9-2.8 Hz) timescales, suggesting that the delta-band is modulated by lexically driven combinatorial processing beyond prosody, and that linguistic content (i.e., structure and meaning) organizes neural oscillations beyond the timescale and rhythmicity of the stimulus. This pattern is consistent with neurophysiologically inspired models of language comprehension (Martin, 2016, 2020; Martin and Doumas, 2017) where oscillations encode endogenously generated linguistic content over and above exogenous or stimulus-driven timing and rhythm information. Biological systems like the brain encode their environment not only by reacting in a series of stimulus-driven responses, but by combining stimulus-driven information with endogenous, internally generated, inferential knowledge and meaning. Understanding language from speech is the human benchmark for this. Much research focuses on the purely stimulus-driven response, but here, we focus on the goal of language behavior: conveying structure and meaning. To that end, we use naturalistic stimuli that contrast acoustic-prosodic and lexical-semantic information to show that, during spoken language comprehension, oscillatory modulations reflect computations related to inferring structure and meaning from the acoustic signal. Our experiment provides the first evidence to date that compositional structure and meaning organize the oscillatory response, above and beyond prosodic and lexical controls.
神经振荡在言语理解过程中跟踪语言信息(Ding 等人,2016 年;Keitel 等人,2018 年),并且已知会受到声学标志和言语可懂度的调制(Doelling 等人,2014 年;Zoefel 和 VanRullen,2015 年)。然而,研究语言跟踪的研究要么依赖于非自然的等时刺激,要么未能完全控制韵律。因此,目前尚不清楚低频活动是否在自然言语中跟踪语言结构,而在自然言语中,语言结构并不遵循这种明显的时间模式。在这里,我们测量了脑电图(EEG),并在不考虑其发生时间尺度的情况下,分别操纵语义和句法信息的存在,同时仔细控制信号中的声学韵律和词汇语义信息。当 29 名成年母语为荷兰语的说话者(22 名女性,7 名男性)听自然说话的荷兰语句子、带有词素和句子韵律的 Jabberwocky 控制、具有词汇内容但没有短语结构的单词列表以及向后声学匹配的控制时,记录了 EEG。互信息(MI)分析揭示了对语言内容的敏感性:句子在词法(0.8-1.1 Hz)和词汇(1.9-2.8 Hz)时间尺度上的 MI 最高,这表明 delta 波段受词汇驱动的组合处理调制,而语言内容(即结构和意义)在刺激的时间尺度和节奏之外组织神经振荡。这种模式与受神经生理学启发的语言理解模型(Martin,2016 年,2020 年;Martin 和 Doumas,2017 年)一致,其中振荡编码内源性生成的语言内容,超出了外源性或刺激驱动的时间和节奏信息。像大脑这样的生物系统不仅通过一系列刺激驱动的反应来对环境做出反应,而且还通过将刺激驱动的信息与内部生成的、内源性的、推断性的知识和意义相结合来对环境做出反应。从言语中理解语言是人类的基准。许多研究都集中在纯粹的刺激驱动反应上,但在这里,我们关注语言行为的目标:传达结构和意义。为此,我们使用自然刺激来对比声学韵律和词汇语义信息,以表明在口语理解过程中,振荡调制反映了从声学信号中推断结构和意义的计算。我们的实验提供了迄今为止的第一个证据,即组合结构和意义组织了振荡反应,超出了韵律和词汇控制。