Department of Psychological and Cognitive Sciences, Tsinghua University, Beijing 100084, China.
Tsinghua Laboratory of Brain and Intelligence, Tsinghua University, Beijing 100084, China.
Soc Cogn Affect Neurosci. 2024 Aug 2;19(1). doi: 10.1093/scan/nsae051.
Recent research has extensively reported the phenomenon of inter-brain neural coupling between speakers and listeners during speech communication. Yet, the specific speech processes underlying this neural coupling remain elusive. To bridge this gap, this study estimated the correlation between the temporal dynamics of speaker-listener neural coupling with speech features, utilizing two inter-brain datasets accounting for different noise levels and listener's language experiences (native vs. non-native). We first derived time-varying speaker-listener neural coupling, extracted acoustic feature (envelope) and semantic features (entropy and surprisal) from speech, and then explored their correlational relationship. Our findings reveal that in clear conditions, speaker-listener neural coupling correlates with semantic features. However, as noise increases, this correlation is only significant for native listeners. For non-native listeners, neural coupling correlates predominantly with acoustic feature rather than semantic features. These results revealed how speaker-listener neural coupling is associated with the acoustic and semantic features under various scenarios, enriching our understanding of the inter-brain neural mechanisms during natural speech communication. We therefore advocate for more attention on the dynamic nature of speaker-listener neural coupling and its modeling with multilevel speech features.
最近的研究广泛报道了在言语交流过程中说话者和听者之间的大脑间神经耦合现象。然而，这种神经耦合背后的具体言语过程仍然难以捉摸。为了弥补这一空白，本研究利用两个涉及不同噪声水平和听者语言经验（母语者与非母语者）的脑间数据集，估计了说话者-听者神经耦合的时间动态与言语特征之间的相关性。我们首先从言语中提取出时变的说话者-听者神经耦合、声学分量（包络）和语义分量（熵和惊讶度），然后探索它们的相关关系。我们的研究结果表明，在清晰的条件下，说话者-听者神经耦合与语义特征相关。然而，随着噪声的增加，这种相关性仅在母语者中显著。对于非母语者，神经耦合主要与声学分量而不是语义特征相关。这些结果揭示了在不同情况下，说话者-听者神经耦合如何与言语的声学分量和语义特征相关联，丰富了我们对自然言语交流中大脑间神经机制的理解。因此，我们主张更多地关注说话者-听者神经耦合的动态特性及其与多层次言语特征的建模。
