Mitchel Aaron D, Weiss Daniel J
Department of Psychology, Bucknell University, Lewisburg, PA 17837, USA.
Department of Psychology and Program in Linguistics, The Pennsylvania State University, 643 Moore Building, University Park, PA 16802, USA.
Lang Cogn Process. 2014;29(7):771-780. doi: 10.1080/01690965.2013.791703.
Speech is typically a multimodal phenomenon, yet few studies have focused on the exclusive contributions of visual cues to language acquisition. To address this gap, we investigated whether visual prosodic information can facilitate speech segmentation. Previous research has demonstrated that language learners can use lexical stress and pitch cues to segment speech and that learners can extract this information from talking faces. Thus, we created an artificial speech stream that contained minimal segmentation cues and paired it with two synchronous facial displays in which visual prosody was either informative or uninformative for identifying word boundaries. Across three familiarisation conditions (audio stream alone, facial streams alone, and paired audiovisual), learning occurred only when the facial displays were informative to word boundaries, suggesting that facial cues can help learners solve the early challenges of language acquisition.
言语通常是一种多模态现象，但很少有研究关注视觉线索对语言习得的独特贡献。为了填补这一空白，我们研究了视觉韵律信息是否能促进语音分割。先前的研究表明，语言学习者可以利用词汇重音和音高线索来分割语音，并且学习者可以从说话的面部中提取这些信息。因此，我们创建了一个包含最少分割线索的人工语音流，并将其与两个同步的面部显示配对，其中视觉韵律对于识别单词边界要么是信息性的，要么是无信息性的。在三种熟悉条件下（单独的音频流、单独的面部流以及配对的视听流），只有当面部显示对单词边界有信息性时才会发生学习，这表明面部线索可以帮助学习者解决语言习得的早期挑战。
