Goldenberg Dolly, Tiede Mark K, Bennett Ryan T, Whalen D H
Haskins Laboratories, New Haven, CT, United States.
Department of Linguistics, University of California, Santa Cruz, Santa Cruz, CA, United States.
Front Hum Neurosci. 2022 Jul 15;16:879981. doi: 10.3389/fnhum.2022.879981. eCollection 2022.
Multimodal integration is the formation of a coherent percept from different sensory inputs such as vision, audition, and somatosensation. Most research on multimodal integration in speech perception has focused on audio-visual integration. In recent years, audio-tactile integration has also been investigated, and it has been established that puffs of air applied to the skin and timed with listening tasks shift the perception of voicing by naive listeners. The current study has replicated and extended these findings by testing the effect of air puffs on gradations of voice onset time along a continuum rather than the voiced and voiceless endpoints of the original work. Three continua were tested: bilabial ("pa/ba"), velar ("ka/ga"), and a vowel continuum ("head/hid") used as a control. The presence of air puffs was found to significantly increase the likelihood of choosing voiceless responses for the two VOT continua but had no effect on choices for the vowel continuum. Analysis of response times revealed that the presence of air puffs lengthened responses for intermediate (ambiguous) stimuli and shortened them for endpoint (non-ambiguous) stimuli. The slowest response times were observed for the intermediate steps for all three continua, but for the bilabial continuum this effect interacted with the presence of air puffs: responses were slower in the presence of air puffs, and faster in their absence. This suggests that during integration auditory and aero-tactile inputs are weighted differently by the perceptual system, with the latter exerting greater influence in those cases where the auditory cues for voicing are ambiguous.
多模态整合是指从视觉、听觉和躯体感觉等不同感官输入中形成连贯的感知。大多数关于言语感知中多模态整合的研究都集中在视听整合上。近年来，听觉-触觉整合也得到了研究，并且已经证实，在聆听任务中，施加在皮肤上并与任务同步的气流会改变未受过训练的听众对浊音的感知。本研究通过测试气流对沿连续体的语音起始时间渐变的影响，而非原始研究中的浊音和清音端点，复制并扩展了这些发现。测试了三个连续体：双唇音（“pa/ba”）、软腭音（“ka/ga”），以及用作对照的元音连续体（“head/hid”）。结果发现，气流的存在显著增加了在两个语音起始时间连续体上选择清音反应的可能性，但对元音连续体的选择没有影响。对反应时间的分析表明，气流的存在延长了对中间（模糊）刺激的反应时间，并缩短了对端点（明确）刺激的反应时间。在所有三个连续体的中间步骤中观察到最慢的反应时间，但对于双唇音连续体，这种效应与气流是否存在相互作用：有气流时反应较慢，无气流时反应较快。这表明在整合过程中，听觉和气动触觉输入在感知系统中的权重不同，在语音听觉线索模糊的情况下，后者的影响更大。
