Suppr超能文献

难以辨认:视觉对言语工作记忆的贡献。

Beyond Recognition: Visual Contributions to Verbal Working Memory.

机构信息

Department of Speech, Language, and Hearing Sciences, University of Florida, Gainesville.

出版信息

J Speech Lang Hear Res. 2022 Jan 12;65(1):253-273. doi: 10.1044/2021_JSLHR-21-00177. Epub 2021 Nov 17.

Abstract

PURPOSE

It is well recognized that adding the visual to the acoustic speech signal improves recognition when the acoustic signal is degraded, but how that visual signal affects postrecognition processes is not so well understood. This study was designed to further elucidate the relationships among auditory and visual codes in working memory, a postrecognition process.

DESIGN

In a main experiment, 80 young adults with normal hearing were tested using an immediate serial recall paradigm. Three types of signals were presented (unprocessed speech, vocoded speech, and environmental sounds) in three conditions (audio-only, audio-video with dynamic visual signals, and audio-picture with static visual signals). Three dependent measures were analyzed: (a) magnitude of the recency effect, (b) overall recall accuracy, and (c) response times, to assess cognitive effort. In a follow-up experiment, 30 young adults with normal hearing were tested largely using the same procedures, but with a slight change in order of stimulus presentation.

RESULTS

The main experiment produced three major findings: (a) unprocessed speech evoked a recency effect of consistent magnitude across conditions; vocoded speech evoked a recency effect of similar magnitude to unprocessed speech only with dynamic visual (lipread) signals; environmental sounds never showed a recency effect. (b) Dynamic and static visual signals enhanced overall recall accuracy to a similar extent, and this enhancement was greater for vocoded speech and environmental sounds than for unprocessed speech. (c) All visual signals reduced cognitive load, except for dynamic visual signals with environmental sounds. The follow-up experiment revealed that dynamic visual (lipread) signals exerted their effect on the vocoded stimuli by enhancing phonological quality.

CONCLUSIONS

Acoustic and visual signals can combine to enhance working memory operations, but the source of these effects differs for phonological and nonphonological signals. Nonetheless, visual information can support better postrecognition processes for patients with hearing loss.

摘要

目的

人们已经认识到,当语音信号受损时,将视觉信号添加到语音信号中可以提高识别率,但人们对视觉信号如何影响识别后的过程还不太了解。本研究旨在进一步阐明工作记忆中听觉和视觉代码之间的关系,工作记忆是识别后的一个过程。

设计

在一项主要实验中,80 名听力正常的年轻人使用即时系列回忆范式进行了测试。三种类型的信号(未经处理的语音、语音编码语音和环境声音)在三种条件下呈现(仅音频、带有动态视觉信号的音频-视频和带有静态视觉信号的音频-图片)。分析了三个依赖指标:(a)近因效应的大小,(b)整体回忆准确性,和(c)反应时间,以评估认知努力。在后续实验中,30 名听力正常的年轻人使用大致相同的程序进行了测试,但刺激呈现的顺序略有变化。

结果

主要实验产生了三个主要发现:(a)未经处理的语音在所有条件下都产生了一致大小的近因效应;语音编码语音仅在带有动态视觉(唇读)信号时产生与未经处理的语音相似大小的近因效应;环境声音从未表现出近因效应。(b)动态和静态视觉信号以相似的程度增强了整体回忆准确性,并且这种增强对于语音编码语音和环境声音比对未经处理的语音更大。(c)所有视觉信号都降低了认知负荷,除了环境声音的动态视觉信号。后续实验表明,动态视觉(唇读)信号通过增强语音质量对语音编码刺激产生影响。

结论

声学和视觉信号可以结合起来增强工作记忆操作,但这些效果的来源对于语音和非语音信号是不同的。尽管如此,视觉信息可以为听力损失患者提供更好的识别后过程支持。

相似文献

1
Beyond Recognition: Visual Contributions to Verbal Working Memory.
J Speech Lang Hear Res. 2022 Jan 12;65(1):253-273. doi: 10.1044/2021_JSLHR-21-00177. Epub 2021 Nov 17.
3
Interactions Between Item Set and Vocoding in Serial Recall.
Ear Hear. 2019 Nov/Dec;40(6):1404-1417. doi: 10.1097/AUD.0000000000000718.
4
Modalities of memory: is reading lips like hearing voices?
Cognition. 2013 Dec;129(3):471-93. doi: 10.1016/j.cognition.2013.08.017. Epub 2013 Sep 14.
5
Children's suffix effects for verbal working memory reflect phonological coding and perceptual grouping.
J Exp Child Psychol. 2019 Jul;183:276-294. doi: 10.1016/j.jecp.2019.03.003. Epub 2019 Mar 29.
7
Reconstructing wholes from parts: effects of modality, age, and hearing loss on word recognition.
Ear Hear. 2013 Mar-Apr;34(2):e14-23. doi: 10.1097/AUD.0b013e31826d0c27.
8
Precision of working memory for speech sounds.
Q J Exp Psychol (Hove). 2015;68(10):2022-40. doi: 10.1080/17470218.2014.1002799. Epub 2015 Mar 11.
9
Auditory recency in immediate memory.
Q J Exp Psychol A. 1993 May;46(2):193-223. doi: 10.1080/14640749308401044.

引用本文的文献

1
How Hearing Loss and Cochlear Implantation Affect Verbal Working Memory: Evidence From Adolescents.
J Speech Lang Hear Res. 2024 Jun 6;67(6):1850-1867. doi: 10.1044/2024_JSLHR-23-00446. Epub 2024 May 7.

本文引用的文献

1
Attention effects in working memory that are asymmetric across sensory modalities.
Mem Cognit. 2021 Jul;49(5):1050-1065. doi: 10.3758/s13421-021-01142-9. Epub 2021 Mar 10.
2
Acoustic-Phonetic Mismatches Impair Serial Recall of Degraded Words.
Audit Percept Cogn. 2020;3(1-2):55-75. doi: 10.1080/25742442.2020.1846012. Epub 2020 Nov 11.
3
Acclimatization to Hearing Aids by Older Adults.
Ear Hear. 2021 Jan/Feb;42(1):193-205. doi: 10.1097/AUD.0000000000000913.
4
Do we use visual codes when information is not presented visually?
Mem Cognit. 2020 Nov;48(8):1522-1536. doi: 10.3758/s13421-020-01054-0.
5
Audiovisual Enhancement of Speech Perception in Noise by School-Age Children Who Are Hard of Hearing.
Ear Hear. 2020 Jul/Aug;41(4):705-719. doi: 10.1097/AUD.0000000000000830.
6
Declarative and procedural working memory updating processes are mutually facilitative.
Atten Percept Psychophys. 2020 May;82(4):1858-1871. doi: 10.3758/s13414-019-01887-1.
7
Modality Effects on Lexical Encoding and Memory Representations of Spoken Words.
Ear Hear. 2020 Jul/Aug;41(4):825-837. doi: 10.1097/AUD.0000000000000801.
8
Theories of Working Memory: Differences in Definition, Degree of Modularity, Role of Attention, and Purpose.
Lang Speech Hear Serv Sch. 2018 Jul 5;49(3):340-355. doi: 10.1044/2018_LSHSS-17-0114.
9
The phonological loop as a buffer store: An update.
Cortex. 2019 Mar;112:91-106. doi: 10.1016/j.cortex.2018.05.015. Epub 2018 May 30.
10
Listening Effort: How the Cognitive Consequences of Acoustic Challenge Are Reflected in Brain and Behavior.
Ear Hear. 2018 Mar/Apr;39(2):204-214. doi: 10.1097/AUD.0000000000000494.

文献AI研究员

20分钟写一篇综述,助力文献阅读效率提升50倍。

立即体验

用中文搜PubMed

大模型驱动的PubMed中文搜索引擎

马上搜索

文档翻译

学术文献翻译模型,支持多种主流文档格式。

立即体验