Pinardi Mattia, Di Stefano Nicola, Di Pino Giovanni, Spence Charles
NeXT Lab, Neurophysiology and Neuroengineering of Human-Technology Interaction Research Unit, Università Campus Bio-Medico di Roma, Rome, Italy.
Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy.
Front Psychol. 2023 Jun 15;14:1190103. doi: 10.3389/fpsyg.2023.1190103. eCollection 2023.
"Crossmodal correspondences" are the consistent mappings between perceptual dimensions or stimuli from different sensory domains, which have been widely observed in the general population and investigated by experimental psychologists in recent years. At the same time, the emerging field of human movement augmentation (i.e., the enhancement of an individual's motor abilities by means of artificial devices) has been struggling with the question of how to relay supplementary information concerning the state of the artificial device and its interaction with the environment to the user, which may help the latter to control the device more effectively. To date, this challenge has not been explicitly addressed by capitalizing on our emerging knowledge concerning crossmodal correspondences, despite these being tightly related to multisensory integration. In this perspective paper, we introduce some of the latest research findings on the crossmodal correspondences and their potential role in human augmentation. We then consider three ways in which the former might impact the latter, and the feasibility of this process. First, crossmodal correspondences, given the documented effect on attentional processing, might facilitate the integration of device status information (e.g., concerning position) coming from different sensory modalities (e.g., haptic and visual), thus increasing their usefulness for motor control and embodiment. Second, by capitalizing on their widespread and seemingly spontaneous nature, crossmodal correspondences might be exploited to reduce the cognitive burden caused by additional sensory inputs and the time required for the human brain to adapt the representation of the body to the presence of the artificial device. Third, to accomplish the first two points, the benefits of crossmodal correspondences should be maintained even after sensory substitution, a strategy commonly used when implementing supplementary feedback.
“跨通道对应关系”是指不同感觉通道的感知维度或刺激之间的一致映射,这种现象在普通人群中已被广泛观察到,并且近年来受到了实验心理学家的研究。与此同时,新兴的人体运动增强领域(即通过人工装置增强个体的运动能力)一直在努力解决如何将有关人工装置状态及其与环境相互作用的补充信息传递给用户的问题,这可能有助于用户更有效地控制该装置。迄今为止,尽管跨通道对应关系与多感官整合密切相关,但尚未通过利用我们关于跨通道对应关系的新知识来明确应对这一挑战。在这篇观点论文中,我们介绍了一些关于跨通道对应关系的最新研究发现及其在人体增强中的潜在作用。然后,我们考虑了前者可能影响后者的三种方式以及这一过程的可行性。首先,鉴于跨通道对应关系对注意力加工有已被记录的影响,它可能有助于整合来自不同感觉模态(如触觉和视觉)的装置状态信息(如关于位置的信息),从而提高这些信息在运动控制和身体表征方面的有用性。其次,利用跨通道对应关系广泛且看似自发的特性,可以减轻额外感官输入造成的认知负担以及人类大脑使身体表征适应人工装置存在所需的时间。第三,为了实现前两点,即使在实施补充反馈时常用的感觉替代之后,跨通道对应关系的益处也应得以保持。