Ferris Thomas K, Sarter Nadine B
Department of Industrial and Operations Engineering, Center for Ergonomics, University of Michigan, 1205 Beal Ave., Ann Arbor, MI 48109, USA.
Hum Factors. 2008 Feb;50(1):17-26. doi: 10.1518/001872008X250566.
This study sought to determine whether performance effects of cross-modal spatial links that were observed in earlier laboratory studies scale to more complex environments and need to be considered in multimodal interface design. It also revisits the unresolved issue of cross-modal cuing asymmetries.
Previous laboratory studies employing simple cues, tasks, and/or targets have demonstrated that the efficiency of processing visual, auditory, and tactile stimuli is affected by the modality, lateralization, and timing of surrounding cues. Very few studies have investigated these cross-modal constraints in the context of more complex environments to determine whether they scale and how complexity affects the nature of cross-modal cuing asymmetries.
Amicroworld simulation of battlefield operations with a complex task set and meaningful visual, auditory, and tactile stimuli was used to investigate cuing effects for all cross-modal pairings.
Significant asymmetric performance effects of cross-modal spatial links were observed. Auditory cues shortened response latencies for collocated visual targets but visual cues did not do the same for collocated auditory targets. Responses to contralateral (rather than ipsilateral) targets were faster for tactually cued auditory targets and each visual-tactile cue-target combination, suggesting an inhibition-of-return effect.
The spatial relationships between multimodal cues and targets significantly affect target response times in complex environments. The performance effects of cross-modal links and the observed cross-modal cuing asymmetries need to be examined in more detail and considered in future interface design.
The findings from this study have implications for the design of multimodal and adaptive interfaces and for supporting attention management in complex, data-rich domains.
本研究旨在确定早期实验室研究中观察到的跨模态空间联系的性能效应是否适用于更复杂的环境，以及在多模态界面设计中是否需要考虑。它还重新审视了未解决的跨模态线索不对称问题。
以往采用简单线索、任务和/或目标的实验室研究表明，视觉、听觉和触觉刺激的处理效率受周围线索的模态、偏侧化和时间的影响。很少有研究在更复杂的环境中研究这些跨模态限制，以确定它们是否适用以及复杂性如何影响跨模态线索不对称的性质。
使用具有复杂任务集以及有意义的视觉、听觉和触觉刺激的战场操作微观世界模拟来研究所有跨模态配对的线索效应。
观察到跨模态空间联系存在显著的不对称性能效应。听觉线索缩短了并置视觉目标的反应潜伏期，但视觉线索对并置听觉目标却没有同样的效果。对于触觉线索提示的听觉目标以及每种视觉 - 触觉线索 - 目标组合，对侧（而非同侧）目标的反应更快，这表明存在返回抑制效应。
多模态线索与目标之间的空间关系在复杂环境中显著影响目标反应时间。跨模态联系的性能效应以及观察到的跨模态线索不对称需要更详细地研究，并在未来的界面设计中加以考虑。
本研究的结果对多模态和自适应界面的设计以及在复杂、数据丰富的领域中支持注意力管理具有启示意义。
