Enge Kajetan, Rind Alexander, Iber Michael, Höldrich Robert, Aigner Wolfgang
Institute of Creative Media Technologies, FH St. Pölten, Campusplatz 1, St. Pölten, 3100 Austria.
Institute of Electronic Music and Acoustics, University of Music and Performing Arts Graz, Leonhardstraße 15, Graz, 8010 Austria.
Pers Ubiquitous Comput. 2023;27(5):1949-1963. doi: 10.1007/s00779-023-01720-5. Epub 2023 Aug 21.
Both sonification and visualization convey information about data by effectively using our human perceptual system, but their ways to transform the data differ. Over the past 30 years, the sonification community has demanded a holistic perspective on data representation, including audio-visual analysis, several times. A design theory of audio-visual analysis would be a relevant step in this direction. An indispensable foundation for this endeavor is a terminology describing the combined design space. To build a bridge between the domains, we adopt three of the established theoretical constructs from visualization theory for the field of sonification. The three constructs are the , the , and the . In our model, we choose time to be the of sonification. are then positioned in time, such as visual marks are positioned in space. are encoded into auditory marks to convey information. The proposed definitions allow discussing visualization and sonification designs as well as multi-modal designs based on a common terminology. While the identified terminology can support audio-visual analytics research, it also provides a new perspective on sonification theory itself.
可听化和可视化都通过有效利用人类感知系统来传达数据信息,但它们转换数据的方式有所不同。在过去30年里,可听化社区多次要求从整体角度看待数据表示,包括视听分析。视听分析的设计理论将是朝着这个方向迈出的重要一步。这项工作不可或缺的基础是一个描述组合设计空间的术语。为了在这些领域之间架起一座桥梁,我们为可听化领域采用了可视化理论中已有的三个理论结构。这三个结构分别是 , ,以及 。在我们的模型中,我们选择时间作为可听化的 。 然后在时间中定位,就像视觉标记在空间中定位一样。 被编码为听觉标记以传达信息。所提出的定义允许基于通用术语讨论可视化和可听化设计以及多模态设计。虽然所确定的术语可以支持视听分析研究,但它也为可听化理论本身提供了一个新视角。
(原文中部分关键内容缺失,用“ ”表示)