Van Kerrebroeck Bavo, Caruso Giusy, Maes Pieter-Jan
Department of Art, Music, and Theatre Sciences, IPEM, Ghent University, Ghent, Belgium.
Front Psychol. 2021 Jun 11;12:663725. doi: 10.3389/fpsyg.2021.663725. eCollection 2021.
Virtual reality (VR) brings radical new possibilities to the empirical study of social music cognition and interaction. In the present article, we consider the role of VR as a research tool, based on its potential to create a sense of "social presence": the illusory feeling of being, and socially interacting, inside a virtual environment. This makes VR promising for bridging ecological validity ("research in the wild") and experimental control ("research in the lab") in empirical music research. A critical assumption however is the actual ability of VR to simulate real-life social interactions, either human-embodied avatars or computer-controlled agents. The mediation of social musical interactions VR is particularly challenging due to their embodied, complex, and emotionally delicate nature. In this article, we introduce a methodological framework to operationalize social presence by a combination of factors across interrelated layers, relating to the performance output, embodied co-regulation, and subjective experiences. This framework provides the basis for the proposal of a pragmatic approach to determine the level of social presence in virtual musical interactions, by comparing the outcomes across the multiple layers with the outcomes of corresponding real-life musical interactions. We applied and tested this pragmatic approach a case-study of piano duet performances of the piece Piano Phase composed by Steve Reich. This case-study indicated that a piano duet performed in VR, in which the real-time interaction between pianists is mediated by embodied avatars, might lead to a strong feeling of social presence, as reflected in the measures of performance output, embodied co-regulation, and subjective experience. In contrast, although a piano duet in VR between an actual pianist and a computer-controlled agent led to a relatively successful performance output, it was inadequate in terms of both embodied co-regulation and subjective experience.
虚拟现实(VR)为社会音乐认知与互动的实证研究带来了全新的可能性。在本文中,我们基于VR创造“社会临场感”的潜力,探讨其作为一种研究工具的作用:即在虚拟环境中存在并进行社会互动的虚幻感觉。这使得VR有望弥合实证音乐研究中的生态效度(“实地研究”)和实验控制(“实验室研究”)。然而,一个关键假设是VR模拟现实生活中社会互动的实际能力,无论是人类化身还是计算机控制的智能体。由于社会音乐互动的具体性、复杂性和情感微妙性,通过VR进行调解尤其具有挑战性。在本文中,我们引入了一个方法框架,通过相互关联的多个层面的因素组合来实现社会临场感的操作化,这些因素涉及表演输出、具体的共同调节和主观体验。该框架为提出一种务实方法奠定了基础,即通过将多个层面的结果与相应现实生活音乐互动的结果进行比较,来确定虚拟音乐互动中的社会临场感水平。我们将这种务实方法应用于并测试了史蒂夫·赖克(Steve Reich)创作的《钢琴阶段》(Piano Phase)钢琴二重奏表演的案例研究。该案例研究表明,在VR中进行的钢琴二重奏表演,其中钢琴家之间的实时互动由具体化身介导,可能会带来强烈的社会临场感,这在表演输出、具体的共同调节和主观体验的测量中都有所体现。相比之下,虽然在VR中实际钢琴家与计算机控制的智能体之间的钢琴二重奏带来了相对成功的表演输出,但在具体的共同调节和主观体验方面都存在不足。