Visi Federico Ghelli, Östersjö Stefan, Ek Robert, Röijezon Ulrik
Gesture Embodiment and Machines in Music (GEMM), School of Music in Piteå, Luleå University of Technology, Luleå, Sweden.
Division of Health, Medicine and Rehabilitation, Department of Health Sciences, Luleå University of Technology, Luleå, Sweden.
Front Psychol. 2020 Dec 4;11:576751. doi: 10.3389/fpsyg.2020.576751. eCollection 2020.
Musical performance is a multimodal experience, for performers and listeners alike. This paper reports on a pilot study which constitutes the first step toward a comprehensive approach to the experience of music as performed. We aim at bridging the gap between qualitative and quantitative approaches, by combining methods for data collection. The purpose is to build a data corpus containing multimodal measures linked to high-level subjective observations. This will allow for a systematic inclusion of the knowledge of music professionals in an analytic framework, which synthesizes methods across established research disciplines. We outline the methods we are currently developing for the creation of a multimodal data corpus dedicated to the analysis and exploration of instrumental music performance from the perspective of embodied music cognition. This will enable the study of the multiple facets of instrumental music performance in great detail, as well as lead to the development of music creation techniques that take advantage of the cross-modal relationships and higher-level qualities emerging from the analysis of this multi-layered, multimodal corpus. The results of the pilot project suggest that qualitative analysis through stimulated recall is an efficient method for generating higher-level understandings of musical performance. Furthermore, the results indicate several directions for further development, regarding observational movement analysis, and computational analysis of coarticulation, chunking, and movement qualities in musical performance. We argue that the development of methods for combining qualitative and quantitative data are required to fully understand expressive musical performance, especially in a broader scenario in which arts, humanities, and science are increasingly entangled. The future work in the project will therefore entail an increasingly multimodal analysis, aiming to become as holistic as is music in performance.
音乐表演对表演者和听众来说都是一种多模态体验。本文报道了一项初步研究，该研究是迈向全面探讨音乐表演体验方法的第一步。我们旨在通过结合数据收集方法来弥合定性和定量方法之间的差距。目的是建立一个数据语料库，其中包含与高层次主观观察相关的多模态测量。这将允许在一个分析框架中系统地纳入音乐专业人士的知识，该框架综合了既定研究学科的方法。我们概述了目前正在开发的方法，用于创建一个多模态数据语料库，从具身音乐认知的角度分析和探索器乐表演。这将能够详细研究器乐表演的多个方面，并导致音乐创作技术的发展，这些技术利用了从对这个多层多模态语料库的分析中出现的跨模态关系和高层次特征。试点项目的结果表明，通过刺激回忆进行定性分析是产生对音乐表演更高层次理解的有效方法。此外，结果指出了在观察性动作分析以及音乐表演中协同发音、组块和动作特征的计算分析方面进一步发展的几个方向。我们认为，需要开发结合定性和定量数据的方法，以充分理解富有表现力的音乐表演，特别是在艺术、人文和科学日益交织的更广泛背景下。因此，该项目未来的工作将需要越来越多的多模态分析，旨在变得像表演中的音乐一样全面。
