Twomey Patrick, Varma Vaibhavsingh, Bush Leslie L, Trkov Mitja
Department of Mechanical Engineering, Henry M. Rowan College of Engineering, Rowan University, Glassboro, NJ, United States.
Department of Theatre and Dance, College of Performing Arts, Rowan University, Glassboro, NJ, United States.
Front Robot AI. 2024 Nov 29;11:1450177. doi: 10.3389/frobt.2024.1450177. eCollection 2024.
The fusion of wearable soft robotic actuators and motion-tracking sensors can enhance dance performance, amplifying its visual language and communicative potential. However, the intricate and unpredictable nature of improvisational dance poses unique challenges for existing motion-tracking methods, underscoring the need for more adaptable solutions. Conventional methods such as optical tracking face limitations due to limb occlusion. The use of inertial measurement units (IMUs) can alleviate some of these challenges; however, their movement detection algorithms are complex and often based on fixed thresholds. Additionally, machine learning algorithms are unsuitable for detecting the arbitrary motion of improvisational dancers due to the non-repetitive and unique nature of their movements, resulting in limited available training data. To address these challenges, we introduce a collider-based movement detection algorithm. Colliders are modeled as virtual mass-spring-damper systems with its response related to dynamics of limb segments. Individual colliders are defined in planes corresponding to the limbs' degrees of freedom. The system responses of these colliders relate to limb dynamics and can be used to quantify dynamic movements such as jab as demonstrated herein. One key advantage of collider dynamics is their ability to capture complex limb movements in their relative frame, as opposed to the global frame, thus avoiding drift issues common with IMUs. Additionally, we propose a simplified movement detection scheme based on individual dynamic system response variable, as opposed to fixed thresholds that consider multiple variables simultaneously (i.e., displacement, velocity, and acceleration). Our approach combines the collider-based algorithm with a hashing method to design a robust and high-speed detection algorithm for improvised dance motions. Experimental results demonstrate that our algorithm effectively detects improvisational dance movements, allowing control of wearable, origami-based soft actuators that can change size and lighting based on detected movements. This innovative method allows dancers to trigger events on stage, creating a unique organic aesthetics that seamlessly integrates technology with spontaneous movements. Our research highlights how this approach not only enriches dance performances by blending tradition and innovation but also enhances the expressive capabilities of dance, demonstrating the potential for technology to elevate and augment this art form.
可穿戴软机器人致动器与运动跟踪传感器的融合可以提升舞蹈表演，增强其视觉语言和交流潜力。然而，即兴舞蹈复杂且不可预测的特性给现有的运动跟踪方法带来了独特挑战，凸显了对更具适应性解决方案的需求。诸如光学跟踪等传统方法由于肢体遮挡而面临局限性。惯性测量单元（IMU）的使用可以缓解其中一些挑战；然而，它们的运动检测算法复杂，且通常基于固定阈值。此外，由于即兴舞者动作的非重复性和独特性，机器学习算法不适用于检测他们的任意动作，导致可用训练数据有限。为应对这些挑战，我们引入了一种基于碰撞器的运动检测算法。碰撞器被建模为虚拟质量 - 弹簧 - 阻尼系统，其响应与肢体节段的动力学相关。各个碰撞器在对应肢体自由度的平面中定义。这些碰撞器的系统响应与肢体动力学相关，并且如本文所示可用于量化诸如刺拳等动态动作。碰撞器动力学的一个关键优势在于它们能够在相对坐标系而非全局坐标系中捕捉复杂的肢体运动，从而避免了IMU常见的漂移问题。此外，我们提出了一种基于单个动态系统响应变量的简化运动检测方案，而非同时考虑多个变量（即位移、速度和加速度）的固定阈值。我们的方法将基于碰撞器的算法与哈希方法相结合，设计出一种用于即兴舞蹈动作的强大且高速的检测算法。实验结果表明，我们的算法能够有效检测即兴舞蹈动作，实现对基于折纸的可穿戴软致动器的控制，该致动器可根据检测到的动作改变尺寸和灯光。这种创新方法使舞者能够在舞台上触发事件，创造出一种独特的有机美学，将技术与自发动作无缝融合。我们的研究强调了这种方法不仅通过融合传统与创新丰富了舞蹈表演，还提升了舞蹈的表达能力，展示了技术提升和拓展这一艺术形式的潜力。
