Swartz Center for Computational Neuroscience, Institute for Neural Computation, University of California San Diego, La Jolla, CA 92093, USA.
Division of Child and Adolescent Psychiatry, Cincinnati Children's Hospital Medical Center, Cincinnati, OH 45229, USA.
Sensors (Basel). 2023 Jul 20;23(14):6542. doi: 10.3390/s23146542.
Full-body motion capture is essential for the study of body movement. Video-based, markerless, mocap systems are, in some cases, replacing marker-based systems, but hybrid systems are less explored. We develop methods for coregistration between 2D video and 3D marker positions when precise spatial relationships are not known a priori. We illustrate these methods on three-ball cascade juggling in which it was not possible to use marker-based tracking of the balls, and no tracking of the hands was possible due to occlusion. Using recorded video and motion capture, we aimed to transform 2D ball coordinates into 3D body space as well as recover details of hand motion. We proposed four linear coregistration methods that differ in how they optimize ball-motion constraints during hold and flight phases, using an initial estimate of hand position based on arm and wrist markers. We found that minimizing the error between ball and hand estimate was globally suboptimal, distorting ball flight trajectories. The best-performing method used gravitational constraints to transform vertical coordinates and ball-hold constraints to transform lateral coordinates. This method enabled an accurate description of ball flight as well as a reconstruction of wrist movements. We discuss these findings in the broader context of video/motion capture coregistration.
全身运动捕捉对于研究身体运动至关重要。在某些情况下，基于视频的无标记运动捕捉系统正在取代基于标记的系统，但混合系统的研究较少。我们开发了在不知道先验精确空间关系的情况下，将 2D 视频和 3D 标记位置进行配准的方法。我们在三个球级联杂耍中展示了这些方法，在这种情况下，无法使用基于标记的球跟踪，并且由于遮挡，无法对手进行跟踪。使用记录的视频和运动捕捉，我们旨在将 2D 球坐标转换为 3D 身体空间，并恢复手部运动的细节。我们提出了四种线性配准方法，它们在持有和飞行阶段如何优化球运动约束方面有所不同，使用基于手臂和手腕标记的初始手部位置估计。我们发现，最小化球和手估计之间的误差在全局上是次优的，会扭曲球的飞行轨迹。表现最好的方法使用重力约束来转换垂直坐标，使用球保持约束来转换横向坐标。该方法能够准确描述球的飞行轨迹，并重建手腕运动。我们在视频/运动捕捉配准的更广泛背景下讨论了这些发现。
