Graduate School of Science and Technology, Keio University, Yokohama 223-8522, Japan.
Faculty of Sport and Health Sciences, Toin University of Yokohama, Yokohama 225-8503, Japan.
Sensors (Basel). 2023 Nov 20;23(22):9282. doi: 10.3390/s23229282.
Action quality assessment (AQA) tasks in computer vision evaluate action quality in videos, and they can be applied to sports for performance evaluation. A typical example of AQA is predicting the final score from a video that captures an entire figure skating program. However, no previous studies have predicted individual jump scores, which are of great interest to competitors because of the high weight of competition. Despite the presence of unnecessary information in figure skating videos, human specialists can focus and reduce information when they evaluate jumps. In this study, we clarified the eye movements of figure skating judges and skaters while evaluating jumps and proposed a prediction model for jump performance that utilized specialists' gaze location to reduce information. Kinematic features obtained from the tracking system were input into the model in addition to videos to improve accuracy. The results showed that skaters focused more on the face, whereas judges focused on the lower extremities. These gaze locations were applied to the model, which demonstrated the highest accuracy when utilizing both specialists' gaze locations. The model outperformed human predictions and the baseline model (RMSE:0.775), suggesting a combination of human specialist knowledge and machine capabilities could yield higher accuracy.
动作质量评估（AQA）任务在计算机视觉中评估视频中的动作质量，可应用于体育领域的表现评估。AQA 的一个典型示例是从捕获整个花样滑冰节目的视频中预测最终得分。然而，之前的研究并没有预测单个跳跃得分，因为竞争的权重很高，所以这对竞争对手来说很感兴趣。尽管花样滑冰视频中存在不必要的信息，但专家在评估跳跃时可以集中精力并减少信息。在这项研究中，我们阐明了花样滑冰裁判和滑冰者在评估跳跃时的眼球运动，并提出了一种利用专家凝视位置来减少信息的跳跃表现预测模型。运动学特征从跟踪系统中获取，除了视频外，还输入到模型中以提高准确性。结果表明，滑冰者更关注面部，而裁判更关注下肢。这些注视位置应用于模型，当同时利用专家的注视位置时，模型表现出最高的准确性。该模型的表现优于人类预测和基线模型（均方根误差：0.775），这表明人类专家知识和机器能力的结合可以提高准确性。
