Higaki Shogo, Menezes Guilherme L, Ferreira Rafael E P, Negreiro Ariana, Cabrera Victor E, Dórea João R R
National Institute of Animal Health, National Agriculture and Food Research Organization, Tsukuba, Ibaraki, 305-0856, Japan; Department of Animal and Dairy Sciences, University of Wisconsin-Madison, Madison, WI 53706.
Department of Animal and Dairy Sciences, University of Wisconsin-Madison, Madison, WI 53706.
J Dairy Sci. 2025 Apr;108(4):3942-3955. doi: 10.3168/jds.2024-25545. Epub 2024 Dec 18.
The objective of this study was to assess the applicability of a computer vision-based keypoint detection technique to extract mobility variables associated with mobility scores from top-view 2-dimensional (2D) videos of dairy cows. In addition, the study determined the potential of a machine learning classification model to predict mobility scores based on the newly extracted mobility variables. A dataset of 256 video clips of individual cows was collected, with each clip recorded from a top-view perspective while the cows were walking. The cows were visually assessed using a 4-level mobility scoring system, comprising score 0 (good mobility: 78 cows), score 1 (imperfect mobility: 71 cows), score 2 (impaired mobility: 87 cows), and score 3 (severely impaired mobility: 20 cows). The video clips were analyzed using a keypoint detection model capable of detecting 10 keypoints (i.e., head, neck, withers, back, hip ridge, tail head, left and right hooks, and left and right pins). From the time-series XY-coordinate data of the keypoints, 25 mobility variables were extracted, including lateral movements of keypoints (10 variables), coefficients of variation of keypoint speeds (10 variables), walking speed (1 variable), and standard deviation of keypoint angles (4 variables: neck angle, withers angle, back angle, and hip angle). Due to the limited number of cows classified as score 3, they were combined with score 2 cows into a single class. Subsequently, a 3-level mobility score classification model (score 0, 1, and 2 + 3) was developed using the random forest algorithm, based on the extracted mobility variables. The model's performance was evaluated using the repeated holdout method, where the dataset was randomly divided into 80% for training and 20% for testing, repeated 10 times. The model's overall 3-class classification performance achieved a weighted kappa coefficient of 0.72 and an area under the curve of the receiver operating characteristic curve of 0.89. Based on the variable importance analysis conducted over the cross-validation, back lateral movement, withers lateral movement, walking speed, and tail head lateral movement were identified as crucial for predicting mobility scores. These findings indicate that the computer vision-based keypoint detection technique is effective for extracting mobility variables relevant to mobility scores from top-view 2D videos, and the machine learning classification model based on the newly extracted variables has the potential for objective mobility scoring in dairy cows.
本研究的目的是评估基于计算机视觉的关键点检测技术在从奶牛的俯视二维（2D）视频中提取与运动评分相关的运动变量方面的适用性。此外，该研究还确定了机器学习分类模型基于新提取的运动变量预测运动评分的潜力。收集了一个包含256个个体奶牛视频片段的数据集，每个片段都是在奶牛行走时从俯视角度录制的。使用四级运动评分系统对奶牛进行视觉评估，该系统包括评分0（良好运动能力：78头奶牛）、评分1（运动能力欠佳：71头奶牛）、评分2（运动能力受损：87头奶牛）和评分3（严重运动能力受损：20头奶牛）。使用能够检测10个关键点（即头部、颈部、肩胛、背部、髋嵴、尾根、左右钩部和左右荐结节）的关键点检测模型对视频片段进行分析。从关键点的时间序列XY坐标数据中，提取了25个运动变量，包括关键点的横向运动（10个变量）、关键点速度的变异系数（10个变量）、行走速度（1个变量）以及关键点角度的标准差（4个变量：颈部角度、肩胛角度、背部角度和髋部角度）。由于分类为评分3的奶牛数量有限，它们与评分2的奶牛合并为一个类别。随后，基于提取的运动变量，使用随机森林算法开发了一个三级运动评分分类模型（评分0、1和2 + 3）。使用重复留出法评估模型的性能，即将数据集随机分为80%用于训练和20%用于测试，重复10次。该模型的总体三类分类性能实现了加权kappa系数为0.72，以及受试者工作特征曲线下面积为0.89。基于在交叉验证中进行的变量重要性分析，确定后外侧运动、肩胛外侧运动、行走速度和尾根外侧运动对于预测运动评分至关重要。这些发现表明，基于计算机视觉的关键点检测技术对于从俯视2D视频中提取与运动评分相关的运动变量是有效的，并且基于新提取变量的机器学习分类模型具有对奶牛进行客观运动评分的潜力。
