Graduate School of Engineering, University of Miyazaki, Miyazaki, 889-2192, Japan.
Honkawa Ranch, Oita, 877-0056, Japan.
Sci Rep. 2023 Oct 13;13(1):17423. doi: 10.1038/s41598-023-44669-4.
In modern cattle farm management systems, video-based monitoring has become important in analyzing the high-level behavior of cattle for monitoring their health and predicting calving for providing timely assistance. Conventionally, sensors have been used for detecting and tracking their activities. As the body-attached sensors cause stress, video cameras can be used as an alternative. However, identifying and tracking individual cattle can be difficult, especially for black and brown varieties that are so similar in appearance. Therefore, we propose a new method of using video cameras for recognizing cattle and tracking their whereabouts. In our approach, we applied a combination of deep learning and image processing techniques to build a robust system. The proposed system processes images in separate stages, namely data pre-processing, cow detection, and cow tracking. Cow detection is performed using a popular instance segmentation network. In the cow tracking stage, for successively associating each cow with the corresponding one in the next frame, we employed the following three features: cow location, appearance features, as well as recent features of the cow region. In doing so, we simply exploited the distance between two gravity center locations of the cow regions. As color and texture suitably define the appearance of an object, we analyze the most appropriate color space to extract color moment features and use a Co-occurrence Matrix (CM) for textural representation. Deep features are extracted from recent cow images using a Convolutional Neural Network (CNN features) and are also jointly applied in the tracking process to boost system performance. We also proposed a robust Multiple Object Tracking (MOT) algorithm for cow tracking by employing multiple features from the cow region. The experimental results proved that our proposed system could handle the problems of MOT and produce reliable performance.
在现代奶牛场管理系统中，基于视频的监测对于分析奶牛的高级行为以监测其健康状况和预测分娩以提供及时的帮助变得非常重要。传统上，使用传感器来检测和跟踪它们的活动。由于身体附着的传感器会引起压力，因此可以使用摄像头作为替代。然而，识别和跟踪个体奶牛可能很困难，特别是对于外观非常相似的黑色和棕色品种。因此，我们提出了一种使用摄像头识别奶牛并跟踪其行踪的新方法。在我们的方法中，我们应用了深度学习和图像处理技术的组合来构建一个强大的系统。该系统在单独的阶段处理图像，即数据预处理、奶牛检测和奶牛跟踪。奶牛检测使用流行的实例分割网络进行。在奶牛跟踪阶段，为了成功地将每头奶牛与下一帧中的对应奶牛相关联，我们采用了以下三个特征：奶牛位置、外观特征以及奶牛区域的最近特征。为此，我们只是利用了奶牛区域的两个重心位置之间的距离。由于颜色和纹理恰当地定义了物体的外观，我们分析了最合适的颜色空间来提取颜色矩特征，并使用共生矩阵 (CM) 进行纹理表示。使用卷积神经网络 (CNN 特征) 从最近的奶牛图像中提取深度特征，并在跟踪过程中联合应用，以提高系统性能。我们还提出了一种稳健的多目标跟踪 (MOT) 算法，通过使用奶牛区域的多个特征来进行奶牛跟踪。实验结果证明，我们提出的系统可以处理 MOT 问题并产生可靠的性能。
