Mantovani Raphael R, Menezes Guilherme L, Dórea João R R
Department of Animal and Dairy Sciences, University of Wisconsin-Madison, Madison, WI 53706.
Department of Biological Systems Engineering, University of Wisconsin-Madison, Madison, WI 53706.
JDS Commun. 2023 Nov 17;5(4):310-316. doi: 10.3168/jdsc.2023-0442. eCollection 2024 Jul.
Respiratory rate (RR) is commonly employed for identifying animals experiencing heat-stress conditions and respiratory diseases. Recent advancements in computer vision algorithms have enabled the estimation of the RR in dairy cows through image-based approaches, with a primary focus on standing positions, thermal imaging, and deep learning techniques. In this study, our objective was to develop a system capable of accurately predicting the RR of lying Holstein cows under unrestrained conditions using red, green, and blue (RGB) and infrared (IR) night vision images. Thirty lactating cows were continuously recorded for 12 h per day over a 3-d period, capturing at least one 30-s video segment of each cow during lying time. A total of 95 videos were manually annotated with rectangular bounding boxes encompassing the flank area (region of interest; ROI) of the lying cows. For future applications, we trained a model for ROI identification using YOLOv8 to avoid manual annotations. The observed RR was determined by visual counting of breaths in each video. To predict the RR, we devised an image processing pipeline involving (1) capturing the ROI for the entire video, (2) reshaping the pixel intensity of each image channel into a 2-dimensional object and calculating its per-frame mean, (3) applying fast Fourier transform (FFT) to the average pixel intensity vector, (4) filtering frequencies specifically associated with respiratory movements, and (5) executing inverse FFT on the denoized data and identifying peaks on the resulting plot, with the count of peaks serving as the predicted RR per minute. The evaluation metrics, root mean squared error of prediction (RMSEP) and R, yielded values of 8.3 breaths/min (17.1% of the mean RR) and 0.77, respectively. To further validate the method, an additional dataset comprising preweaning dairy calves was used, consisting of 42 observations from 25 calves. The RMSEP and R values for this dataset were 13.0 breaths/min and 0.73, respectively. The model trained to identify the ROI exhibited a precision of 100%, a recall of 71.8%, and an score of 83.6% for bounding box detection. These are promising results for the implementation of this pipeline in future studies. The application of FFT to signals acquired from both RGB and IR images proved to be an effective and accurate method for computing the RR of cows in unrestrained conditions.
呼吸频率（RR）通常用于识别处于热应激状态和患有呼吸系统疾病的动物。计算机视觉算法的最新进展使得通过基于图像的方法来估计奶牛的RR成为可能，主要集中在站立姿势、热成像和深度学习技术上。在本研究中，我们的目标是开发一种系统，该系统能够使用红、绿、蓝（RGB）和红外（IR）夜视图像在不受限制的条件下准确预测躺卧的荷斯坦奶牛的RR。在3天的时间里，每天对30头泌乳奶牛连续记录12小时，在每头奶牛躺卧期间至少捕捉一个30秒的视频片段。总共95个视频被手动标注了围绕躺卧奶牛侧腹区域（感兴趣区域；ROI）的矩形边界框。为了未来的应用，我们使用YOLOv8训练了一个用于ROI识别的模型，以避免手动标注。通过视觉计数每个视频中的呼吸次数来确定观察到的RR。为了预测RR，我们设计了一个图像处理管道，包括（1）捕捉整个视频的ROI，（2）将每个图像通道的像素强度重塑为二维对象并计算其每帧均值，（3）对平均像素强度向量应用快速傅里叶变换（FFT），（4）过滤与呼吸运动特别相关的频率，以及（5）对去噪后的数据执行逆FFT并在得到的图上识别峰值，峰值的计数作为每分钟预测的RR。评估指标，预测的均方根误差（RMSEP）和R，分别产生了8.3次呼吸/分钟（平均RR的17.1%）和0.77的值。为了进一步验证该方法，使用了一个包含断奶前奶牛犊的额外数据集，该数据集由来自25头犊牛的42次观察组成。该数据集的RMSEP和R值分别为13.0次呼吸/分钟和0.73。训练用于识别ROI的模型在边界框检测方面表现出100%的精度、71.8%的召回率和83.6%的F1分数。这些结果对于该管道在未来研究中的实施是很有前景的。将FFT应用于从RGB和IR图像获取的信号被证明是一种在不受限制的条件下计算奶牛RR的有效且准确的方法。
