Department of Computer and Information Science, Korea University, Sejong City 30019, Republic of Korea.
Department of Computer Convergence Software, Sejong Campus, Korea University, Sejong City 30019, Republic of Korea.
Sensors (Basel). 2023 Mar 7;23(6):2892. doi: 10.3390/s23062892.
The popularity of dogs has been increasing owing to factors such as the physical and mental health benefits associated with raising them. While owners care about their dogs' health and welfare, it is difficult for them to assess these, and frequent veterinary checkups represent a growing financial burden. In this study, we propose a behavior-based video summarization and visualization system for monitoring a dog's behavioral patterns to help assess its health and welfare. The system proceeds in four modules: (1) a video data collection and preprocessing module; (2) an object detection-based module for retrieving image sequences where the dog is alone and cropping them to reduce background noise; (3) a dog behavior recognition module using two-stream EfficientNetV2 to extract appearance and motion features from the cropped images and their respective optical flow, followed by a long short-term memory (LSTM) model to recognize the dog's behaviors; and (4) a summarization and visualization module to provide effective visual summaries of the dog's location and behavior information to help assess and understand its health and welfare. The experimental results show that the system achieved an average F1 score of 0.955 for behavior recognition, with an execution time allowing real-time processing, while the summarization and visualization results demonstrate how the system can help owners assess and understand their dog's health and welfare.
由于与饲养相关的身心健康益处等因素，狗的受欢迎程度一直在增加。虽然主人关心他们的狗的健康和福利，但他们很难评估这些，频繁的兽医检查代表着越来越大的经济负担。在这项研究中，我们提出了一种基于行为的视频摘要和可视化系统，用于监测狗的行为模式，以帮助评估其健康和福利。该系统分为四个模块：（1）视频数据采集和预处理模块；（2）基于对象检测的模块，用于检索狗独自出现的图像序列，并对其进行裁剪以减少背景噪声；（3）使用双流 EfficientNetV2 的狗行为识别模块，从裁剪后的图像及其各自的光流中提取外观和运动特征，然后使用长短期记忆（LSTM）模型识别狗的行为；（4）摘要和可视化模块，为狗的位置和行为信息提供有效的可视化摘要，以帮助评估和理解其健康和福利。实验结果表明，该系统在行为识别方面的平均 F1 得分为 0.955，执行时间允许实时处理，而摘要和可视化结果展示了系统如何帮助主人评估和理解他们的狗的健康和福利。
