Robinson Isaac, Glidden-Handgis George, Panchal Neekesh, Insel Nathan, Wheeler Travis J
University of Arizona.
Wilfrid Laurier University.
bioRxiv. 2025 Aug 26:2025.08.11.669786. doi: 10.1101/2025.08.11.669786.
Recent advances in computer vision have enabled the development of automated animal behavior observation tools. Several software packages currently exist for concurrently tracking pose in multiple animals; however, existing tools still face challenges in maintaining animal identities across frames and can demand extensive human oversight and editing. Here we report on DIPLOMAT, a Deep learning-based, Identity-Preserving, Labeled-Object Multi-Animal Tracker, which implements automated algorithms to improve identity continuity, supplemented by an efficient human interface to help eliminate remaining errors. DIPLOMAT is designed to perform multi-animal tracking by building on the per-frame pose prediction models of two state-of-the-art tools, DeepLabCut and SLEAP, applying novel methods to tolerate occlusion and preserve animal identity across frames. Notable features include leveraging model-derived positional probabilities to compute independent maximum probability traces across frames of a video, use of video-specific skeletal constraints, and implementation of an efficient user interface for resolving errors. On the MABe mouse tracking benchmark, automated tracking with DIPLOMAT reduces body identity swaps by 75%, while remaining errors are easily eradicated with manual correction.
计算机视觉领域的最新进展推动了自动动物行为观察工具的发展。目前有几个软件包可用于同时跟踪多只动物的姿态；然而，现有工具在跨帧保持动物身份方面仍面临挑战，并且可能需要大量人工监督和编辑。在此，我们报告了DIPLOMAT，一种基于深度学习的、保留身份的、标记对象多动物跟踪器，它实现了自动算法以提高身份连续性，并辅以高效的人机界面来帮助消除剩余错误。DIPLOMAT旨在通过基于两个最先进工具DeepLabCut和SLEAP的逐帧姿态预测模型进行构建，应用新颖方法来容忍遮挡并跨帧保留动物身份，从而执行多动物跟踪。显著特征包括利用模型衍生的位置概率来计算视频各帧之间独立的最大概率轨迹、使用特定于视频的骨骼约束以及实现用于解决错误的高效用户界面。在MABe小鼠跟踪基准测试中，使用DIPLOMAT进行自动跟踪可将身体身份交换减少75%，而剩余错误通过人工校正很容易消除。
