Department of Computing & Mathematical Sciences, California Institute of Technology, Pasadena, United States.
Division of Biology and Biological Engineering 156-29, TianQiao and Chrissy Chen Institute for Neuroscience, California Institute of Technology, Pasadena, United States.
Elife. 2021 Nov 30;10:e63720. doi: 10.7554/eLife.63720.
The study of naturalistic social behavior requires quantification of animals' interactions. This is generally done through manual annotation-a highly time-consuming and tedious process. Recent advances in computer vision enable tracking the pose (posture) of freely behaving animals. However, automatically and accurately classifying complex social behaviors remains technically challenging. We introduce the Mouse Action Recognition System (MARS), an automated pipeline for pose estimation and behavior quantification in pairs of freely interacting mice. We compare MARS's annotations to human annotations and find that MARS's pose estimation and behavior classification achieve human-level performance. We also release the pose and annotation datasets used to train MARS to serve as community benchmarks and resources. Finally, we introduce the Behavior Ensemble and Neural Trajectory Observatory (BENTO), a graphical user interface for analysis of multimodal neuroscience datasets. Together, MARS and BENTO provide an end-to-end pipeline for behavior data extraction and analysis in a package that is user-friendly and easily modifiable.
自然主义社会行为的研究需要对动物的互动进行量化。这通常通过手动注释来完成，这是一个非常耗时和乏味的过程。计算机视觉的最新进展使得可以跟踪自由行为动物的姿势（姿势）。然而，自动准确地对复杂的社会行为进行分类仍然具有技术挑战性。我们引入了 Mouse Action Recognition System（MARS），这是一个用于对自由互动的老鼠对进行姿势估计和行为量化的自动化管道。我们将 MARS 的注释与人工注释进行比较，发现 MARS 的姿势估计和行为分类达到了人类水平。我们还发布了用于训练 MARS 的姿势和注释数据集，以作为社区基准和资源。最后，我们引入了 Behavior Ensemble and Neural Trajectory Observatory（BENTO），这是一个用于分析多模态神经科学数据集的图形用户界面。总之，MARS 和 BENTO 提供了一个端到端的行为数据提取和分析管道，该管道在一个用户友好且易于修改的包中。
