Sclocco Alessio, Ong Shirlyn Jia Yun, Pyay Aung Sai Yan, Teseo Serafino
School of Biological Sciences, Nanyang Technological University, Singapore.
Netherlands eScience Center, Amsterdam, North Holland, The Netherlands.
R Soc Open Sci. 2021 Mar 31;8(3):202033. doi: 10.1098/rsos.202033.
Automatic video tracking has become a standard tool for investigating the social behaviour of insects. The recent integration of computer vision in tracking technologies will probably lead to fully automated behavioural pattern classification within the next few years. However, many current systems rely on offline data analysis and use computationally expensive techniques to track pre-recorded videos. To address this gap, we developed BACH (Behaviour Analysis maCHine), a software that performs video tracking of insect groups in real time. BACH uses object recognition via convolutional neural networks and identifies individually tagged insects via an existing matrix code recognition algorithm. We compared the tracking performances of BACH and a human observer (HO) across a series of short videos of ants moving in a two-dimensional arena. We found that BACH detected ant shapes only slightly worse than the HO. However, its matrix code-mediated identification of individual ants only attained human-comparable levels when ants moved relatively slowly, and fell when ants walked relatively fast. This happened because BACH had a relatively low efficiency in detecting matrix codes in blurry images of ants walking at high speeds. BACH needs to undergo hardware and software adjustments to overcome its present limits. Nevertheless, our study emphasizes the possibility of, and the need for, further integrating real-time data analysis into the study of animal behaviour. This will accelerate data generation, visualization and sharing, opening possibilities for conducting fully remote collaborative experiments.
自动视频跟踪已成为研究昆虫社会行为的标准工具。计算机视觉最近在跟踪技术中的整合，可能会在未来几年内实现完全自动化的行为模式分类。然而，许多当前系统依赖离线数据分析，并使用计算成本高昂的技术来跟踪预先录制的视频。为了弥补这一差距，我们开发了BACH（行为分析机器），这是一款能够实时对昆虫群体进行视频跟踪的软件。BACH通过卷积神经网络进行目标识别，并通过现有的矩阵码识别算法识别个体标记的昆虫。我们在一系列蚂蚁在二维场地中移动的短视频中，比较了BACH和人类观察者（HO）的跟踪性能。我们发现，BACH检测蚂蚁形状的能力仅略逊于HO。然而，其通过矩阵码对个体蚂蚁的识别，只有在蚂蚁移动相对较慢时才能达到与人类相当的水平，而在蚂蚁行走较快时则会下降。出现这种情况是因为BACH在检测高速行走蚂蚁的模糊图像中的矩阵码时效率相对较低。BACH需要进行硬件和软件调整以克服其当前的局限性。尽管如此，我们的研究强调了将实时数据分析进一步整合到动物行为研究中的可能性和必要性。这将加速数据的生成、可视化和共享，为进行完全远程的协作实验开辟可能性。
