Maslen Ben, Popovic Gordana, Wang Dadong, Jansen Andrew, Warton David
School of Mathematics and Statistics University of New South Wales Sydney New South Wales Australia.
Evolution and Ecology Research Centre University of New South Wales Sydney New South Wales Australia.
Ecol Evol. 2025 Aug 19;15(8):e71996. doi: 10.1002/ece3.71996. eCollection 2025 Aug.
The rise of AI has seen an explosion in the use of deep learning methods that automate the analysis of image and video data, saving ecologists vast amounts of time and resources. Ecological imagery poses unique challenges; however, with cryptic species struggling to be detected among poor visibility and diverse environments. We propose leveraging movement information to attempt to improve the predictions produced by a high-performing object detection algorithm. Frame differencing, background subtraction, optical flow and multi-object tracking are trialed on four diverse datasets containing over 35,000 annotated images sourced from terrestrial, marine and freshwater habitats. We find that leveraging movement information is useful for smaller sized studies and rarer species, however is not needed for well annotated studies (> 400 annotations per class). Out of the methods that utilise movement, we find that a simple 'differencing' of neighbouring frames generally performed the best, whilst attempting to track taxa to boost prediction scores performed poorly. Other studies in this area tend to focus only on 1-2 datasets and a single method that utilises movement information, making it difficult for ecologists to generalise results. Our study provides key lessons for ecologists to determine whether it is useful to incorporate methods that leverage movement information when attempting to automatically predict taxa. We offer straightforward code for practical implementation via our GitHub repository, BenMaslen/MCD, along with an evaluation benchmark dataset called 'Tassie BRUV' that can be accessed from the Dryad public repository https://doi.org/10.5061/dryad.sbcc2frf7.
随着人工智能的兴起，深度学习方法在图像和视频数据分析自动化中的应用呈爆炸式增长，为生态学家节省了大量时间和资源。然而，生态图像带来了独特的挑战，在能见度差和环境多样的情况下，隐秘物种难以被发现。我们建议利用运动信息来尝试改进高性能目标检测算法产生的预测。我们在四个不同的数据集上对帧差法、背景减法、光流法和多目标跟踪法进行了试验，这些数据集包含来自陆地、海洋和淡水栖息地的35000多张带注释的图像。我们发现，利用运动信息对规模较小的研究和较罕见的物种有用，但对于注释完善的研究（每类>400个注释）则不需要。在利用运动的方法中，我们发现相邻帧的简单“差分”通常表现最佳，而试图跟踪分类单元以提高预测分数的效果不佳。该领域的其他研究往往只关注1-2个数据集和一种利用运动信息的单一方法，这使得生态学家难以将结果推广。我们的研究为生态学家提供了关键经验，以确定在尝试自动预测分类单元时纳入利用运动信息的方法是否有用。我们通过GitHub仓库BenMaslen/MCD提供了用于实际实现的简单代码，以及一个名为“塔斯马尼亚BRUV”的评估基准数据集，该数据集可从Dryad公共仓库https://doi.org/10.5061/dryad.sbcc2frf7访问。
