IEEE Trans Image Process. 2019 Mar;28(3):1133-1148. doi: 10.1109/TIP.2018.2875335. Epub 2018 Oct 10.
Automated recognition of mouse behaviors is crucial in studying psychiatric and neurologic diseases. To achieve this objective, it is very important to analyze the temporal dynamics of mouse behaviors. In particular, the change between mouse neighboring actions is swift in a short period. In this paper, we develop and implement a novel hidden Markov model (HMM) algorithm to describe the temporal characteristics of mouse behaviors. In particular, we here propose a hybrid deep learning architecture, where the first unsupervised layer relies on an advanced spatial-temporal segment Fisher vector encoding both visual and contextual features. Subsequent supervised layers based on our segment aggregate network are trained to estimate the state-dependent observation probabilities of the HMM. The proposed architecture shows the ability to discriminate between visually similar behaviors and results in high recognition rates with the strength of processing imbalanced mouse behavior datasets. Finally, we evaluate our approach using JHuang's and our own datasets, and the results show that our method outperforms other state-of-the-art approaches.
自动化识别鼠标行为在研究精神疾病和神经疾病方面至关重要。为了实现这一目标，分析鼠标行为的时间动态性非常重要。特别是，鼠标相邻动作之间的变化在短时间内非常迅速。在本文中，我们开发并实现了一种新颖的隐马尔可夫模型（HMM）算法来描述鼠标行为的时间特征。特别地，我们在这里提出了一种混合深度学习架构，其中第一个无监督层依赖于先进的时空分段 Fisher 向量，该向量同时编码视觉和上下文特征。基于我们的分段聚合网络的后续监督层被训练来估计 HMM 的状态相关观测概率。所提出的架构具有区分视觉相似行为的能力，并且在处理不平衡的鼠标行为数据集方面具有很高的识别率。最后，我们使用 JHuang 的数据集和我们自己的数据集来评估我们的方法，结果表明我们的方法优于其他最先进的方法。
