Li Jing, Huang Qingwang, Du Yingjun, Zhen Xiantong, Chen Shengyong, Shao Ling
IEEE Trans Image Process. 2022;31:275-286. doi: 10.1109/TIP.2021.3130545. Epub 2021 Dec 7.
Abnormal crowd behavior detection has recently attracted increasing attention due to its wide applications in computer vision research areas. However, it is still an extremely challenging task due to the great variability of abnormal behavior coupled with huge ambiguity and uncertainty of video contents. To tackle these challenges, we propose a new probabilistic framework named variational abnormal behavior detection (VABD), which can detect abnormal crowd behavior in video sequences. We make three major contributions: (1) We develop a new probabilistic latent variable model that combines the strengths of the U-Net and conditional variational auto-encoder, which also are the backbone of our model; (2) We propose a motion loss based on an optical flow network to impose the motion consistency of generated video frames and input video frames; (3) We embed a Wasserstein generative adversarial network at the end of the backbone network to enhance the framework performance. VABD can accurately discriminate abnormal video frames from video sequences. Experimental results on UCSD, CUHK Avenue, IITB-Corridor, and ShanghaiTech datasets show that VABD outperforms the state-of-the-art algorithms on abnormal crowd behavior detection. Without data augmentation, our VABD achieves 72.24% in terms of AUC on IITB-Corridor, which surpasses the state-of-the-art methods by nearly 5%.
由于异常人群行为检测在计算机视觉研究领域的广泛应用，近年来它受到了越来越多的关注。然而，由于异常行为的巨大变异性以及视频内容的高度模糊性和不确定性，它仍然是一项极具挑战性的任务。为了应对这些挑战，我们提出了一种名为变分异常行为检测（VABD）的新概率框架，该框架可以检测视频序列中的异常人群行为。我们做出了三项主要贡献：（1）我们开发了一种新的概率潜在变量模型，该模型结合了U-Net和条件变分自动编码器的优势，它们也是我们模型的主干；（2）我们基于光流网络提出了一种运动损失，以强制生成的视频帧和输入视频帧的运动一致性；（3）我们在主干网络末尾嵌入了一个Wasserstein生成对抗网络，以提高框架性能。VABD可以准确地从视频序列中区分出异常视频帧。在UCSD、香港中文大学林荫道、印度理工学院孟买分校走廊和上海科技数据集上的实验结果表明，VABD在异常人群行为检测方面优于现有算法。在没有数据增强的情况下，我们的VABD在印度理工学院孟买分校走廊数据集上的AUC达到了72.24%，比现有最先进方法高出近5%。
