Li Lin, Hu Zeyu, Yang Xubo
Shanghai International Automobile City (Group) Co., Ltd., Shanghai, 201805 China.
School of Software, School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University, Shanghai, 200240 China.
J Shanghai Jiaotong Univ Sci. 2021;26(5):587-597. doi: 10.1007/s12204-021-2348-7. Epub 2021 Oct 28.
Analyzing a vehicle's abnormal behavior in surveillance videos is a challenging field, mainly due to the wide variety of anomaly cases and the complexity of surveillance videos. In this study, a novel intelligent vehicle behavior analysis framework based on a digital twin is proposed. First, detecting vehicles based on deep learning is implemented, and Kalman filtering and feature matching are used to track vehicles. Subsequently, the tracked vehicle is mapped to a digital-twin virtual scene developed in the Unity game engine, and each vehicle's behavior is tested according to the customized detection conditions set up in the scene. The stored behavior data can be used to reconstruct the scene again in Unity for a secondary analysis. The experimental results using real videos from traffic cameras illustrate that the detection rate of the proposed framework is close to that of the state-of-the-art abnormal event detection systems. In addition, the implementation and analysis process show the usability, generalization, and effectiveness of the proposed framework.
在监控视频中分析车辆的异常行为是一个具有挑战性的领域，主要是因为异常情况种类繁多且监控视频复杂。在本研究中，提出了一种基于数字孪生的新型智能车辆行为分析框架。首先，实现基于深度学习的车辆检测，并使用卡尔曼滤波和特征匹配来跟踪车辆。随后，将跟踪到的车辆映射到在Unity游戏引擎中开发的数字孪生虚拟场景中，并根据场景中设置的定制检测条件测试每辆车的行为。存储的行为数据可用于在Unity中再次重建场景以进行二次分析。使用交通摄像头的真实视频进行的实验结果表明，所提出框架的检测率接近最先进的异常事件检测系统。此外，实施和分析过程表明了所提出框架的可用性、通用性和有效性。
