Barthelemy Johan, Iqbal Umair, Qian Yan, Amirghasemi Mehrdad, Perez Pascal
Faculty of Engineering and Information Sciences, University of Wollongong, Wollongong, NSW 2522, Australia.
Centre for Geotechnical Science and Engineering, School of Engineering, University of Newcastle, Newcastle, NSW 2308, Australia.
Sensors (Basel). 2024 Dec 19;24(24):8102. doi: 10.3390/s24248102.
Public transportation systems play a vital role in modern cities, but they face growing security challenges, particularly related to incidents of violence. Detecting and responding to violence in real time is crucial for ensuring passenger safety and the smooth operation of these transport networks. To address this issue, we propose an advanced artificial intelligence (AI) solution for identifying unsafe behaviours in public transport. The proposed approach employs deep learning action recognition models and utilises technologies like NVIDIA DeepStream SDK, Amazon Web Services (AWS) DirectConnect, local edge computing server, ONNXRuntime and MQTT to accelerate the end-to-end pipeline. The solution captures video streams from remote train stations closed circuit television (CCTV) networks, processes the data in the cloud, applies the action recognition model, and transmits the results to a live web application. A temporal pyramid network (TPN) action recognition model was trained on a newly curated video dataset mixing open-source resources and live simulated trials to identify the unsafe behaviours. The base model was able to achieve a validation accuracy of 93% when trained using open-source dataset samples and was improved to 97% when live simulated dataset was included during the training. The developed AI system was deployed at Wollongong Train Station (NSW, Australia) and showcased impressive accuracy in detecting violence incidents during an 8-week test period, achieving a reliable false-positive (FP) rate of 23%. While the AI correctly identified 30 true-positive incidents, there were 6 cases of false negatives (FNs) where violence incidents were missed during the rainy weather suggesting more data in the training dataset related to bad weather. The AI model's continuous retraining capability ensures its adaptability to various real-world scenarios, making it a valuable tool for enhancing safety and the overall passenger experience in public transport settings.
公共交通系统在现代城市中发挥着至关重要的作用，但它们面临着日益严峻的安全挑战，尤其是与暴力事件相关的挑战。实时检测和应对暴力行为对于确保乘客安全和这些交通网络的顺畅运行至关重要。为了解决这个问题，我们提出了一种先进的人工智能（AI）解决方案，用于识别公共交通中的不安全行为。所提出的方法采用深度学习动作识别模型，并利用NVIDIA DeepStream SDK、亚马逊网络服务（AWS）DirectConnect、本地边缘计算服务器、ONNXRuntime和MQTT等技术来加速端到端流程。该解决方案从远程火车站的闭路电视（CCTV）网络捕获视频流，在云端处理数据，应用动作识别模型，并将结果传输到实时网络应用程序。一个时间金字塔网络（TPN）动作识别模型在一个新策划的视频数据集上进行了训练，该数据集混合了开源资源和实时模拟试验，以识别不安全行为。当使用开源数据集样本进行训练时，基础模型能够达到93%的验证准确率，当在训练期间纳入实时模拟数据集时，准确率提高到了97%。所开发的人工智能系统部署在卧龙岗火车站（新南威尔士州，澳大利亚），并在为期8周的测试期内展示了在检测暴力事件方面令人印象深刻的准确率，实现了23%的可靠误报率。虽然人工智能正确识别了30起真阳性事件，但在雨天有6例假阴性（FN）情况，即暴力事件被遗漏，这表明训练数据集中需要更多与恶劣天气相关的数据。人工智能模型的持续再训练能力确保了它对各种现实世界场景的适应性，使其成为提高公共交通环境中安全性和整体乘客体验的宝贵工具。
