Zou Jiajun, Wan Zhiping, Wang Feng, Ye Shitong, Liu Shaojiang
School of Information and Intelligence Engineering, Guangzhou Xinhua University, Dongguan, 523133, China.
Artificial Intelligence Institute of Guangzhou Huashang College, Guangzhou, 511300, China.
Sci Rep. 2025 Aug 14;15(1):29899. doi: 10.1038/s41598-025-15162-x.
With the rapid development of intelligent transportation systems, the challenge of achieving efficient and accurate multimodal traffic data transmission and collaborative processing in complex network environments with bandwidth limitations, signal interference, and high concurrency has become a key issue that needs to be addressed. This paper proposes a Self-supervised Multi-modal and Reinforcement learning-based Traffic data semantic collaboration Transmission mechanism (SMART), aiming to optimize the transmission efficiency and robustness of multimodal data through a combination of self-supervised learning and reinforcement learning. The sending end employs a self-supervised conditional variational autoencoder and Transformer-DRL-based dynamic semantic compression strategy to intelligently filter and transmit the most core semantic information from video, radar, and LiDAR data. The receiving end combines Transformer and graph neural networks for deep decoding and feature fusion of multimodal data, while also using reinforcement learning self-supervised multi-task optimization engine to collaboratively enhance multiple task scenarios (such as traffic accident detection and vehicle behavior recognition). Experimental results show that SMART significantly outperforms traditional methods in low signal-to-noise ratio, high packet loss rate, and large-scale concurrency environments, excelling in key indicators such as semantic similarity, transmission efficiency, robustness, and end-to-end latency, demonstrating its effectiveness and innovation in smart transportation scenarios.
随着智能交通系统的快速发展，在存在带宽限制、信号干扰和高并发的复杂网络环境中实现高效准确的多模态交通数据传输与协同处理面临的挑战，已成为一个亟待解决的关键问题。本文提出了一种基于自监督多模态和强化学习的交通数据语义协作传输机制（SMART），旨在通过自监督学习和强化学习相结合的方式，优化多模态数据的传输效率和鲁棒性。发送端采用自监督条件变分自编码器和基于Transformer-DRL的动态语义压缩策略，从视频、雷达和激光雷达数据中智能筛选并传输最核心的语义信息。接收端将Transformer和图神经网络相结合，对多模态数据进行深度解码和特征融合，同时还使用强化学习自监督多任务优化引擎，协同增强多种任务场景（如交通事故检测和车辆行为识别）。实验结果表明，在低信噪比、高丢包率和大规模并发环境中，SMART显著优于传统方法，在语义相似度、传输效率、鲁棒性和端到端延迟等关键指标上表现出色，证明了其在智能交通场景中的有效性和创新性。
