Ren Shunli, Chen Siheng, Zhang Wenjun
Cooperative Medianet Innovation Center, Shanghai Jiao Tong University, Shanghai 200240, China.
Sensors (Basel). 2024 Sep 27;24(19):6263. doi: 10.3390/s24196263.
Collaboration among road agents, such as connected autonomous vehicles and roadside units, enhances driving performance by enabling the exchange of valuable information. However, existing collaboration methods predominantly focus on perception tasks and rely on single-frame static information sharing, which limits the effective exchange of temporal data and hinders broader applications of collaboration. To address this challenge, we propose CoPnP, a novel collaborative joint perception and prediction system, whose core innovation is to realize multi-frame spatial-temporal information sharing. To achieve effective and communication-efficient information sharing, two novel designs are proposed: (1) a task-oriented spatial-temporal information-refinement model, which filters redundant and noisy multi-frame features into concise representations; (2) a spatial-temporal importance-aware feature-fusion model, which comprehensively fuses features from various agents. The proposed CoPnP expands the benefits of collaboration among road agents to the joint perception and prediction task. The experimental results demonstrate that CoPnP outperforms existing state-of-the-art collaboration methods, achieving a significant performance-communication trade-off and yielding up to 11.51%/10.34% Intersection over union and 12.31%/10.96% video panoptic quality gains over single-agent PnP on the OPV2V/V2XSet datasets.
诸如联网自动驾驶车辆和路边单元等道路智能体之间的协作，通过实现有价值信息的交换来提升驾驶性能。然而，现有的协作方法主要侧重于感知任务，且依赖单帧静态信息共享，这限制了时态数据的有效交换，并阻碍了协作的更广泛应用。为应对这一挑战，我们提出了CoPnP，这是一种新颖的协作式联合感知与预测系统，其核心创新在于实现多帧时空信息共享。为实现高效且通信高效的信息共享，我们提出了两种新颖的设计：（1）一种面向任务的时空信息细化模型，该模型将冗余和有噪声的多帧特征过滤为简洁的表示形式；（2）一种时空重要性感知特征融合模型，该模型全面融合来自各种智能体的特征。所提出的CoPnP将道路智能体之间协作的优势扩展到了联合感知与预测任务。实验结果表明，CoPnP优于现有的先进协作方法，在OPV2V/V2XSet数据集上，相较于单智能体PnP，实现了显著的性能-通信权衡，在交并比上提升了高达11.51%/10.34%，在视频全景质量上提升了12.31%/10.96%。
