Liu Qi, Li Xueyuan, Tang Yujie, Gao Xin, Yang Fan, Li Zirui
School of Mechanical Engineering, Beijing Institute of Technology, Beijing 100811, China.
Faculty of Computer Science, Dalhousie University, Halifax, NS B3H 4R2, Canada.
Sensors (Basel). 2023 Oct 3;23(19):8229. doi: 10.3390/s23198229.
The proper functioning of connected and autonomous vehicles (CAVs) is crucial for the safety and efficiency of future intelligent transport systems. Meanwhile, transitioning to fully autonomous driving requires a long period of mixed autonomy traffic, including both CAVs and human-driven vehicles. Thus, collaborative decision-making technology for CAVs is essential to generate appropriate driving behaviors to enhance the safety and efficiency of mixed autonomy traffic. In recent years, deep reinforcement learning (DRL) methods have become an efficient way in solving decision-making problems. However, with the development of computing technology, graph reinforcement learning (GRL) methods have gradually demonstrated the large potential to further improve the decision-making performance of CAVs, especially in the area of accurately representing the mutual effects of vehicles and modeling dynamic traffic environments. To facilitate the development of GRL-based methods for autonomous driving, this paper proposes a review of GRL-based methods for the decision-making technologies of CAVs. Firstly, a generic GRL framework is proposed in the beginning to gain an overall understanding of the decision-making technology. Then, the GRL-based decision-making technologies are reviewed from the perspective of the construction methods of mixed autonomy traffic, methods for graph representation of the driving environment, and related works about graph neural networks (GNN) and DRL in the field of decision-making for autonomous driving. Moreover, validation methods are summarized to provide an efficient way to verify the performance of decision-making methods. Finally, challenges and future research directions of GRL-based decision-making methods are summarized.
联网和自动驾驶车辆（CAV）的正常运行对于未来智能交通系统的安全和效率至关重要。与此同时，向完全自动驾驶的过渡需要一段很长的混合自主交通时期，包括CAV和人类驾驶车辆。因此，CAV的协同决策技术对于产生适当的驾驶行为以提高混合自主交通的安全性和效率至关重要。近年来，深度强化学习（DRL）方法已成为解决决策问题的一种有效方式。然而，随着计算技术的发展，图强化学习（GRL）方法已逐渐显示出进一步提高CAV决策性能的巨大潜力，特别是在准确表示车辆相互影响和对动态交通环境建模方面。为了促进基于GRL的自动驾驶方法的发展，本文对基于GRL的CAV决策技术进行了综述。首先，在开头提出了一个通用的GRL框架，以全面了解决策技术。然后，从混合自主交通的构建方法、驾驶环境的图表示方法以及自动驾驶决策领域中关于图神经网络（GNN）和DRL的相关工作等角度对基于GRL的决策技术进行了综述。此外，总结了验证方法，以提供一种验证决策方法性能的有效途径。最后，总结了基于GRL的决策方法面临的挑战和未来研究方向。
