Wang Huan, Wang Jintao
College of Artificial Intelligence and Automation, Hohai University, Changzhou, 213200, China.
College of information and Network Engineering, Anhui Science and Technology University, Chuzhou, 233030, China.
Sci Rep. 2024 Feb 23;14(1):4458. doi: 10.1038/s41598-024-54938-5.
In the realm of air combat, autonomous decision-making in regard to Unmanned Aerial Vehicle (UAV) has emerged as a critical force. However, prevailing autonomous decision-making algorithms in this domain predominantly rely on rule-based methods, proving challenging to design and implement optimal solutions in complex multi-UAV combat environments. This paper proposes a novel approach to multi-UAV air combat decision-making utilizing hierarchical reinforcement learning. First, a hierarchical decision-making network is designed based on tactical action types to streamline the complexity of the maneuver decision-making space. Second, the high-quality combat experience gained from training is decomposed, with the aim of augmenting the quantity of valuable experiences and alleviating the intricacies of strategy learning. Finally, the performance of the algorithm is validated using the advanced UAV simulation platform JSBSim. Through comparisons with various baseline algorithms, our experiments demonstrate the superior performance of the proposed method in both even and disadvantaged air combat environments.
在空战领域，无人机的自主决策已成为一股关键力量。然而，该领域现有的自主决策算法主要依赖基于规则的方法，在复杂的多无人机作战环境中设计和实施最优解决方案具有挑战性。本文提出了一种利用分层强化学习的多无人机空战决策新方法。首先，基于战术行动类型设计了一个分层决策网络，以简化机动决策空间的复杂性。其次，对训练中获得的高质量作战经验进行分解，目的是增加有价值经验的数量并减轻策略学习的复杂性。最后，使用先进的无人机仿真平台JSBSim验证了算法的性能。通过与各种基线算法的比较，我们的实验证明了所提方法在均势和劣势空战环境中的优越性能。
