Air Force Engineering University, Xi'an, China.
Southeast University, Nanjing, China.
Comput Intell Neurosci. 2022 Mar 3;2022:1477078. doi: 10.1155/2022/1477078. eCollection 2022.
With the rapid development of unmanned combat aerial vehicle (UCAV)-related technologies, UCAVs are playing an increasingly important role in military operations. It has become an inevitable trend in the development of future air combat battlefields that UCAVs complete air combat tasks independently to acquire air superiority. In this paper, the UCAV maneuver decision problem in continuous action space is studied based on the deep reinforcement learning strategy optimization method. The UCAV platform model of continuous action space was established. Focusing on the problem of insufficient exploration ability of Ornstein-Uhlenbeck (OU) exploration strategy in the deep deterministic policy gradient (DDPG) algorithm, a heuristic DDPG algorithm was proposed by introducing heuristic exploration strategy, and then a UCAV air combat maneuver decision method based on a heuristic DDPG algorithm is proposed. The superior performance of the algorithm is verified by comparison with different algorithms in the test environment, and the effectiveness of the decision method is verified by simulation of air combat tasks with different difficulty and attack modes.
随着无人机（UCAV）相关技术的快速发展，UCAV 在军事行动中发挥着越来越重要的作用。UCAV 独立完成空战任务以获得空中优势，这已经成为未来空战战场发展的必然趋势。本文基于深度强化学习策略优化方法研究了连续动作空间中的 UCAV 机动决策问题。建立了连续动作空间的 UCAV 平台模型。针对深度确定性策略梯度（DDPG）算法中 Ornstein-Uhlenbeck（OU）探索策略探索能力不足的问题，通过引入启发式探索策略，提出了一种启发式 DDPG 算法，然后提出了一种基于启发式 DDPG 算法的 UCAV 空战机动决策方法。通过在测试环境中与不同算法进行比较，验证了算法的优越性能，并通过模拟不同难度和攻击模式的空战任务验证了决策方法的有效性。
