Shu Xiulan, Lin Anping, Wen Xupeng
School of Intelligent Manufacturing Engineering, Zhanjiang University of Science and Technology, Zhanjiang 524000, China.
School of Physics and Electronic Electrical Engineering, Xiangnan University, Chenzhou 423000, China.
Sensors (Basel). 2024 Oct 18;24(20):6698. doi: 10.3390/s24206698.
With the rapid advancement of drone technology, the efficient distribution of drones has garnered significant attention. Central to this discourse is the energy consumption of drones, a critical metric for assessing energy-efficient distribution strategies. Accordingly, this study delves into the energy consumption factors affecting drone distribution. A primary challenge in drone distribution lies in devising optimal, energy-efficient routes for drones. However, traditional routing algorithms, predominantly heuristic-based, exhibit certain limitations. These algorithms often rely on heuristic rules and expert knowledge, which can constrain their ability to escape local optima. Motivated by these shortcomings, we propose a novel multi-agent deep reinforcement learning algorithm that integrates a drone energy consumption model, namely EMADRL. The EMADRL algorithm first formulates the drone routing problem within a multi-agent reinforcement learning framework. It subsequently designs a strategy network model comprising multiple agent networks, tailored to address the node adjacency and masking complexities typical of multi-depot vehicle routing problem. Training utilizes strategy gradient algorithms and attention mechanisms. Furthermore, local and sampling search strategies are introduced to enhance solution quality. Extensive experimentation demonstrates that EMADRL consistently achieves high-quality solutions swiftly. A comparative analysis against contemporary algorithms reveals EMADRL's superior energy efficiency, with average energy savings of 5.96% and maximum savings reaching 12.45%. Thus, this approach offers a promising new avenue for optimizing energy consumption in last-mile distribution scenarios.
随着无人机技术的迅速发展，无人机的高效配送已引起广泛关注。在这一讨论的核心是无人机的能量消耗，这是评估节能配送策略的关键指标。因此，本研究深入探讨了影响无人机配送的能量消耗因素。无人机配送的一个主要挑战在于为无人机设计最优的节能路线。然而，传统的路由算法主要基于启发式算法，存在一定的局限性。这些算法通常依赖于启发式规则和专家知识，这可能限制它们逃离局部最优的能力。受这些缺点的启发，我们提出了一种新颖的多智能体深度强化学习算法，该算法集成了无人机能量消耗模型，即EMADRL。EMADRL算法首先在多智能体强化学习框架内制定无人机路由问题。随后，它设计了一个由多个智能体网络组成的策略网络模型，以解决多仓库车辆路由问题中典型的节点邻接和掩码复杂性。训练使用策略梯度算法和注意力机制。此外，还引入了局部和采样搜索策略以提高解决方案质量。大量实验表明，EMADRL能够迅速持续地实现高质量的解决方案。与当代算法的对比分析表明，EMADRL具有更高的能源效率，平均节能5.96%，最大节能达到12.45%。因此，这种方法为优化最后一英里配送场景中的能量消耗提供了一条有前景的新途径。
