Liu Xiangyu, Tan Ying
IEEE Trans Cybern. 2022 Jan;52(1):252-264. doi: 10.1109/TCYB.2020.2979803. Epub 2022 Jan 11.
In multiagent reinforcement learning (MARL), it is crucial for each agent to model the relation with its neighbors. Existing approaches usually resort to concatenate the features of multiple neighbors, fixing the size and the identity of the inputs. But these settings are inflexible and unscalable. In this article, we propose an attentive relational encoder (ARE), which is a novel scalable feedforward neural module, to attentionally aggregate an arbitrary-sized neighboring feature set for state representation in the decentralized MARL. The ARE actively selects the relevant information from the neighboring agents and is permutation invariant, computationally efficient, and flexible to interactive multiagent systems. Our method consistently outperforms the latest competing decentralized MARL methods in several multiagent tasks. In particular, it shows strong cooperative performance in challenging StarCraft micromanagement tasks and achieves over a 96% winning rate against the most difficult noncheating built-in artificial intelligence bots.
在多智能体强化学习（MARL）中，每个智能体对其与邻居的关系进行建模至关重要。现有方法通常采用拼接多个邻居的特征，固定输入的大小和身份。但这些设置缺乏灵活性且不可扩展。在本文中，我们提出了一种注意力关系编码器（ARE），它是一种新颖的可扩展前馈神经模块，用于在分散式MARL中注意力聚合任意大小的相邻特征集以进行状态表示。ARE从相邻智能体中主动选择相关信息，具有排列不变性、计算效率高且对交互式多智能体系统具有灵活性。我们的方法在多个多智能体任务中始终优于最新的竞争分散式MARL方法。特别是，它在具有挑战性的星际争霸微观管理任务中表现出强大的合作性能，对阵最困难的非作弊内置人工智能机器人时胜率超过96%。
