IEEE Trans Pattern Anal Mach Intell. 2023 Jun;45(6):6688-6702. doi: 10.1109/TPAMI.2020.3008558. Epub 2023 May 5.
Pedestrians and drivers are expected to safely navigate complex urban environments along with several non cooperating agents. Autonomous vehicles will soon replicate this capability. Each agent acquires a representation of the world from an egocentric perspective and must make decisions ensuring safety for itself and others. This requires to predict motion patterns of observed agents for a far enough future. In this paper we propose MANTRA, a model that exploits memory augmented networks to effectively predict multiple trajectories of other agents, observed from an egocentric perspective. Our model stores observations in memory and uses trained controllers to write meaningful pattern encodings and read trajectories that are most likely to occur in future. We show that our method is able to natively perform multi-modal trajectory prediction obtaining state-of-the art results on four datasets. Moreover, thanks to the non-parametric nature of the memory module, we show how once trained our system can continuously improve by ingesting novel patterns.
行人和驾驶员需要在复杂的城市环境中与多个不合作的代理安全导航。自动驾驶汽车很快将具备这种能力。每个代理都从自身视角获取对世界的表示，并必须做出决策，以确保自身和他人的安全。这需要预测观测到的代理在足够远的未来的运动模式。在本文中，我们提出了 MANTRA，这是一种利用记忆增强网络有效预测从自身视角观察到的其他代理的多个轨迹的模型。我们的模型将观察结果存储在内存中，并使用训练好的控制器来编写有意义的模式编码，并读取未来最有可能出现的轨迹。我们表明，我们的方法能够进行原生的多模态轨迹预测，在四个数据集上取得了最先进的结果。此外，由于内存模块的非参数性质，我们展示了一旦经过训练，我们的系统如何通过摄取新的模式来持续改进。
