Zhang Wenzhi, He Li, Wang Hongwei, Yuan Liang, Xiao Wendong
School of Mechanical Engineering, Xinjiang University, Urumqi 830046, China.
School of Information Science and Technology, Beijing University of Chemical Technology, Beijing 100029, China.
Entropy (Basel). 2023 Jun 30;25(7):1007. doi: 10.3390/e25071007.
Visual navigation based on deep reinforcement learning requires a large amount of interaction with the environment, and due to the reward sparsity, it requires a large amount of training time and computational resources. In this paper, we focus on sample efficiency and navigation performance and propose a framework for visual navigation based on multiple self-supervised auxiliary tasks. Specifically, we present an LSTM-based dynamics model and an attention-based image-reconstruction model as auxiliary tasks. These self-supervised auxiliary tasks enable agents to learn navigation strategies directly from the original high-dimensional images without relying on ResNet features by constructing latent representation learning. Experimental results show that without manually designed features and prior demonstrations, our method significantly improves the training efficiency and outperforms the baseline algorithms on the simulator and real-world image datasets.
基于深度强化学习的视觉导航需要与环境进行大量交互，并且由于奖励稀疏性，需要大量的训练时间和计算资源。在本文中，我们关注样本效率和导航性能，并提出了一个基于多个自监督辅助任务的视觉导航框架。具体来说，我们提出了基于长短期记忆网络（LSTM）的动力学模型和基于注意力的图像重建模型作为辅助任务。这些自监督辅助任务通过构建潜在表征学习，使智能体能够直接从原始高维图像中学习导航策略，而无需依赖残差网络（ResNet）特征。实验结果表明，在没有人工设计特征和先验演示的情况下，我们的方法显著提高了训练效率，并且在模拟器和真实世界图像数据集上优于基线算法。
