Zare Maryam, Kebria Parham M, Khosravi Abbas, Nahavandi Saeid
IEEE Trans Cybern. 2024 Dec;54(12):7173-7186. doi: 10.1109/TCYB.2024.3395626. Epub 2024 Nov 27.
In recent years, the development of robotics and artificial intelligence (AI) systems has been nothing short of remarkable. As these systems continue to evolve, they are being utilized in increasingly complex and unstructured environments, such as autonomous driving, aerial robotics, and natural language processing. As a consequence, programming their behaviors manually or defining their behavior through the reward functions [as done in reinforcement learning (RL)] has become exceedingly difficult. This is because such environments require a high degree of flexibility and adaptability, making it challenging to specify an optimal set of rules or reward signals that can account for all the possible situations. In such environments, learning from an expert's behavior through imitation is often more appealing. This is where imitation learning (IL) comes into play - a process where desired behavior is learned by imitating an expert's behavior, which is provided through demonstrations.This article aims to provide an introduction to IL and an overview of its underlying assumptions and approaches. It also offers a detailed description of recent advances and emerging areas of research in the field. Additionally, this article discusses how researchers have addressed common challenges associated with IL and provides potential directions for future research. Overall, the goal of this article is to provide a comprehensive guide to the growing field of IL in robotics and AI.
近年来，机器人技术和人工智能（AI）系统的发展堪称卓越。随着这些系统不断演进，它们正被应用于越来越复杂和非结构化的环境中，如自动驾驶、空中机器人技术以及自然语言处理。因此，手动对其行为进行编程或通过奖励函数（如在强化学习（RL）中那样）来定义其行为变得极其困难。这是因为此类环境需要高度的灵活性和适应性，要指定一组能涵盖所有可能情况的最优规则或奖励信号颇具挑战。在这样的环境中，通过模仿专家行为进行学习通常更具吸引力。这就是模仿学习（IL）发挥作用之处——通过模仿由示范提供的专家行为来学习期望行为的过程。本文旨在介绍模仿学习，并概述其基本假设和方法。它还详细描述了该领域的最新进展和新兴研究领域。此外，本文讨论了研究人员如何应对与模仿学习相关的常见挑战，并为未来研究提供了潜在方向。总体而言，本文的目标是为机器人技术和人工智能中不断发展的模仿学习领域提供全面指南。
