IEEE Trans Cybern. 2013 Oct;43(5):1429-41. doi: 10.1109/TCYB.2013.2275291. Epub 2013 Aug 21.
The ability to perceive humans is an essential requirement for safe and efficient human-robot interaction. In real-world applications, the need for a robot to interact in real time with multiple humans in a dynamic, 3-D environment presents a significant challenge. The recent availability of commercial color-depth cameras allow for the creation of a system that makes use of the depth dimension, thus enabling a robot to observe its environment and perceive in the 3-D space. Here we present a system for 3-D multiple human perception in real time from a moving robot equipped with a color-depth camera and a consumer-grade computer. Our approach reduces computation time to achieve real-time performance through a unique combination of new ideas and established techniques. We remove the ground and ceiling planes from the 3-D point cloud input to separate candidate point clusters. We introduce the novel information concept, depth of interest, which we use to identify candidates for detection, and that avoids the computationally expensive scanning-window methods of other approaches. We utilize a cascade of detectors to distinguish humans from objects, in which we make intelligent reuse of intermediary features in successive detectors to improve computation. Because of the high computational cost of some methods, we represent our candidate tracking algorithm with a decision directed acyclic graph, which allows us to use the most computationally intense techniques only where necessary. We detail the successful implementation of our novel approach on a mobile robot and examine its performance in scenarios with real-world challenges, including occlusion, robot motion, nonupright humans, humans leaving and reentering the field of view (i.e., the reidentification challenge), human-object and human-human interaction. We conclude with the observation that the incorporation of the depth information, together with the use of modern techniques in new ways, we are able to create an accurate system for real-time 3-D perception of humans by a mobile robot.
感知人类的能力是安全高效人机交互的基本要求。在实际应用中，机器人需要在动态、三维环境中实时与多个人类进行交互，这是一个重大挑战。最近商用彩色深度相机的出现使得我们能够创建一个利用深度维度的系统，从而使机器人能够观察其环境并在三维空间中感知。在这里，我们提出了一种实时从配备彩色深度相机和消费级计算机的移动机器人中进行三维多人感知的系统。我们的方法通过新颖的想法和成熟的技术的独特结合，减少了计算时间，实现了实时性能。我们从三维点云输入中去除地面和天花板平面，以分离候选点簇。我们引入了新颖的信息概念——兴趣深度，用于识别检测候选对象，并避免了其他方法中计算成本高昂的扫描窗口方法。我们利用级联检测器来区分人与物体，在其中，我们在连续的检测器中智能地重复使用中间特征，以提高计算效率。由于某些方法的计算成本很高，我们使用决策有向无环图来表示候选跟踪算法，这使我们能够仅在必要时使用最具计算密集性的技术。我们详细介绍了我们的新方法在移动机器人上的成功实现，并在包括遮挡、机器人运动、非直立人类、人类离开和重新进入视野（即重新识别挑战）、人机和人人交互在内的具有真实世界挑战的场景中检查其性能。我们的观察结果表明，通过结合深度信息和以新方式使用现代技术，我们能够为移动机器人实时三维感知人类创建一个准确的系统。
