Chen Tao, Tippur Megha, Wu Siyang, Kumar Vikash, Adelson Edward, Agrawal Pulkit
Improbable AI Laboratory, Massachusetts Institute of Technology, Cambridge, MA 02139, USA.
Computer Science and Artificial Intelligence Laboratory (CSAIL), Massachusetts Institute of Technology, Cambridge, MA 02139, USA.
Sci Robot. 2023 Nov 22;8(84):eadc9244. doi: 10.1126/scirobotics.adc9244.
In-hand object reorientation is necessary for performing many dexterous manipulation tasks, such as tool use in less structured environments, which remain beyond the reach of current robots. Prior works built reorientation systems assuming one or many of the following conditions: reorienting only specific objects with simple shapes, limited range of reorientation, slow or quasi-static manipulation, simulation-only results, the need for specialized and costly sensor suites, and other constraints that make the system infeasible for real-world deployment. We present a general object reorientation controller that does not make these assumptions. It uses readings from a single commodity depth camera to dynamically reorient complex and new object shapes by any rotation in real time, with the median reorientation time being close to 7 seconds. The controller was trained using reinforcement learning in simulation and evaluated in the real world on new object shapes not used for training, including the most challenging scenario of reorienting objects held in the air by a downward-facing hand that must counteract gravity during reorientation. Our hardware platform only used open-source components that cost less than 5000 dollars. Although we demonstrate the ability to overcome assumptions in prior work, there is ample scope for improving absolute performance. For instance, the challenging duck-shaped object not used for training was dropped in 56% of the trials. When it was not dropped, our controller reoriented the object within 0.4 radians (23°) 75% of the time.
在手中对物体进行重新定向对于执行许多灵巧的操作任务至关重要，例如在结构较松散的环境中使用工具，而这仍是当前机器人无法做到的。先前的工作构建重新定向系统时假设了以下一个或多个条件：仅对形状简单的特定物体进行重新定向、重新定向范围有限、操作缓慢或近似静态、仅模拟结果、需要专门且昂贵的传感器套件以及其他使系统无法用于实际部署的限制。我们提出了一种通用的物体重新定向控制器，该控制器不做这些假设。它使用单个商用深度相机的读数，以实时通过任意旋转对复杂和新的物体形状进行动态重新定向，重新定向的平均时间接近7秒。该控制器在模拟中使用强化学习进行训练，并在现实世界中对未用于训练的新物体形状进行评估，包括最具挑战性的场景，即通过向下的手在空中握住物体进行重新定向，在重新定向过程中必须抵消重力。我们的硬件平台仅使用了成本低于5000美元的开源组件。尽管我们展示了克服先前工作中假设的能力，但仍有很大的提升绝对性能的空间。例如，在56%的试验中，未用于训练的具有挑战性的鸭形物体会掉落。当它没有掉落时，我们的控制器在75%的时间内将物体重新定向到0.4弧度（23°）以内。
