Tatiya Gyan, Hosseini Ramtin, Hughes Michael C, Sinapov Jivko
Department of Computer Science, Tufts University, Medford, MA, United States.
Front Robot AI. 2020 Oct 9;7:522141. doi: 10.3389/frobt.2020.522141. eCollection 2020.
From an early age, humans learn to develop an intuition for the physical nature of the objects around them by using exploratory behaviors. Such exploration provides observations of how objects feel, sound, look, and move as a result of actions applied on them. Previous works in robotics have shown that robots can also use such behaviors (e.g., lifting, pressing, shaking) to infer object properties that camera input alone cannot detect. Such learned representations are specific to each individual robot and cannot currently be transferred directly to another robot with different sensors and actions. Moreover, sensor failure can cause a robot to lose a specific sensory modality which may prevent it from using perceptual models that require it as input. To address these limitations, we propose a framework for knowledge transfer across behaviors and sensory modalities such that: (1) knowledge can be transferred from one or more robots to another, and, (2) knowledge can be transferred from one or more sensory modalities to another. We propose two different models for transfer based on variational auto-encoders and encoder-decoder networks. The main hypothesis behind our approach is that if two or more robots share multi-sensory object observations of a shared set of objects, then those observations can be used to establish mappings between multiple features spaces, each corresponding to a combination of an exploratory behavior and a sensory modality. We evaluate our approach on a category recognition task using a dataset in which a robot used 9 behaviors, coupled with 4 sensory modalities, performed multiple times on 100 objects. The results indicate that sensorimotor knowledge about objects can be transferred both across behaviors and across sensory modalities, such that a new robot (or the same robot, but with a different set of sensors) can bootstrap its category recognition models without having to exhaustively explore the full set of objects.
从幼年起，人类就通过探索行为来培养对周围物体物理性质的直觉。这种探索提供了关于物体在受到作用时的触感、声音、外观和运动方式的观察。机器人领域的先前研究表明，机器人也可以利用此类行为（例如，举起、按压、摇晃）来推断仅靠摄像头输入无法检测到的物体属性。这样学到的表征是每个机器人特有的，目前无法直接转移到具有不同传感器和行为的另一个机器人上。此外，传感器故障可能导致机器人失去特定的传感模态，这可能会阻止它使用需要该传感模态作为输入的感知模型。为了解决这些限制，我们提出了一个跨行为和传感模态进行知识转移的框架，使得：（1）知识可以从一个或多个机器人转移到另一个机器人，并且，（2）知识可以从一种或多种传感模态转移到另一种传感模态。我们基于变分自编码器和编码器 - 解码器网络提出了两种不同的转移模型。我们方法背后的主要假设是，如果两个或更多机器人共享一组共享物体的多感官观察结果，那么这些观察结果可用于在多个特征空间之间建立映射，每个特征空间对应于一种探索行为和一种传感模态的组合。我们使用一个数据集在类别识别任务上评估我们的方法，在该数据集中，一个机器人使用9种行为，并结合4种传感模态，对100个物体进行了多次操作。结果表明，关于物体的感觉运动知识可以跨行为和跨传感模态进行转移，这样一个新机器人（或者同一个机器人，但使用不同的传感器组）可以引导其类别识别模型，而无需详尽地探索整个物体集。
