Hanson Nathaniel, Lvov Gary, Padir Taşkın
Institute for Experiential Robotics, Northeastern University, Boston, MA, United States.
Department of Electrical and Computer Engineering, Northeastern University, Boston, MA, United States.
Front Robot AI. 2022 Oct 14;9:982131. doi: 10.3389/frobt.2022.982131. eCollection 2022.
Cluttered environments with partial object occlusions pose significant challenges to robot manipulation. In settings composed of one dominant object type and various undesirable contaminants, occlusions make it difficult to both recognize and isolate undesirable objects. Spatial features alone are not always sufficiently distinct to reliably identify anomalies under multiple layers of clutter, with only a fractional part of the object exposed. We create a multi-modal data representation of cluttered object scenes pairing depth data with a registered hyperspectral data cube. Hyperspectral imaging provides pixel-wise Visible Near-Infrared (VNIR) reflectance spectral curves which are invariant in similar material types. Spectral reflectance data is grounded in the chemical-physical properties of an object, making spectral curves an excellent modality to differentiate inter-class material types. Our approach proposes a new automated method to perform hyperspectral anomaly detection in cluttered workspaces with the goal of improving robot manipulation. We first assume the dominance of a single material class, and coarsely identify the dominant, non-anomalous class. Next these labels are used to train an unsupervised autoencoder to identify anomalous pixels through reconstruction error. To tie our anomaly detection to robot actions, we then apply a set of heuristically-evaluated motion primitives to perturb and further expose local areas containing anomalies. The utility of this approach is demonstrated in numerous cluttered environments including organic and inorganic materials. In each of our four constructed scenarios, our proposed anomaly detection method is able to consistently increase the exposed surface area of anomalies. Our work advances robot perception for cluttered environments by incorporating multi-modal anomaly detection aided by hyperspectral sensing into detecting fractional object presence without need for laboriously curated labels.
存在部分物体遮挡的杂乱环境给机器人操作带来了重大挑战。在由一种主要物体类型和各种不良污染物组成的场景中，遮挡使得识别和分离不良物体都变得困难。仅靠空间特征并不总是足够明显，无法在多层杂乱情况下可靠地识别异常，因为物体只有一小部分暴露在外。我们创建了一种杂乱物体场景的多模态数据表示，将深度数据与注册的高光谱数据立方体配对。高光谱成像提供逐像素的可见近红外（VNIR）反射光谱曲线，这些曲线在相似材料类型中是不变的。光谱反射数据基于物体的化学物理性质，使光谱曲线成为区分不同类别材料类型的优秀模态。我们的方法提出了一种新的自动化方法，用于在杂乱工作空间中进行高光谱异常检测，目的是改善机器人操作。我们首先假设单一材料类别的主导地位，并粗略识别主导的、非异常类别。接下来，这些标签用于训练无监督自动编码器，通过重建误差来识别异常像素。为了将我们的异常检测与机器人动作联系起来，我们然后应用一组经过启发式评估的运动原语来扰动并进一步暴露包含异常的局部区域。这种方法的效用在包括有机和无机材料在内的众多杂乱环境中得到了证明。在我们构建的四个场景中的每一个中，我们提出的异常检测方法都能够持续增加异常的暴露表面积。我们的工作通过将高光谱传感辅助的多模态异常检测纳入检测部分物体存在，而无需费力策划标签，推进了机器人在杂乱环境中的感知能力。
