IEEE Trans Pattern Anal Mach Intell. 2023 Mar;45(3):3292-3310. doi: 10.1109/TPAMI.2022.3179507. Epub 2023 Feb 3.
For the last few decades, several major subfields of artificial intelligence including computer vision, graphics, and robotics have progressed largely independently from each other. Recently, however, the community has realized that progress towards robust intelligent systems such as self-driving cars requires a concerted effort across the different fields. This motivated us to develop KITTI-360, successor of the popular KITTI dataset. KITTI-360 is a suburban driving dataset which comprises richer input modalities, comprehensive semantic instance annotations and accurate localization to facilitate research at the intersection of vision, graphics and robotics. For efficient annotation, we created a tool to label 3D scenes with bounding primitives and developed a model that transfers this information into the 2D image domain, resulting in over 150k images and 1B 3D points with coherent semantic instance annotations across 2D and 3D. Moreover, we established benchmarks and baselines for several tasks relevant to mobile perception, encompassing problems from computer vision, graphics, and robotics on the same dataset, e.g., semantic scene understanding, novel view synthesis and semantic SLAM. KITTI-360 will enable progress at the intersection of these research areas and thus contribute towards solving one of today's grand challenges: the development of fully autonomous self-driving systems.
在过去的几十年里，人工智能的几个主要子领域，包括计算机视觉、图形和机器人技术，在很大程度上是相互独立发展的。然而，最近，该领域的研究人员已经意识到，要实现自动驾驶汽车等强大智能系统的进步，需要在不同领域进行协同努力。这促使我们开发了 KITTI-360，这是广受欢迎的 KITTI 数据集的后续版本。KITTI-360 是一个郊区驾驶数据集，它包含更丰富的输入模式、全面的语义实例注释和精确的定位，以促进视觉、图形和机器人技术交叉领域的研究。为了高效地进行注释，我们创建了一个工具来用边界框标注 3D 场景，并开发了一个模型，将这些信息转换到 2D 图像域中，从而在 2D 和 3D 之间产生了超过 150k 张图像和 10 亿个具有连贯语义实例注释的 3D 点。此外，我们为几个与移动感知相关的任务建立了基准和基线，涵盖了来自计算机视觉、图形和机器人技术的问题，例如语义场景理解、新视图合成和语义 SLAM。KITTI-360 将促进这些研究领域的交叉发展，从而有助于解决当今的一个重大挑战：开发完全自主的自动驾驶系统。
