School of Information Science and Technology, ShanghaiTech University, Shanghai 201210, China.
Shanghai Engineering Research Center of Intelligent Vision and Imaging, ShanghaiTech University, Shanghai 201210, China.
Sensors (Basel). 2022 Jul 29;22(15):5687. doi: 10.3390/s22155687.
We present a new solution to tracking and mapping with an event camera. The motion of the camera contains both rotation and translation displacements in the plane, and the displacements happen in an arbitrarily structured environment. As a result, the image matching may no longer be represented by a low-dimensional homographic warping, thus complicating an application of the commonly used Image of Warped Events (IWE). We introduce a new solution to this problem by performing contrast maximization in 3D. The 3D location of the rays cast for each event is smoothly varied as a function of a continuous-time motion parametrization, and the optimal parameters are found by maximizing the contrast in a volumetric ray density field. Our method thus performs joint optimization over motion and structure. The practical validity of our approach is supported by an application to AGV motion estimation and 3D reconstruction with a single vehicle-mounted event camera. The method approaches the performance obtained with regular cameras and eventually outperforms in challenging visual conditions.
我们提出了一种新的基于事件相机的跟踪和建图解决方案。相机的运动包含平面内的旋转和平移位移，并且位移发生在任意结构的环境中。因此，图像匹配可能不再由低维单应变换表示，从而使常用的扭曲事件图像（IWE）的应用变得复杂。我们通过在 3D 中进行对比度最大化来解决这个问题。对于每个事件投射的光线的 3D 位置作为连续时间运动参数化的平滑变化，通过最大化体积射线密度场中的对比度来找到最佳参数。因此，我们的方法可以对运动和结构进行联合优化。我们的方法应用于 AGV 运动估计和使用单个车载事件相机进行 3D 重建，证明了其实际有效性。该方法在具有挑战性的视觉条件下，其性能接近常规相机的性能，并最终超越常规相机。
