Department of Electronic Materials Engeering, Kwangwoon University, Seoul 01897, Korea.
Sensors (Basel). 2022 Nov 7;22(21):8563. doi: 10.3390/s22218563.
Due to the amount of transmitted data and the security of personal or private information in wireless communication, there are cases where the information for a multimedia service should be directly transferred from the user's device to the cloud server without the captured original images. This paper proposes a new method to generate 3D (dimensional) keypoints based on a user's mobile device with a commercial RGB camera in a distributed computing environment such as a cloud server. The images are captured with a moving camera and 2D keypoints are extracted from them. After executing feature extraction between continuous frames, disparities are calculated between frames using the relationships between matched keypoints. The physical distance of the baseline is estimated by using the motion information of the camera, and the actual distance is calculated by using the calculated disparity and the estimated baseline. Finally, 3D keypoints are generated by adding the extracted 2D keypoints to the calculated distance. A keypoint-based scene change method is proposed as well. Due to the existing similarity between continuous frames captured from a camera, not all 3D keypoints are transferred and stored, only the new ones. Compared with the ground truth of the TUM dataset, the average error of the estimated 3D keypoints was measured as 5.98 mm, which shows that the proposed method has relatively good performance considering that it uses a commercial RGB camera on a mobile device. Furthermore, the transferred 3D keypoints were decreased to about 73.6%.
由于无线通信中传输的数据量和个人或私人信息的安全性，存在多媒体服务的信息应直接从用户设备传输到云服务器而无需捕获原始图像的情况。本文提出了一种新方法，即在云服务器等分布式计算环境中，使用商用 RGB 相机从用户的移动设备生成 3D（维度）关键点。使用移动相机捕获图像，并从中提取 2D 关键点。在连续帧之间执行特征提取后，使用匹配关键点之间的关系计算帧之间的视差。通过使用相机的运动信息来估计基线的物理距离，并使用计算出的视差和估计的基线来计算实际距离。最后，通过将提取的 2D 关键点添加到计算出的距离中，生成 3D 关键点。还提出了基于关键点的场景变化方法。由于从相机捕获的连续帧之间存在相似性，并非所有 3D 关键点都被传输和存储，仅传输和存储新的关键点。与 TUM 数据集的真实值相比，估计的 3D 关键点的平均误差为 5.98mm，这表明考虑到它在移动设备上使用商用 RGB 相机，该方法具有相对较好的性能。此外，传输的 3D 关键点减少到约 73.6%。
