Annu Int Conf IEEE Eng Med Biol Soc. 2022 Jul;2022:881-885. doi: 10.1109/EMBC48229.2022.9871060.
In recent years, augmented reality (AR) technologies have been widespread for supporting various kinds of tasks, by superimposing useful information on the users' view of the real environments. In endoscopic diagnosis, AR systems can be helpful as an aid in presenting information to endoscopists who have their hands full. In this paper, we propose a system that can superimpose shapes, which are reconstructed from an endoscope image, onto the field of view. The feature of the proposed system is that it reconstructs 3D shapes from the images captured by the endoscope and superimposes them onto the real views. As a result, the superimposed view allows the doctor to keep operating the endoscope while observing the patient's internal body with additional information. The proposed system is composed of the reconstruction module and the display module. The reconstruction module is for acquiring 3D shapes based on an active stereo method. In particular, we propose a novel projection pattern that can reconstruct wide areas of the endoscopic view. The display module shows the 3D shape obtained by the reconstructed module, superimposing on the field of view. In the experiments, we show that it is possible to perform a wide range of dense 3D reconstructions using the new projection patterns. In addition, we confirmed the usefulness of the AR system by interviewing medical doctors.
近年来，增强现实（AR）技术已广泛应用于支持各种任务，通过将有用的信息叠加到用户对真实环境的视图上。在内窥镜诊断中，AR 系统可以作为一种辅助工具，为忙于操作的内窥镜医生提供信息。在本文中，我们提出了一种可以将从内窥镜图像重建的形状叠加到视场中的系统。该系统的特点是，它可以从内窥镜拍摄的图像中重建 3D 形状，并将其叠加到真实视图上。因此，叠加视图允许医生在观察患者内部身体的同时，通过附加信息继续操作内窥镜。该系统由重建模块和显示模块组成。重建模块用于基于主动立体方法获取 3D 形状。特别是，我们提出了一种新的投影模式，可以重建内窥镜视图的广阔区域。显示模块显示由重建模块获得的 3D 形状，叠加在视场中。在实验中，我们展示了使用新的投影模式可以进行广泛的密集 3D 重建。此外，我们通过采访医生来确认 AR 系统的有用性。
