Furukawa Ryo, Chen Elvis, Sagawa Ryusuke, Oka Shiro, Kawasaki Hiroshi
Department of Informatics Kindai University Higashihiroshima Japan.
Robarts Research Institute London Canada.
Healthc Technol Lett. 2024 Mar 8;11(2-3):196-205. doi: 10.1049/htl2.12083. eCollection 2024 Apr-Jun.
Accurate 3D shape measurement is crucial for surgical support and alignment in robotic surgery systems. Stereo cameras in laparoscopes offer a potential solution; however, their accuracy in stereo image matching diminishes when the target image has few textures. Although stereo matching with deep learning has gained significant attention, supervised learning requires a large dataset of images with depth annotations, which are scarce for laparoscopes. Thus, there is a strong demand to explore alternative methods for depth reconstruction or annotation for laparoscopes. Active stereo techniques are a promising approach for achieving 3D reconstruction without textures. In this study, a 3D shape reconstruction method is proposed using an ultra-small patterned projector attached to a laparoscopic arm to address these issues. The pattern projector emits a structured light with a grid-like pattern that features node-wise modulation for positional encoding. To scan the target object, multiple images are taken while the projector is in motion, and the relative poses of the projector and a camera are auto-calibrated using a differential rendering technique. In the experiment, the proposed method is evaluated by performing 3D reconstruction using images obtained from a surgical robot and comparing the results with a ground-truth shape obtained from X-ray CT.
精确的三维形状测量对于机器人手术系统中的手术支持和对齐至关重要。腹腔镜中的立体相机提供了一种潜在的解决方案；然而，当目标图像纹理较少时，它们在立体图像匹配中的准确性会降低。尽管深度学习立体匹配已受到广泛关注，但监督学习需要大量带有深度注释的图像数据集，而腹腔镜的此类数据集很少。因此，迫切需要探索用于腹腔镜深度重建或注释的替代方法。主动立体技术是一种无需纹理即可实现三维重建的有前途的方法。在本研究中，提出了一种三维形状重建方法，该方法使用连接到腹腔镜臂的超小型图案投影仪来解决这些问题。图案投影仪发射具有网格状图案的结构化光，该图案具有用于位置编码的逐节点调制。为了扫描目标物体，在投影仪移动时拍摄多幅图像，并使用差分渲染技术自动校准投影仪和相机的相对姿态。在实验中，通过使用从手术机器人获得的图像进行三维重建，并将结果与从X射线计算机断层扫描获得的真实形状进行比较，对所提出的方法进行了评估。
