Makki Karim, Chandelon Kilian, Bartoli Adrien
EnCoV, Institut Pascal, UMR6602 CNRS/UCA, Clermont-Ferrand, France.
Direction de la Recherche Clinique et de l'Innovation, CHU de, Clermont-Ferrand, France.
Int J Comput Assist Radiol Surg. 2023 Jul;18(7):1323-1328. doi: 10.1007/s11548-023-02904-3. Epub 2023 May 4.
To detect specularities as elliptical blobs in endoscopy. The rationale is that in the endoscopic setting, specularities are generally small and that knowing the ellipse coefficients allows one to reconstruct the surface normal. In contrast, previous works detect specular masks as free-form shapes and consider the specular pixels as nuisance.
A pipeline combining deep learning with handcrafted steps for specularity detection. This pipeline is general and accurate in the context of endoscopic applications involving multiple organs and moist tissues. A fully convolutional network produces an initial mask which specifically finds specular pixels, being mainly composed of sparsely distributed blobs. Standard ellipse fitting follows for local segmentation refinement in order to only keep the blobs fulfilling the conditions for successful normal reconstruction.
Convincing results in detection and reconstruction on synthetic and real images, showing that the elliptical shape prior improves the detection itself in both colonoscopy and kidney laparoscopy. The pipeline achieved a mean Dice of 84% and 87% respectively in test data for these two use cases, and allows one to exploit the specularities as useful information for inferring sparse surface geometry. The reconstructed normals are in good quantitative agreement with external learning-based depth reconstruction methods manifested, as shown by an average angular discrepancy of [Formula: see text] in colonoscopy.
First fully automatic method to exploit specularities in endoscopic 3D reconstruction. Because the design of current reconstruction methods can vary considerably for different applications, our elliptical specularity detection could be of potential interest in clinical practice thanks to its simplicity and generalisability. In particular, the obtained results are promising towards future integration with learning-based depth inference and SfM methods.
在内窥镜检查中检测作为椭圆形斑点的镜面反射。其基本原理是，在内窥镜环境中，镜面反射通常较小，并且知道椭圆系数可以重建表面法线。相比之下，先前的工作将镜面反射掩码检测为自由形式的形状，并将镜面反射像素视为干扰因素。
一种将深度学习与手工步骤相结合的用于镜面反射检测的流程。该流程在涉及多个器官和湿润组织的内窥镜应用中具有通用性和准确性。一个全卷积网络生成一个初始掩码，该掩码专门用于找到镜面反射像素，主要由稀疏分布的斑点组成。随后进行标准椭圆拟合以细化局部分割，以便仅保留满足成功法线重建条件的斑点。
在合成图像和真实图像上的检测和重建取得了令人信服的结果，表明椭圆形先验在结肠镜检查和肾脏腹腔镜检查中都改进了检测本身。对于这两个用例，该流程在测试数据中的平均骰子系数分别达到了84%和87%，并且允许将镜面反射作为推断稀疏表面几何形状的有用信息加以利用。重建的法线与基于外部学习的深度重建方法在定量上具有良好的一致性，如结肠镜检查中平均角度差异为[公式：见原文]所示。
在内窥镜三维重建中利用镜面反射的首个全自动方法。由于当前重建方法的设计因不同应用可能有很大差异，我们的椭圆形镜面反射检测因其简单性和通用性在临床实践中可能具有潜在价值。特别是，所获得的结果对于未来与基于学习的深度推断和结构从运动（SfM）方法的集成很有前景。
