Zhuang Jiafu, Zeng Pan, Zhuang Wei, Guo Xiaoyu, Liu Peizhong
IEEE Trans Vis Comput Graph. 2024 Aug;30(8):5553-5565. doi: 10.1109/TVCG.2023.3294845. Epub 2024 Jul 1.
The analysis of 3D meshes with deep learning has become prevalent in computer graphics. As an essential structure, hierarchical representation is critical for mesh pooling in multiscale analysis. Existing clustering-based mesh hierarchy construction methods involve nonlinear discretization optimization operations, making them nondifferential and challenging to embed in other trainable networks for learning. Inspired by deep superpixel learning methods in image processing, we extend them from 2D images to 3D meshes by proposing a novel differentiable chart-based segmentation method named geodesic differential supervertex (GDSV). The key to the GDSV method is to ensure that the geodesic position updates are differentiable while satisfying the constraint that the renewed supervertices lie on the manifold surface. To this end, in addition to using the differential SLIC clustering algorithm to update the nonpositional features of the supervertices, a reparameterization trick, the Gumbel-Softmax trick, is employed to renew the geodesic positions of the supervertices. Therefore, the geodesic position update problem is converted into a linear matrix multiplication issue. The GDSV method can be an independent module for chart-based segmentation tasks. Meanwhile, it can be combined with the front-end feature learning network and the back-end task-specific network as a plug-in-plug-out module for training; and be applied to tasks such as shape classification, part segmentation, and 3D scene understanding. Experimental results show the excellent performance of our proposed algorithm on a range of datasets.
利用深度学习对三维网格进行分析在计算机图形学中已变得十分普遍。作为一种重要结构,层次表示对于多尺度分析中的网格池化至关重要。现有的基于聚类的网格层次结构构建方法涉及非线性离散化优化操作,这使得它们不可微,并且难以嵌入到其他可训练网络中进行学习。受图像处理中深度超像素学习方法的启发,我们通过提出一种名为测地线微分超顶点(GDSV)的基于图表的新型可微分割方法,将其从二维图像扩展到三维网格。GDSV方法的关键在于确保测地线位置更新是可微的,同时满足更新后的超顶点位于流形表面的约束。为此,除了使用微分SLIC聚类算法来更新超顶点的非位置特征外,还采用了一种重新参数化技巧,即Gumbel-Softmax技巧,来更新超顶点的测地线位置。因此,测地线位置更新问题被转化为一个线性矩阵乘法问题。GDSV方法可以作为基于图表的分割任务的一个独立模块。同时,它可以与前端特征学习网络和后端特定任务网络相结合,作为一个即插即用的模块进行训练;并应用于形状分类、部件分割和三维场景理解等任务。实验结果表明了我们提出的算法在一系列数据集上的优异性能。