Xie Jianwen, Zheng Zilong, Gao Ruiqi, Wang Wenguan, Zhu Song-Chun, Wu Ying Nian
IEEE Trans Pattern Anal Mach Intell. 2022 May;44(5):2468-2484. doi: 10.1109/TPAMI.2020.3045010. Epub 2022 Apr 1.
3D data that contains rich geometry information of objects and scenes is valuable for understanding 3D physical world. With the recent emergence of large-scale 3D datasets, it becomes increasingly crucial to have a powerful 3D generative model for 3D shape synthesis and analysis. This paper proposes a deep 3D energy-based model to represent volumetric shapes. The maximum likelihood training of the model follows an "analysis by synthesis" scheme. The benefits of the proposed model are six-fold: first, unlike GANs and VAEs, the model training does not rely on any auxiliary models; second, the model can synthesize realistic 3D shapes by Markov chain Monte Carlo (MCMC); third, the conditional model can be applied to 3D object recovery and super resolution; fourth, the model can serve as a building block in a multi-grid modeling and sampling framework for high resolution 3D shape synthesis; fifth, the model can be used to train a 3D generator via MCMC teaching; sixth, the unsupervisedly trained model provides a powerful feature extractor for 3D data, which is useful for 3D object classification. Experiments demonstrate that the proposed model can generate high-quality 3D shape patterns and can be useful for a wide variety of 3D shape analysis.
包含物体和场景丰富几何信息的3D数据对于理解3D物理世界很有价值。随着近期大规模3D数据集的出现，拥有一个强大的用于3D形状合成与分析的3D生成模型变得越来越关键。本文提出了一种基于能量的深度3D模型来表示体形状。该模型的最大似然训练遵循“通过合成进行分析”的方案。所提出模型的优点有六个方面：第一，与生成对抗网络（GAN）和变分自编码器（VAE）不同，该模型训练不依赖任何辅助模型；第二，该模型可以通过马尔可夫链蒙特卡罗（MCMC）合成逼真的3D形状；第三，条件模型可应用于3D物体恢复和超分辨率；第四，该模型可作为高分辨率3D形状合成的多网格建模和采样框架中的一个构建模块；第五，该模型可用于通过MCMC教学训练3D生成器；第六，无监督训练的模型为3D数据提供了一个强大的特征提取器，这对于3D物体分类很有用。实验表明，所提出的模型可以生成高质量的3D形状模式，并且可用于各种3D形状分析。
