Han Yuqi, Yu Tao, Yu Xiaohang, Xu Di, Zheng Binge, Dai Zonghong, Yang Changpeng, Wang Yuwang, Dai Qionghai
IEEE Trans Vis Comput Graph. 2025 Sep;31(9):6053-6066. doi: 10.1109/TVCG.2024.3490840.
The neural radiance field (NeRF) achieved remarkable success in modeling 3D scenes and synthesizing high-fidelity novel views. However, existing NeRF-based methods focus more on making full use of high-resolution images to generate high-resolution novel views, but less considering the generation of high-resolution details given only low-resolution images. In analogy to the extensive usage of image super-resolution, NeRF super-resolution is an effective way to generate low-resolution-guided high-resolution 3D scenes and holds great potential applications. Up to now, such an important topic is still under-explored. In this article, we propose a NeRF super-resolution method, named Super-NeRF, to generate high-resolution NeRF from only low-resolution inputs. Given multi-view low-resolution images, Super-NeRF constructs a multi-view consistency-controlling super-resolution module to generate various view-consistent high-resolution details for NeRF. Specifically, an optimizable latent code is introduced for each input view to control the generated reasonable high-resolution 2D images satisfying view consistency. The latent codes of each low-resolution image are optimized synergistically with the target Super-NeRF representation to utilize the view consistency constraint inherent in NeRF construction. We verify the effectiveness of Super-NeRF on synthetic, real-world, and even AI-generated NeRFs. Super-NeRF achieves state-of-the-art NeRF super-resolution performance on high-resolution detail generation and cross-view consistency.
神经辐射场（NeRF）在3D场景建模和合成高保真新视图方面取得了显著成功。然而，现有的基于NeRF的方法更多地侧重于充分利用高分辨率图像来生成高分辨率新视图，而较少考虑仅给定低分辨率图像时高分辨率细节的生成。类似于图像超分辨率的广泛应用，NeRF超分辨率是生成低分辨率引导的高分辨率3D场景的有效方法，具有巨大的潜在应用价值。到目前为止，这样一个重要的课题仍未得到充分探索。在本文中，我们提出了一种名为Super-NeRF的NeRF超分辨率方法，仅从低分辨率输入生成高分辨率NeRF。给定多视图低分辨率图像，Super-NeRF构建一个多视图一致性控制超分辨率模块，为NeRF生成各种视图一致的高分辨率细节。具体来说，为每个输入视图引入一个可优化的潜在代码，以控制生成满足视图一致性的合理高分辨率2D图像。每个低分辨率图像的潜在代码与目标Super-NeRF表示协同优化，以利用NeRF构建中固有的视图一致性约束。我们在合成、真实世界甚至人工智能生成的NeRF上验证了Super-NeRF的有效性。Super-NeRF在高分辨率细节生成和跨视图一致性方面实现了领先的NeRF超分辨率性能。
