Shi Xuehuai, Wang Lili, Liu Xinda, Wu Jian, Shao Zhiwen
IEEE Trans Vis Comput Graph. 2025 Sep;31(9):5039-5054. doi: 10.1109/TVCG.2024.3429416.
Foveated rendering provides an idea for improving the image synthesis performance of neural radiance fields (NeRF) methods. In this article, we propose a scene-aware foveated neural radiance fields method to synthesize high-quality foveated images in complex VR scenes at high frame rates. First, we construct a multi-ellipsoidal neural representation to enhance the neural radiance field's representation capability in salient regions of complex VR scenes based on the scene content. Then, we introduce a uniform sampling based foveated neural radiance field framework to improve the foveated image synthesis performance with one-pass color inference, and improve the synthesis quality by leveraging the foveated scene-aware objective function. Our method synthesizes high-quality binocular foveated images at the average frame rate of 66 frames per second ($FPS$FPS) in complex scenes with high occlusion, intricate textures, and sophisticated geometries. Compared with the state-of-the-art foveated NeRF method, our method achieves significantly higher synthesis quality in both the foveal and peripheral regions with 1.41-1.46× speedup. We also conduct a user study to prove that the perceived quality of our method has a high visual similarity with the ground truth.
注视点渲染为提高神经辐射场（NeRF）方法的图像合成性能提供了一种思路。在本文中，我们提出了一种场景感知注视点神经辐射场方法，以高帧率在复杂虚拟现实（VR）场景中合成高质量的注视点图像。首先，基于场景内容构建多椭球神经表示，以增强神经辐射场在复杂VR场景显著区域的表示能力。然后，引入基于均匀采样的注视点神经辐射场框架，通过单次颜色推理提高注视点图像合成性能，并利用注视点场景感知目标函数提高合成质量。我们的方法在具有高遮挡、复杂纹理和精细几何结构的复杂场景中，以平均每秒66帧（$FPS$）的帧率合成高质量的双目注视点图像。与当前最先进的注视点NeRF方法相比，我们的方法在中央凹和周边区域均实现了显著更高的合成质量，加速比为1.41 - 1.46倍。我们还进行了用户研究，以证明我们方法的感知质量与真实情况具有高度的视觉相似性。
