Martínez-Miranzo David, Suescun-Ferrandiz Sergio, Cazorla Miguel, Gomez-Donoso Francisco
University Institute for Computer Research, University of Alicante.
University Institute for Computer Research, University of Alicante;
J Vis Exp. 2025 Jun 27(220). doi: 10.3791/68386.
The ability to create photorealistic 3D reconstructions of real-world environments from standard color images has numerous applications in fields such as healthcare, education, and industry. This paper presents a novel system that integrates advanced techniques such as structure from motion, incremental scene registration, and Gaussian Splatting to reconstruct detailed 3D models. These models are seamlessly integrated into virtual reality environments to enable immersive interaction. The pipeline begins with the capture of 360-degree images using a matrix-based pattern for comprehensive coverage. The images are processed using COLMAP to estimate camera poses and generate a sparse point cloud. Gaussian Splatting refines the reconstruction by optimizing the positions, shapes, and appearance of the points to produce highly realistic 3D environments. These models are then rendered in Unity, enabling interaction by VR headsets and supporting additional features such as avatars driven by large language models. One use case demonstrates the versatility of the system. In healthcare, the approach creates controlled, familiar virtual spaces for therapy, particularly beneficial for people with autism spectrum disorders. The immersive environments and interactive avatars provide a safe and comfortable environment for personalized therapy. This system offers significant advantages, including high visual fidelity, ease of image acquisition, and immersive user experiences. However, challenges remain, such as the computational cost of Gaussian Splatting and its reliance on accurate camera pose estimation. By overcoming these limitations, this method has the potential to transform applications in various domains, ranging from therapeutic interventions to industrial design and cultural preservation.
从标准彩色图像创建真实世界环境的逼真3D重建的能力在医疗保健、教育和工业等领域有众多应用。本文提出了一种新颖的系统，该系统集成了诸如运动结构、增量场景配准和高斯点云渲染等先进技术，以重建详细的3D模型。这些模型被无缝集成到虚拟现实环境中，以实现沉浸式交互。该流程首先使用基于矩阵的模式捕获360度图像，以实现全面覆盖。使用COLMAP对图像进行处理，以估计相机姿态并生成稀疏点云。高斯点云渲染通过优化点的位置、形状和外观来细化重建，以生成高度逼真的3D环境。然后在Unity中渲染这些模型，通过VR头显实现交互，并支持诸如由大语言模型驱动的化身等附加功能。一个用例展示了该系统的多功能性。在医疗保健领域，该方法为治疗创建可控的、熟悉的虚拟空间，对自闭症谱系障碍患者特别有益。沉浸式环境和交互式化身提供了一个安全舒适的个性化治疗环境。该系统具有显著优势，包括高视觉保真度、易于图像采集和沉浸式用户体验。然而，挑战依然存在，例如高斯点云渲染的计算成本及其对准确相机姿态估计的依赖。通过克服这些限制，该方法有可能改变从治疗干预到工业设计和文化保护等各个领域的应用。
