Diao Zhuo, Yamashita Hayato, Abe Masayuki
Graduate School of Engineering Science, Osaka University, 1-3 Machikaneyama, Toyonaka, Osaka, 560-8531, Japan.
Sci Rep. 2025 May 20;15(1):17490. doi: 10.1038/s41598-025-01578-y.
We present a metaverse laboratory system that integrates mixed reality (MR) technologies with scanning probe microscopy (SPM) for interactive atomic-scale visualization and manipulation. In order to accommodate both the visualization and input of SPM data in a virtual environment and the physical interaction with SPM-related equipment in the laboratory, the system incorporates a virtual reality (VR) and augmented reality (AR) framework to enable seamless switching between these two environments. Utilizing the pose-tracking capabilities in AR, users can intuitively interact with virtual interface elements and three-dimensional objects through physical hand gesture input to control SPM parameters and probe positioning. The system provides real-time visualization of scanned surfaces at the atomic scale in the virtual environment, enabling immediate feedback during experiments. To demonstrate the system's capabilities, we performed atomic manipulation experiments using hand gestures for lateral probe positioning, showing how MR-enhanced SPM can simplify nanoscale operations and improve experimental efficiency. Our integrated MR-SPM system allows users to conduct experiments via the metaverse platform while enhancing the human-instrument interaction experience. It extends the practical utility required for both real-time physical and virtual environment SPM operations in the laboratory, making nanoscale research more accessible and intuitive.
我们展示了一个元宇宙实验室系统,该系统将混合现实(MR)技术与扫描探针显微镜(SPM)集成在一起,用于交互式原子尺度可视化和操作。为了在虚拟环境中兼顾SPM数据的可视化和输入以及在实验室中与SPM相关设备进行物理交互,该系统采用了虚拟现实(VR)和增强现实(AR)框架,以实现这两种环境之间的无缝切换。利用AR中的姿态跟踪功能,用户可以通过物理手势输入直观地与虚拟界面元素和三维物体进行交互,以控制SPM参数和探针定位。该系统在虚拟环境中提供原子尺度扫描表面的实时可视化,从而在实验过程中实现即时反馈。为了展示该系统的能力,我们使用手势进行横向探针定位,开展了原子操纵实验,展示了MR增强的SPM如何简化纳米级操作并提高实验效率。我们的集成MR-SPM系统允许用户通过元宇宙平台进行实验,同时增强人机交互体验。它扩展了实验室中实时物理和虚拟环境SPM操作所需的实际效用,使纳米级研究更容易进行且更直观。