Beijing Engineering Research Center of Mixed Reality and Advanced Display, Beijing Institute of Technology, Beijing 100081, China.
Sensors (Basel). 2023 Apr 19;23(8):4113. doi: 10.3390/s23084113.
In mixed-reality (MR) telecollaboration, the local environment is remotely presented to a remote user wearing a virtual reality (VR) head-mounted display (HMD) via a video capture device. However, remote users frequently face challenges in naturally and actively manipulating their viewpoints. In this paper, we propose a telepresence system with viewpoint control, which involves a robotic arm equipped with a stereo camera in the local environment. This system enables remote users to actively and flexibly observe the local environment by moving their heads to manipulate the robotic arm. Additionally, to solve the problem of the limited field of view of the stereo camera and limited movement range of the robotic arm, we propose a 3D reconstruction method combined with a stereo video field-of-view enhancement technique to guide remote users to move within the movement range of the robotic arm and provide them with a larger range of local environment perception. Finally, a mixed-reality telecollaboration prototype was built, and two user studies were conducted to evaluate the overall system. User study A evaluated the interaction efficiency, system usability, workload, copresence, and user satisfaction of our system from the remote user's perspective, and the results showed that our system can effectively improve the interaction efficiency while achieving a better user experience than two traditional view-sharing techniques based on 360 video and based on the local user's first-person view. User study B evaluated our MR telecollaboration system prototype from both the remote-user side and the local-user side as a whole, providing directions and suggestions for the subsequent design and improvement of our mixed-reality telecollaboration system.
在混合现实 (MR) 远程协作中，通过视频捕获设备将本地环境远程呈现给佩戴虚拟现实 (VR) 头戴式显示器 (HMD) 的远程用户。然而，远程用户在自然、主动地操控视点方面经常面临挑战。在本文中，我们提出了一种具有视点控制的临场感系统，该系统涉及配备立体相机的机械臂，该立体相机位于本地环境中。该系统使远程用户能够通过移动头部来操控机械臂，从而主动灵活地观察本地环境。此外，为了解决立体相机视场有限和机械臂运动范围有限的问题，我们提出了一种 3D 重建方法，结合立体视频视场增强技术，引导远程用户在机械臂的运动范围内移动，并为他们提供更大的本地环境感知范围。最后，构建了一个混合现实远程协作原型，并进行了两项用户研究来评估整个系统。用户研究 A 从远程用户的角度评估了我们的系统的交互效率、系统可用性、工作量、临场感和用户满意度，结果表明我们的系统可以有效地提高交互效率，同时实现比基于 360 度视频和基于本地用户第一人称视图的两种传统视图共享技术更好的用户体验。用户研究 B 从远程用户和本地用户两个方面对我们的混合现实远程协作系统原型进行了整体评估，为我们的混合现实远程协作系统的后续设计和改进提供了方向和建议。
