Department of Industrial Engineering, Clemson University, 110 Freeman Hall, Clemson, SC 29634, United States; Risk Engineering and System Analytics Institute, Glenn Department of Civil Engineering, Clemson University, Lowry Hall, Clemson, SC 29634, United States.
Department of Industrial Engineering, Clemson University, 110 Freeman Hall, Clemson, SC 29634, United States.
Appl Ergon. 2017 Nov;65:501-514. doi: 10.1016/j.apergo.2017.02.011. Epub 2017 Feb 28.
Collaborative virtual reality-based systems have integrated high fidelity voice-based communication, immersive audio and screen-sharing tools into virtual environments. Such three-dimensional collaborative virtual environments can mirror the collaboration among usability test participants and facilitators when they are physically collocated, potentially enabling moderated usability tests to be conducted effectively when the facilitator and participant are located in different places. We developed a virtual collaborative three-dimensional remote moderated usability testing laboratory and employed it in a controlled study to evaluate the effectiveness of moderated usability testing in a collaborative virtual reality-based environment with two other moderated usability testing methods: the traditional lab approach and Cisco WebEx, a web-based conferencing and screen sharing approach. Using a mixed methods experimental design, 36 test participants and 12 test facilitators were asked to complete representative tasks on a simulated online shopping website. The dependent variables included the time taken to complete the tasks; the usability defects identified and their severity; and the subjective ratings on the workload index, presence and satisfaction questionnaires. Remote moderated usability testing methodology using a collaborative virtual reality system performed similarly in terms of the total number of defects identified, the number of high severity defects identified and the time taken to complete the tasks with the other two methodologies. The overall workload experienced by the test participants and facilitators was the least with the traditional lab condition. No significant differences were identified for the workload experienced with the virtual reality and the WebEx conditions. However, test participants experienced greater involvement and a more immersive experience in the virtual environment than in the WebEx condition. The ratings for the virtual environment condition were not significantly different from those for the traditional lab condition. The results of this study suggest that participants were productive and enjoyed the virtual lab condition, indicating the potential of a virtual world based approach as an alternative to conventional approaches for synchronous usability testing.
协作式虚拟现实系统将高保真语音通信、沉浸式音频和屏幕共享工具集成到虚拟环境中。这种三维协作式虚拟环境可以模拟可用性测试参与者和协调员在物理上位于同一地点时的协作，当协调员和参与者位于不同地点时，可能会有效地进行有监督的可用性测试。我们开发了一个虚拟协作式三维远程监督可用性测试实验室，并在一项对照研究中使用它来评估在协作式虚拟现实环境中进行有监督的可用性测试的有效性，同时还使用了另外两种有监督的可用性测试方法：传统实验室方法和 Cisco WebEx，这是一种基于网络的会议和屏幕共享方法。使用混合方法实验设计，要求 36 名测试参与者和 12 名测试协调员在一个模拟的在线购物网站上完成代表性任务。因变量包括完成任务所需的时间；确定的可用性缺陷及其严重程度；以及在工作负荷指数、存在和满意度问卷上的主观评分。使用协作式虚拟现实系统进行远程监督式可用性测试方法在识别出的缺陷总数、识别出的高严重程度缺陷数量以及完成任务所需的时间方面与其他两种方法相似。测试参与者和协调员经历的整体工作负荷在传统实验室条件下最小。虚拟现实和 WebEx 条件下的工作负荷没有明显差异。然而，测试参与者在虚拟环境中体验到的参与度和沉浸感要大于在 WebEx 条件下。虚拟环境条件的评分与传统实验室条件的评分没有显著差异。这项研究的结果表明，参与者具有生产力并且喜欢虚拟实验室条件，这表明基于虚拟世界的方法作为传统同步可用性测试方法的替代方法具有潜力。
