IEEE Trans Vis Comput Graph. 2024 Aug;30(8):4648-4664. doi: 10.1109/TVCG.2023.3276291. Epub 2024 Jul 1.
To understand current practice and explore the potential for more comprehensive evaluations of 3D immersive sketching, drawing, and painting, we present a survey of evaluation methodologies used in existing 3D sketching research, a breakdown and discussion of important phases (sub-tasks) in the 3D sketching process, and a framework that suggests how these factors can inform evaluation strategies in future 3D sketching research. Existing evaluations identified in the survey are organized and discussed within three high-level categories: 1) evaluating the 3D sketching activity, 2) evaluating 3D sketching tools, and 3) evaluating 3D sketching artifacts. The new framework suggests targeting evaluations to one or more of these categories and identifying relevant user populations. In addition, building upon the discussion of the different phases of the 3D sketching process, the framework suggests to evaluate relevant sketching tasks, which may range from low-level perception and hand movements to high-level conceptual design. Finally, we discuss limitations and challenges that arise when evaluating 3D sketching, including a lack of standardization of evaluation methods and multiple, potentially conflicting, ways to evaluate the same task and user interface usability; we also identify opportunities for more holistic evaluations. We hope the results can contribute to accelerating research in this domain and, ultimately, broad adoption of immersive sketching systems.
为了了解当前的实践情况，并探索更全面地评估 3D 沉浸式草图绘制和绘画的潜力，我们对现有的 3D 草图研究中使用的评估方法进行了调查，对 3D 草图过程中的重要阶段（子任务）进行了细分和讨论，并提出了一个框架，建议这些因素如何为未来的 3D 草图研究中的评估策略提供信息。调查中确定的现有评估方法被组织并讨论在三个高级别类别中：1）评估 3D 草图绘制活动，2）评估 3D 草图绘制工具，以及 3）评估 3D 草图绘制制品。新框架建议将评估目标针对这些类别中的一个或多个类别，并确定相关的用户群体。此外，在讨论 3D 草图绘制过程的不同阶段的基础上，该框架建议评估相关的草图绘制任务，这些任务可能从低级别的感知和手部运动到高级别的概念设计不等。最后，我们讨论了评估 3D 草图绘制时出现的限制和挑战，包括评估方法缺乏标准化，以及评估同一任务和用户界面可用性的多种、潜在冲突的方法；我们还确定了更全面评估的机会。我们希望研究结果有助于加速该领域的研究，并最终广泛采用沉浸式草图绘制系统。
