Ceneda Davide, Collins Christopher, El-Assady Mennatallah, Miksch Silvia, Tominski Christian, Arleo Alessio
IEEE Trans Vis Comput Graph. 2024 Jan;30(1):997-1007. doi: 10.1109/TVCG.2023.3327152. Epub 2023 Dec 25.
Guidance can support users during the exploration and analysis of complex data. Previous research focused on characterizing the theoretical aspects of guidance in visual analytics and implementing guidance in different scenarios. However, the evaluation of guidance-enhanced visual analytics solutions remains an open research question. We tackle this question by introducing and validating a practical evaluation methodology for guidance in visual analytics. We identify eight quality criteria to be fulfilled and collect expert feedback on their validity. To facilitate actual evaluation studies, we derive two sets of heuristics. The first set targets heuristic evaluations conducted by expert evaluators. The second set facilitates end-user studies where participants actually use a guidance-enhanced system. By following such a dual approach, the different quality criteria of guidance can be examined from two different perspectives, enhancing the overall value of evaluation studies. To test the practical utility of our methodology, we employ it in two studies to gain insight into the quality of two guidance-enhanced visual analytics solutions, one being a work-in-progress research prototype, and the other being a publicly available visualization recommender system. Based on these two evaluations, we derive good practices for conducting evaluations of guidance in visual analytics and identify pitfalls to be avoided during such studies.
在复杂数据的探索和分析过程中，引导可以为用户提供支持。以往的研究主要集中在描述视觉分析中引导的理论方面，以及在不同场景中实现引导。然而，对引导增强型视觉分析解决方案的评估仍然是一个开放的研究问题。我们通过引入并验证一种用于视觉分析中引导的实用评估方法来解决这个问题。我们确定了八个需要满足的质量标准，并收集了关于其有效性的专家反馈。为了促进实际评估研究，我们得出了两组启发式方法。第一组针对专家评估人员进行的启发式评估。第二组便于进行最终用户研究，即参与者实际使用引导增强型系统。通过采用这种双重方法，可以从两个不同的角度检查引导的不同质量标准，从而提高评估研究的整体价值。为了测试我们方法的实际效用，我们在两项研究中使用它，以深入了解两种引导增强型视觉分析解决方案的质量，一种是正在进行的研究原型，另一种是公开可用的可视化推荐系统。基于这两项评估，我们得出了在视觉分析中进行引导评估的良好实践，并确定了在此类研究中应避免的陷阱。
