Lee Sam Yu-Te, Ma Kwan-Liu
IEEE Trans Vis Comput Graph. 2025 Sep;31(9):5532-5546. doi: 10.1109/TVCG.2024.3459961.
Sensemaking on a large collection of documents (corpus) is a challenging task often found in fields such as market research, legal studies, intelligence analysis, political science, or computational linguistics. Previous works approach this problem from topic- and entity-based perspectives, but the capability of the underlying NLP model limits their effectiveness. Recent advances in prompting with LLMs present opportunities to enhance such approaches with higher accuracy and customizability. However, poorly designed prompts and visualizations could mislead users into falsely interpreting the visualizations and hinder the system's trustworthiness. In this paper, we address this issue by taking into account the user analysis tasks and visualization goals in the prompt-based data extraction stage, thereby extending the concept of Model Alignment. We present HINTs, a VA system for supporting sensemaking on large collections of documents, combining previous entity-based and topic-based approaches. The visualization pipeline of HINTs consists of three stages. First, entities and topics are extracted from the corpus with prompts. Then, the result is modeled as a hypergraph and hierarchically clustered. Finally, an enhanced space-filling curve layout is applied to visualize the hypergraph for interactive exploration. The system further integrates an LLM-based intelligent chatbot agent in the interface to facilitate the sensemaking of interested documents. To demonstrate the generalizability and effectiveness of the HINTs system, we present two case studies on different domains and a comparative user study. We report our insights on the behavior patterns and challenges when intelligent agents are used to facilitate sensemaking. We find that while intelligent agents can address many challenges in sensemaking, the visual hints that visualizations provide are still necessary. We discuss limitations and future work for combining interactive visualization and LLMs more profoundly to better support corpus analysis.
对大量文档（语料库）进行意义构建是一项具有挑战性的任务，常见于市场研究、法律研究、情报分析、政治学或计算语言学等领域。以往的工作从基于主题和实体的角度来处理这个问题，但底层自然语言处理模型的能力限制了它们的有效性。基于大语言模型的提示技术的最新进展为以更高的准确性和可定制性增强此类方法提供了机会。然而，设计不佳的提示和可视化可能会误导用户错误地解释可视化结果，并阻碍系统的可信度。在本文中，我们通过在基于提示的数据提取阶段考虑用户分析任务和可视化目标来解决这个问题，从而扩展了模型对齐的概念。我们提出了HINTs，这是一个用于支持对大量文档进行意义构建的可视化分析系统，它结合了以前基于实体和基于主题的方法。HINTs的可视化管道包括三个阶段。首先，使用提示从语料库中提取实体和主题。然后，将结果建模为超图并进行层次聚类。最后，应用增强的空间填充曲线布局来可视化超图，以便进行交互式探索。该系统还在界面中集成了一个基于大语言模型的智能聊天机器人代理，以促进对感兴趣文档的意义构建。为了证明HINTs系统的通用性和有效性，我们展示了两个不同领域的案例研究以及一项比较用户研究。我们报告了在使用智能代理促进意义构建时关于行为模式和挑战的见解。我们发现，虽然智能代理可以解决意义构建中的许多挑战，但可视化提供的视觉提示仍然是必要的。我们讨论了更深入地结合交互式可视化和大语言模型以更好地支持语料库分析的局限性和未来工作。
