Hao Guozhi, Wu Jun, Pan Qianqian, Morello Rosario
Graduate School of Information, Production and Systems, Waseda University, Fukuoka, 808-0135, Japan.
Department of Information Engineering, Infrastructure and Sustainable Energy, University Mediterranea of Reggio Calabria, Via Graziella, Reggio Calabria, 89122, Italy.
Sci Rep. 2024 Jul 16;14(1):16375. doi: 10.1038/s41598-024-66708-4.
Large language models (LLMs) are becoming a significant source of content generation in social networks, which is a typical complex adaptive system (CAS). However, due to their hallucinatory nature, LLMs produce false information that can spread through social networks, which will impact the stability of the whole society. The uncertainty of LLMs false information spread within social networks is attributable to the diversity of individual behaviors, intricate interconnectivity, and dynamic network structures. Quantifying the uncertainty of false information spread by LLMs in social networks is beneficial for preemptively devising strategies to defend against threats. To address these challenges, we propose an LLMs hallucination-aware dynamic modeling method via agent-based probability distributions, spread popularity and community affiliation, to quantify the uncertain spreading of LLMs hallucination in social networks. We set up the node attributes and behaviors in the model based on real-world data. For evaluation, we consider the spreaders, informed people, discerning and unwilling non-spreaders as indicators, and quantified the spreading under different LLMs task situations, such as QA, dialogue, and summarization, as well as LLMs versions. Furthermore, we conduct experiments using real-world LLM hallucination data combined with social network features to ensure the validity of the proposed quantifying scheme.
大语言模型(LLMs)正在成为社交网络中内容生成的一个重要来源,社交网络是一个典型的复杂自适应系统(CAS)。然而,由于其幻觉性质,大语言模型会产生可能在社交网络中传播的虚假信息,这将影响整个社会的稳定性。大语言模型虚假信息在社交网络内传播的不确定性归因于个体行为的多样性、错综复杂的相互关联性以及动态的网络结构。量化大语言模型在社交网络中传播虚假信息的不确定性,有利于预先制定抵御威胁的策略。为应对这些挑战,我们提出一种基于智能体概率分布、传播热度和社区归属的大语言模型幻觉感知动态建模方法,以量化大语言模型幻觉在社交网络中的不确定传播。我们基于现实世界数据设置模型中的节点属性和行为。为进行评估,我们将传播者、知情者、有辨别力且不愿意传播者作为指标,并量化了在不同大语言模型任务情境(如问答、对话和摘要)以及大语言模型版本下的传播情况。此外,我们结合社交网络特征使用现实世界的大语言模型幻觉数据进行实验,以确保所提出量化方案的有效性。
J Med Internet Res. 2024-5-13
J Esthet Restor Dent. 2023-10
JMIR Med Inform. 2024-5-10
NPJ Digit Med. 2023-12-2
Ann Bot. 2020-9-14