Jiang Leilei, Zhu Guixiang, Sun Jianshan, Cao Jie, Wu Jia
College of Management, Hefei University of Technology, Hefei, 230009, China.
College of Information Engineering, Nanjing University of Finance and Economic, Nanjing, 210023, China.
Sci Rep. 2025 May 29;15(1):18777. doi: 10.1038/s41598-025-03893-w.
Large Language Models (LLMs) are transforming various aspects of our daily lives and work through their generated content, known as Artificial Intelligence Generated Content (AIGC). To effectively harness this change, it is essential to understand the limitations within these models. While extensive prior research has addressed biases in OpenAI's ChatGPT, limited attention has been given to biases present in Chinese Large Language Models (C-LLMs). This study systematically examines biases in five representative C-LLMs. We collected 90 Chinese surnames derived from authoritative demographic statistics and 12 occupations covering various professional sectors as input prompts. Each prompt was generated three times by the C-LLMs, resulting in a dataset comprising 16,200 generated personal profiles. We then evaluated these profiles for biases regarding gender, region, age, and educational background. Our findings reveal that the content produced by each examined C-LLMs exhibits significant gender and regional biases, as well as age and educational stereotypes. Notably, while most models can generate some unbiased content, ChatGLM stands out as the exception. In contrast, Tongyiqianwen is the only model that may refuse to generate certain content, due to its strong privacy protection mechanisms. We also further analyze the underlying mechanisms of bias formation by examining different stages of the model lifecycle and considering the unique characteristics of the Chinese linguistic and sociocultural context. This paper will contribute substantially to the literature on biases in C-LLMs and provide important insights for users aiming to utilize these models more effectively and ethically.
大语言模型（LLMs）正在通过其生成的内容，即人工智能生成内容（AIGC），改变我们日常生活和工作的各个方面。为了有效利用这一变化，了解这些模型的局限性至关重要。虽然之前有大量研究探讨了OpenAI的ChatGPT中的偏差，但对中国大语言模型（C-LLMs）中存在的偏差关注较少。本研究系统地考察了五个有代表性的C-LLMs中的偏差。我们从权威人口统计数据中收集了90个中国姓氏，并选取了涵盖各个专业领域的12种职业作为输入提示。每个提示由C-LLMs生成三次，从而得到一个包含16200个生成的个人资料的数据集。然后，我们评估了这些资料在性别、地区、年龄和教育背景方面的偏差。我们的研究结果表明，每个被考察的C-LLMs生成的内容都存在显著的性别和地区偏差，以及年龄和教育刻板印象。值得注意的是，虽然大多数模型都能生成一些无偏差的内容，但ChatGLM是个例外。相比之下，由于其强大的隐私保护机制，通义千问是唯一可能拒绝生成某些内容的模型。我们还通过考察模型生命周期的不同阶段，并考虑中国语言和社会文化背景的独特特征，进一步分析了偏差形成的潜在机制。本文将为有关C-LLMs偏差的文献做出重大贡献，并为旨在更有效、更符合道德地使用这些模型的用户提供重要见解。
