Osborne Merrick R, Bailey Erica R
U.C. Berkeley, Haas School of Business, Berkeley, United States.
Sci Rep. 2025 Feb 1;15(1):3980. doi: 10.1038/s41598-025-86623-6.
Artificial intelligence ("AI") has the potential to vastly improve human decision-making. In line with this, researchers have increasingly sought to understand how people view AI, often documenting skepticism and even outright aversion to these tools. In the present research, we complement these findings by documenting the performance of LLMs in the personal advice domain. In addition, we shift the focus in a new direction-exploring how interacting with AI tools, specifically large language models, impacts the user's view of themselves. In five preregistered experiments (N = 1,722), we explore evaluations of human- and ChatGPT-generated advice along three dimensions: quality, effectiveness, and authenticity. We find that ChatGPT produces superior advice relative to the average online participant even in a domain in which people strongly prefer human-generated advice (dating and relationships). We also document a bias against ChatGPT-generated advice which is present only when participants are aware the advice was generated by ChatGPT. Novel to the present investigation, we then explore how interacting with these tools impacts self-evaluations. We manipulate the order in which people interact with these tools relative to self-generation and find that generating advice before interacting with ChatGPT advice boosts the quality ratings of the ChatGPT advice. At the same time, interacting with ChatGPT-generated advice before self-generating advice decreases self-ratings of authenticity. Taken together, we document a bias towards AI in the context of personal advice. Further, we identify an important externality in the use of these tools-they can invoke social comparisons of me vs. the machine.
人工智能（“AI”）有潜力极大地改善人类决策。与此一致的是，研究人员越来越多地试图了解人们如何看待人工智能，经常记录下对这些工具的怀疑甚至完全厌恶。在本研究中，我们通过记录大语言模型在个人建议领域的表现来补充这些发现。此外，我们将重点转向一个新方向——探索与人工智能工具，特别是大语言模型的交互如何影响用户对自己的看法。在五个预先注册的实验（N = 1722）中，我们从三个维度探索对人类生成和ChatGPT生成的建议的评价：质量、有效性和真实性。我们发现，即使在人们强烈倾向于人类生成的建议的领域（约会和人际关系），ChatGPT生成的建议相对于普通在线参与者而言也更优。我们还记录了对ChatGPT生成的建议的偏见，这种偏见仅在参与者意识到建议是由ChatGPT生成时才会出现。本研究的新颖之处在于，我们随后探索了与这些工具的交互如何影响自我评估。我们操纵人们与这些工具交互相对于自我生成的顺序，发现先与ChatGPT的建议交互再自我生成建议会提高对ChatGPT建议的质量评级。与此同时，在自我生成建议之前与ChatGPT生成的建议交互会降低对真实性的自我评级。综上所述，我们记录了在个人建议背景下对人工智能的偏见。此外，我们识别出使用这些工具时一个重要的外部效应——它们会引发我与机器的社会比较。
