Kim Jin Yong, Lester Corey, Yang X Jessie
University of Michigan, USA.
Hum Factors. 2025 Mar 19:187208251326795. doi: 10.1177/00187208251326795.
ObjectiveWe investigated how various error patterns from an AI aid in the nonbinary decision scenario influence human operators' trust in the AI system and their task performance.BackgroundExisting research on trust in automation/autonomy predominantly uses the signal detection theory (SDT) to model autonomy performance. The SDT classifies the world into binary states and hence oversimplifies the interaction observed in real-world scenarios. Allowing multi-class classification of the world reveals intriguing error patterns previously unexplored in prior literature.MethodThirty-five participants completed 60 trials of a simulated mental rotation task assisted by an AI with 70-80% reliability. Participants' trust in and dependence on the AI system and their performance were measured. By combining participants' initial performance and the AI aid's performance, five distinct patterns emerged. Mixed-effects models were built to examine the effects of different patterns on trust adjustment, performance, and reaction time.ResultsVarying error patterns from AI impacted performance, reaction times, and trust. Some AI errors provided false reassurance, misleading operators into believing their incorrect decisions were correct, worsening performance and trust. Paradoxically, some AI errors prompted safety checks and verifications, which, despite causing a moderate decrease in trust, ultimately enhanced overall performance.ConclusionThe findings demonstrate that the types of errors made by an AI system significantly affect human trust and performance, emphasizing the need to model the complicated human-AI interaction in real life.ApplicationThese insights can guide the development of AI systems that classify the state of the world into multiple classes, enabling the operators to make more informed and accurate decisions based on feedback.
目的
我们研究了人工智能辅助在非二元决策场景中的各种错误模式如何影响人类操作员对人工智能系统的信任及其任务表现。
背景
现有关于对自动化/自主性信任的研究主要使用信号检测理论（SDT）来模拟自主性表现。信号检测理论将世界分为二元状态，因此过度简化了现实世界场景中观察到的交互。允许对世界进行多类别分类揭示了先前文献中未探索的有趣错误模式。
方法
35名参与者完成了60次由可靠性为70%-80%的人工智能辅助的模拟心理旋转任务试验。测量了参与者对人工智能系统的信任和依赖以及他们的表现。通过结合参与者的初始表现和人工智能辅助的表现，出现了五种不同的模式。建立混合效应模型以检查不同模式对信任调整、表现和反应时间的影响。
结果
人工智能的不同错误模式影响了表现、反应时间和信任。一些人工智能错误提供了虚假的安心感，误导操作员相信他们的错误决策是正确的，从而使表现和信任恶化。矛盾的是，一些人工智能错误促使进行安全检查和验证，尽管这导致信任适度下降，但最终提高了整体表现。
结论
研究结果表明，人工智能系统所犯错误的类型会显著影响人类的信任和表现，强调了在现实生活中对复杂的人机交互进行建模的必要性。
应用
这些见解可以指导将世界状态分类为多个类别的人工智能系统的开发，使操作员能够根据反馈做出更明智和准确的决策。
