The Institute of Software, Chinese Academy of Sciences, Beijing, China.
Department of Computer Science, University of Toronto, Toronto, ON, Canada.
J Med Internet Res. 2021 Jan 6;23(1):e19928. doi: 10.2196/19928.
Artificial intelligence (AI)-driven chatbots are increasingly being used in health care, but most chatbots are designed for a specific population and evaluated in controlled settings. There is little research documenting how health consumers (eg, patients and caregivers) use chatbots for self-diagnosis purposes in real-world scenarios.
The aim of this research was to understand how health chatbots are used in a real-world context, what issues and barriers exist in their usage, and how the user experience of this novel technology can be improved.
We employed a data-driven approach to analyze the system log of a widely deployed self-diagnosis chatbot in China. Our data set consisted of 47,684 consultation sessions initiated by 16,519 users over 6 months. The log data included a variety of information, including users' nonidentifiable demographic information, consultation details, diagnostic reports, and user feedback. We conducted both statistical analysis and content analysis on this heterogeneous data set.
The chatbot users spanned all age groups, including middle-aged and older adults. Users consulted the chatbot on a wide range of medical conditions, including those that often entail considerable privacy and social stigma issues. Furthermore, we distilled 2 prominent issues in the use of the chatbot: (1) a considerable number of users dropped out in the middle of their consultation sessions, and (2) some users pretended to have health concerns and used the chatbot for nontherapeutic purposes. Finally, we identified a set of user concerns regarding the use of the chatbot, including insufficient actionable information and perceived inaccurate diagnostic suggestions.
Although health chatbots are considered to be convenient tools for enhancing patient-centered care, there are issues and barriers impeding the optimal use of this novel technology. Designers and developers should employ user-centered approaches to address the issues and user concerns to achieve the best uptake and utilization. We conclude the paper by discussing several design implications, including making the chatbots more informative, easy-to-use, and trustworthy, as well as improving the onboarding experience to enhance user engagement.
人工智能（AI）驱动的聊天机器人在医疗保健领域的应用日益广泛，但大多数聊天机器人都是为特定人群设计的，并在受控环境中进行评估。很少有研究记录健康消费者（例如患者和护理人员）如何在真实场景中使用聊天机器人进行自我诊断。
本研究旨在了解健康聊天机器人在真实环境中是如何使用的，在使用过程中存在哪些问题和障碍，以及如何改进用户对这项新技术的体验。
我们采用数据驱动的方法分析了在中国广泛部署的一款自我诊断聊天机器人的系统日志。我们的数据集中包含 16519 名用户在 6 个月内发起的 47684 次咨询会话。日志数据包括用户的不可识别人口统计信息、咨询详细信息、诊断报告和用户反馈等各种信息。我们对这个异构数据集进行了统计分析和内容分析。
聊天机器人的用户涵盖了所有年龄段，包括中年和老年用户。用户咨询的医疗状况范围广泛，包括那些经常涉及到相当大的隐私和社会耻辱问题的状况。此外，我们总结出在使用聊天机器人时存在两个突出问题：（1）相当数量的用户在咨询过程中中途退出，（2）一些用户假装存在健康问题，并利用聊天机器人进行非治疗性目的。最后，我们确定了用户对使用聊天机器人的一系列关注，包括信息不足和感知诊断建议不准确。
尽管健康聊天机器人被认为是增强以患者为中心的护理的便捷工具，但仍存在一些问题和障碍阻碍了这项新技术的最佳应用。设计者和开发者应该采用以用户为中心的方法来解决这些问题和用户关注，以实现最佳的采用和利用。我们在文章结尾讨论了一些设计启示，包括使聊天机器人更加信息丰富、易于使用和值得信赖，以及改善入门体验以提高用户参与度。
