Biro Joshua, Linder Courtney, Neyens David
Department of Industrial Engineering, Clemson University, Clemson, SC, United States.
JMIR Hum Factors. 2023 Feb 1;10:e41017. doi: 10.2196/41017.
The rising adoption of telehealth provides new opportunities for more effective and equitable health care information mediums. The ability of chatbots to provide a conversational, personal, and comprehendible avenue for learning about health care information make them a promising tool for addressing health care inequity as health care trends continue toward web-based and remote processes. Although chatbots have been studied in the health care domain for their efficacy for smoking cessation, diet recommendation, and other assistive applications, few studies have examined how specific design characteristics influence the effectiveness of chatbots in providing health information.
Our objective was to investigate the influence of different design considerations on the effectiveness of an educational health care chatbot.
A 2×3 between-subjects study was performed with 2 independent variables: a chatbot's complexity of responses (eg, technical or nontechnical language) and the presented qualifications of the chatbot's persona (eg, doctor, nurse, or nursing student). Regression models were used to evaluate the impact of these variables on 3 outcome measures: effectiveness, usability, and trust. A qualitative transcript review was also done to review how participants engaged with the chatbot.
Analysis of 71 participants found that participants who received technical language responses were significantly more likely to be in the high effectiveness group, which had higher improvements in test scores (odds ratio [OR] 2.73, 95% CI 1.05-7.41; P=.04). Participants with higher health literacy (OR 2.04, 95% CI 1.11-4.00, P=.03) were significantly more likely to trust the chatbot. The participants engaged with the chatbot in a variety of ways, with some taking a conversational approach and others treating the chatbot more like a search engine.
Given their increasing popularity, it is vital that we consider how chatbots are designed and implemented. This study showed that factors such as chatbots' persona and language complexity are two design considerations that influence the ability of chatbots to successfully provide health care information.
远程医疗的日益普及为更有效和公平的医疗保健信息媒介提供了新机遇。聊天机器人能够提供一种对话式、个性化且易于理解的途径来了解医疗保健信息，随着医疗保健趋势持续朝着基于网络和远程流程发展，这使其成为解决医疗保健不平等问题的一个有前景的工具。尽管聊天机器人在医疗保健领域已针对其戒烟、饮食建议及其他辅助应用的功效进行了研究，但很少有研究考察特定设计特征如何影响聊天机器人在提供健康信息方面的有效性。
我们的目的是研究不同设计考量对教育型医疗保健聊天机器人有效性的影响。
进行了一项2×3被试间研究，有两个自变量：聊天机器人回复的复杂性（例如，专业或非专业语言）以及聊天机器人角色所呈现的资质（例如，医生、护士或护理专业学生）。回归模型用于评估这些变量对三个结果指标的影响：有效性、可用性和信任度。还进行了定性的文字记录审查，以考察参与者与聊天机器人的互动方式。
对71名参与者的分析发现，收到专业语言回复的参与者显著更有可能属于高效能组，该组在测试分数上有更高的提升（优势比[OR]2.73，95%置信区间1.05 - 7.41；P = 0.04）。健康素养较高的参与者（OR 2.04，95%置信区间1.11 - 4.00，P = 0.03）显著更有可能信任聊天机器人。参与者以多种方式与聊天机器人互动，一些人采用对话方式，而另一些人则更将聊天机器人当作搜索引擎。
鉴于聊天机器人越来越受欢迎，我们必须考虑其设计和实施方式。本研究表明，聊天机器人的角色和语言复杂性等因素是影响聊天机器人成功提供医疗保健信息能力的两个设计考量因素。
