Department of Philosophy, Social and Human Sciences and Education, University of Perugia, Perugia, Italy.
Department of Cognitive Psychology and Ergonomics, Faculty of BMS, University of Twente, Enschede, The Netherlands.
Disabil Rehabil Assist Technol. 2020 Oct;15(7):832-837. doi: 10.1080/17483107.2020.1775313. Epub 2020 Jun 18.
People with disabilities or special needs can benefit from AI-based conversational agents (i.e., chatbots) that are used for competence training and well-being management. Assessing the quality of interactions with these chatbots is key to being able to reduce dissatisfaction with them and to understanding their potential long-term benefit. This in turn will help to increase adherence to their use, thereby improving the quality of life of the large population of end-users that they are able to serve. Following Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) methodology, we systematically reviewed the literature on methods of assessing the perceived quality of interactions with chatbots using the from Scopus and the Web of Science electronic databases. Using the Boolean operators (AND/OR) the keywords chatbot*, conversational agent*, special needs, disability were combined. Revealed that only 15 of 192 papers on this topic included people with disabilities or special needs in their assessments. The results also highlighted the lack of a shared theoretical framework for assessing the perceived quality of interactions with chatbots. Systematic procedures based on reliable and valid methodologies continue to be needed in this field. The current lack of reliable tools and systematic methods to assess chatbots for people with disabilities and special needs is concerning, and ultimately, it may also lead to unreliable systems entering the market with disruptive consequences for people.Implications for rehabilitationChatbots applied in rehabilitation are mainly tested in terms of clinical effectiveness and validity with a minimal focus on measuring the quality of the interactionThe usability and interactive properties of chatbots applied in rehabilitation are not comparable as each tool is measured in different wayThe lack of a common framework to assess chatbots exposes people with disability and special needs to the risk of using unreliable tools.
残疾人或有特殊需求的人可以从基于人工智能的对话代理（即聊天机器人）中受益，这些代理可用于能力培训和幸福感管理。评估与这些聊天机器人的交互质量是减少对它们的不满并了解其潜在长期效益的关键。这反过来将有助于提高对其使用的依从性，从而提高它们能够服务的大量最终用户的生活质量。根据系统评价和荟萃分析的首选报告项目 (PRISMA) 方法，我们系统地回顾了使用 Scopus 和 Web of Science 电子数据库评估与聊天机器人交互感知质量的方法的文献。使用布尔运算符（AND/OR），将关键字“chatbot*”、“conversational agent*”、“特殊需求”、“残疾”结合起来。结果表明，在这个主题的 192 篇论文中，只有 15 篇论文将残疾人和有特殊需求的人纳入评估。结果还突出了缺乏评估与聊天机器人交互感知质量的共享理论框架。该领域仍需要基于可靠和有效的方法的系统程序。当前，缺乏可靠的工具和系统的方法来评估残疾人和有特殊需求的人的聊天机器人令人担忧，最终，这也可能导致不可靠的系统进入市场，给人们带来破坏性的后果。康复的意义康复中应用的聊天机器人主要是根据临床效果和有效性进行测试，而很少关注衡量交互质量康复中应用的聊天机器人的可用性和交互属性不可比，因为每个工具都以不同的方式进行测量缺乏评估聊天机器人的通用框架使残疾人和有特殊需求的人面临使用不可靠工具的风险。
