School of Health Sciences, Swinburne University of Technology, Hawthorn, Australia.
School of Computing, Engineering and Mathematical Sciences, La Trobe University, Bundoora, Australia.
JMIR Ment Health. 2024 Sep 9;11:e58974. doi: 10.2196/58974.
The demand for mental health (MH) services in the community continues to exceed supply. At the same time, technological developments make the use of artificial intelligence-empowered conversational agents (CAs) a real possibility to help fill this gap.
The objective of this review was to identify existing empathic CA design architectures within the MH care sector and to assess their technical performance in detecting and responding to user emotions in terms of classification accuracy. In addition, the approaches used to evaluate empathic CAs within the MH care sector in terms of their acceptability to users were considered. Finally, this review aimed to identify limitations and future directions for empathic CAs in MH care.
A systematic literature search was conducted across 6 academic databases to identify journal articles and conference proceedings using search terms covering 3 topics: "conversational agents," "mental health," and "empathy." Only studies discussing CA interventions for the MH care domain were eligible for this review, with both textual and vocal characteristics considered as possible data inputs. Quality was assessed using appropriate risk of bias and quality tools.
A total of 19 articles met all inclusion criteria. Most (12/19, 63%) of these empathic CA designs in MH care were machine learning (ML) based, with 26% (5/19) hybrid engines and 11% (2/19) rule-based systems. Among the ML-based CAs, 47% (9/19) used neural networks, with transformer-based architectures being well represented (7/19, 37%). The remaining 16% (3/19) of the ML models were unspecified. Technical assessments of these CAs focused on response accuracies and their ability to recognize, predict, and classify user emotions. While single-engine CAs demonstrated good accuracy, the hybrid engines achieved higher accuracy and provided more nuanced responses. Of the 19 studies, human evaluations were conducted in 16 (84%), with only 5 (26%) focusing directly on the CA's empathic features. All these papers used self-reports for measuring empathy, including single or multiple (scale) ratings or qualitative feedback from in-depth interviews. Only 1 (5%) paper included evaluations by both CA users and experts, adding more value to the process.
The integration of CA design and its evaluation is crucial to produce empathic CAs. Future studies should focus on using a clear definition of empathy and standardized scales for empathy measurement, ideally including expert assessment. In addition, the diversity in measures used for technical assessment and evaluation poses a challenge for comparing CA performances, which future research should also address. However, CAs with good technical and empathic performance are already available to users of MH care services, showing promise for new applications, such as helpline services.
社区对心理健康 (MH) 服务的需求持续超过供应。与此同时，技术发展使得使用人工智能赋能的对话代理 (CA) 成为帮助填补这一空白的现实可能性。
本综述的目的是确定 MH 护理领域中现有的移情 CA 设计架构，并根据分类准确性评估它们在检测和响应用户情绪方面的技术性能。此外，还考虑了 MH 护理领域内用于评估移情 CA 的方法，以评估其对用户的可接受性。最后，本综述旨在确定 MH 护理中移情 CA 的局限性和未来方向。
对 6 个学术数据库进行了系统的文献检索，以确定使用涵盖 3 个主题的期刊文章和会议记录：“对话代理”、“心理健康”和“同理心”。只有讨论 MH 护理领域 CA 干预的文章符合本综述的纳入标准，文本和语音特征均被视为可能的数据输入。使用适当的偏倚风险和质量工具评估质量。
共有 19 篇文章符合所有纳入标准。这些 MH 护理中的移情 CA 设计中，大多数（12/19，63%）是基于机器学习 (ML) 的，其中 26%（5/19）是混合引擎，11%（2/19）是基于规则的系统。在基于 ML 的 CA 中，47%（9/19）使用神经网络，基于转换器的架构得到了很好的体现（7/19，37%）。剩下的 16%（3/19）的 ML 模型未指定。这些 CA 的技术评估侧重于响应准确性及其识别、预测和分类用户情绪的能力。虽然单引擎 CA 表现出良好的准确性，但混合引擎实现了更高的准确性，并提供了更细致的响应。在这 19 项研究中，有 16 项（84%）进行了人类评估，其中只有 5 项（26%）直接关注 CA 的移情特征。所有这些论文都使用自我报告来衡量同理心，包括单或多（量表）评分或深入访谈的定性反馈。只有 1 篇（5%）论文同时包括 CA 用户和专家的评估，为该过程增加了更多价值。
CA 设计及其评估的结合对于产生移情 CA 至关重要。未来的研究应侧重于使用同理心的明确定义和同理心测量的标准化量表，理想情况下包括专家评估。此外，技术评估和评估中使用的措施多样性给 CA 性能的比较带来了挑战，未来的研究也应该解决这一问题。然而，具有良好技术和移情性能的 CA 已经可供 MH 护理服务的用户使用，为新的应用程序（如求助热线服务）展示了前景。
