Abu-Salih Bilal, Alhabashneh Mohammad, Zhu Dengya, Awajan Albara, Alshamaileh Yazan, Al-Shboul Bashar, Alshraideh Mohammad
The University of Jordan, Amman, Jordan.
Curtin University, Perth, Australia.
Heliyon. 2023 Apr 28;9(5):e15926. doi: 10.1016/j.heliyon.2023.e15926. eCollection 2023 May.
The development of emotion detection technology has emerged as an efficient possibility in the corporate sector due to the nearly limitless uses of this new discipline, particularly with the unceasing propagation of social data. In recent years, the electronic marketplace has witnessed the establishment of various start-up businesses with an almost sole focus on building new commercial and open-source tools and APIs for emotion detection and recognition. Yet, these tools and APIs must be continuously reviewed and evaluated, and their performances should be reported and discussed. There is a lack of research to empirically compare current emotion detection technologies in terms of the results obtained from each model using the same textual dataset. Also, there is a lack of comparative studies that apply benchmark comparisons to social data. This study compares eight technologies: IBM Watson Natural Language Understanding, ParallelDots, Symanto - Ekman, Crystalfeel, Text to Emotion, Senpy, Textprobe, and Natural Language Processing Cloud. The comparison was undertaken using two different datasets. The emotions from the chosen datasets were then derived using the incorporated APIs. The performance of these APIs was assessed using the aggregated scores they delivered and the theoretically proven evaluation metrics such as the micro-average of accuracy, classification error, precision, recall, and f1-score. Lastly, the assessment of these APIs incorporating the evaluation measures is reported and discussed.
由于这一新学科几乎有无穷无尽的用途，尤其是随着社交数据的不断传播，情感检测技术的发展已成为企业领域一种有效的可能性。近年来，电子市场见证了各种初创企业的成立，这些企业几乎只专注于开发用于情感检测和识别的新商业及开源工具与应用程序编程接口（API）。然而，这些工具和API必须不断接受审查和评估，其性能也应予以报告和讨论。目前缺乏实证研究来比较当前情感检测技术在使用相同文本数据集从每个模型获得的结果方面的情况。此外，也缺乏将基准比较应用于社交数据的比较研究。本研究比较了八种技术：IBM Watson自然语言理解、ParallelDots、Symanto - 埃克曼、Crystalfeel、文本转情感、Senpy、Textprobe和自然语言处理云。比较是使用两个不同的数据集进行的。然后使用所包含的API从选定的数据集中提取情感。这些API的性能通过它们给出的综合分数以及诸如准确率的微观平均值、分类误差、精确率、召回率和F1分数等经过理论验证的评估指标来评估。最后，报告并讨论了结合评估措施对这些API的评估情况。
