Sharifpoor Elham, Okhovati Maryam, Ghazizadeh-Ahsaee Mostafa, Avaz Beigi Mina
Medical Library and Information Sciences Department, Medical Informatics Research Center, Institute for Futures Studies in Health, Kerman University of Medical Sciences, Kerman, Iran.
Department of Computer Engineering, Shahid Bahonar University of Kerman, Kerman, Iran.
BMC Med Inform Decis Mak. 2025 Feb 11;25(1):73. doi: 10.1186/s12911-025-02895-y.
Despite recent progress in misinformation detection methods, further investigation is required to develop more robust fact-checking models with particular consideration for the unique challenges of health information sharing. This study aimed to identify the most effective approach for detecting and classifying reliable information versus misinformation health content shared on Twitter/X related to COVID-19.
We have used 7 different machine learning/deep learning models. Tweets were collected, processed, labeled, and analyzed using relevant keywords and hashtags, then classified into two distinct datasets: "Trustworthy information" versus "Misinformation", through a labeling process. The cosine similarity metric was employed to address oversampling the minority of the Trustworthy information class, ensuring a more balanced representation of both classes for training and testing purposes. Finally, the performance of the various fact-checking models was analyzed and compared using accuracy, precision, recall, and F1-score ROC curve, and AUC.
For measures of accuracy, precision, F1 score, and recall, the average values of TextConvoNet were found to be 90.28, 90.28, 90.29, and 0.9030, respectively. ROC AUC was 0.901."Trustworthy information" class achieved an accuracy of 85%, precision of 93%, recall of 86%, and F1 score of 89%. These values were higher than other models. Moreover, its performance in the misinformation category was even more impressive, with an accuracy of 94%, precision of 88%, recall of 94%, and F1 score of 91%.
This study showed that TextConvoNet was the most effective in detecting and classifying trustworthy information V.S misinformation related to health issues that have been shared on Twitter/X.
尽管最近在错误信息检测方法方面取得了进展，但仍需要进一步研究，以开发出更强大的事实核查模型，尤其要考虑到健康信息共享所面临的独特挑战。本研究旨在确定检测和分类在推特/X上分享的与COVID-19相关的可靠信息与错误信息健康内容的最有效方法。
我们使用了7种不同的机器学习/深度学习模型。通过相关关键词和主题标签收集、处理、标记和分析推文，然后通过标记过程将其分类为两个不同的数据集：“可信信息”与“错误信息”。采用余弦相似性度量来处理可信信息类中少数类的过采样问题，以确保两个类在训练和测试目的上有更平衡的表示。最后，使用准确率、精确率、召回率、F1分数、ROC曲线和AUC对各种事实核查模型的性能进行分析和比较。
在准确率、精确率、F1分数和召回率方面，TextConvoNet的平均值分别为90.28、90.28、90.29和0.9030。ROC AUC为0.901。“可信信息”类的准确率为85%，精确率为93%，召回率为86%，F1分数为89%。这些值高于其他模型。此外，它在错误信息类别中的表现更令人印象深刻，准确率为94%，精确率为88%，召回率为94%，F1分数为91%。
本研究表明，TextConvoNet在检测和分类推特/X上分享的与健康问题相关的可信信息与错误信息方面最为有效。
