Nabożny Aleksandra, Balcerzak Bartłomiej, Morzy Mikołaj, Wierzbicki Adam, Savov Pavel, Warpechowski Kamil
Gdańsk University of Technology, Gdańsk, Poland.
Polish-Japanese Academy of Information Technology, Warsaw, Poland.
World Wide Web. 2023;26(2):773-798. doi: 10.1007/s11280-022-01084-5. Epub 2022 Aug 12.
Fighting medical disinformation in the era of the pandemic is an increasingly important problem. Today, automatic systems for assessing the credibility of medical information do not offer sufficient precision, so human supervision and the involvement of medical expert annotators are required. Our work aims to optimize the utilization of medical experts' time. We also equip them with tools for semi-automatic initial verification of the credibility of the annotated content. We introduce a general framework for filtering medical statements that do not require manual evaluation by medical experts, thus focusing annotation efforts on non-credible medical statements. Our framework is based on the construction of filtering classifiers adapted to narrow thematic categories. This allows medical experts to fact-check and identify over two times more non-credible medical statements in a given time interval without applying any changes to the annotation flow. We verify our results across a broad spectrum of medical topic areas. We perform quantitative, as well as exploratory analysis on our output data. We also point out how those filtering classifiers can be modified to provide experts with different types of feedback without any loss of performance.
在疫情时代对抗医学虚假信息是一个日益重要的问题。如今，评估医学信息可信度的自动系统精度不足，因此需要人工监督和医学专家注释者的参与。我们的工作旨在优化医学专家时间的利用。我们还为他们配备了用于对注释内容的可信度进行半自动初步验证的工具。我们引入了一个用于筛选无需医学专家人工评估的医学陈述的通用框架，从而将注释工作集中在不可信的医学陈述上。我们的框架基于构建适用于狭窄主题类别的筛选分类器。这使得医学专家能够在给定时间间隔内对两倍以上的不可信医学陈述进行事实核查和识别，而无需对注释流程进行任何更改。我们在广泛的医学主题领域验证了我们的结果。我们对输出数据进行了定量分析以及探索性分析。我们还指出了如何修改那些筛选分类器，以便在不损失任何性能的情况下为专家提供不同类型的反馈。
