Yamamoto Shintaro, Lauscher Anne, Ponzetto Simone Paolo, Glavaš Goran, Morishima Shigeo
Department of Pure and Applied Physics, Waseda University, Tokyo, Japan.
Data and Web Science Group, University of Mannheim, Mannheim, Germany.
Front Res Metr Anal. 2021 Aug 19;6:719004. doi: 10.3389/frma.2021.719004. eCollection 2021.
The exponential growth of scientific literature yields the need to support users to both effectively and efficiently analyze and understand the some body of research work. This exploratory process can be facilitated by providing graphical abstracts-a visual summary of a scientific publication. Accordingly, previous work recently presented an initial study on automatic identification of a central figure in a scientific publication, to be used as the publication's visual summary. This study, however, have been limited only to a single (biomedical) domain. This is primarily because the current state-of-the-art relies on supervised machine learning, typically relying on the existence of large amounts of labeled data: the only existing annotated data set until now covered only the biomedical publications. In this work, we build a novel benchmark data set for visual summary identification from scientific publications, which consists of papers presented at conferences from several areas of computer science. We couple this contribution with a new self-supervised learning approach to learn a heuristic matching of in-text references to figures with figure captions. Our self-supervised pre-training, executed on a large unlabeled collection of publications, attenuates the need for large annotated data sets for visual summary identification and facilitates domain transfer for this task. We evaluate our self-supervised pretraining for visual summary identification on both the existing biomedical and our newly presented computer science data set. The experimental results suggest that the proposed method is able to outperform the previous state-of-the-art without any task-specific annotations.
科学文献的指数级增长使得有必要支持用户有效且高效地分析和理解某一研究工作主体。通过提供图形摘要（科学出版物的视觉总结）可以促进这一探索过程。因此，先前的工作最近提出了一项关于自动识别科学出版物中核心人物以用作出版物视觉总结的初步研究。然而，这项研究仅限于单一（生物医学）领域。这主要是因为当前的先进技术依赖于监督机器学习，通常依赖于大量标记数据的存在：到目前为止，唯一现有的注释数据集仅涵盖生物医学出版物。在这项工作中，我们为从科学出版物中识别视觉总结构建了一个新的基准数据集，该数据集由计算机科学几个领域的会议上发表的论文组成。我们将这一贡献与一种新的自监督学习方法相结合，以学习文本参考文献与带有图注的图表之间启发式匹配。我们在大量未标记的出版物集合上执行的自监督预训练减少了对用于视觉总结识别的大型注释数据集的需求，并促进了该任务的领域转移。我们在现有的生物医学数据集和我们新提出的计算机科学数据集上评估了用于视觉总结识别的自监督预训练。实验结果表明，所提出的方法能够在没有任何特定任务注释的情况下优于先前的先进技术。
