Lister Hill National Center for Biomedical Communications, National Library of Medicine, Bethesda, MD 20894, United States.
Lister Hill National Center for Biomedical Communications, National Library of Medicine, Bethesda, MD 20894, United States.
J Biomed Inform. 2019 Mar;91:103123. doi: 10.1016/j.jbi.2019.103123. Epub 2019 Feb 10.
Quantifying scientific impact of researchers and journals relies largely on citation counts, despite the acknowledged limitations of this approach. The need for more suitable alternatives has prompted research into developing advanced metrics, such as h-index and Relative Citation Ratio (RCR), as well as better citation categorization schemes to capture the various functions that citations serve in a publication. One such scheme involves citation sentiment: whether a reference paper is cited positively (agreement with the findings of the reference paper), negatively (disagreement), or neutrally. The ability to classify citation function in this manner can be viewed as a first step toward a more fine-grained bibliometrics. In this study, we compared several approaches, varying in complexity, for classification of citation sentiment in clinical trial publications. Using a corpus of 285 discussion sections from as many publications (a total of 4,182 citations), we developed a rule-based method as well as supervised machine learning models based on support vector machines (SVM) and two variants of deep neural networks; namely, convolutional neural network (CNN) and bidirectional long short-term memory (BiLSTM). A CNN model augmented with hand-crafted features yielded the best performance (0.882 accuracy and 0.721 macro-F on held-out set). Our results show that baseline performances of traditional supervised learning algorithms and deep neural network architectures are similar and that hand-crafted features based on sentiment dictionaries and rhetorical structure allow neural network approaches to outperform traditional machine learning approaches for this task. We make the rule-based method and the best-performing neural network model publicly available at: https://github.com/kilicogluh/clinical-citation-sentiment.
尽管这种方法存在公认的局限性,但研究者和期刊的科学影响力的量化在很大程度上仍然依赖于引文数量。因此,需要开发更合适的替代方法,这促使研究人员开发了更先进的指标,如 h 指数和相对引文率(RCR),以及更好的引文分类方案,以捕捉引文在出版物中所起的各种作用。其中一种方案涉及引文情绪:参考论文是被正面引用(与参考论文的发现一致)、负面引用(不一致)还是中性引用。以这种方式对引文功能进行分类的能力可以被视为迈向更细粒度的文献计量学的第一步。在这项研究中,我们比较了几种方法,这些方法在复杂性上有所不同,用于对临床试验出版物中的引文情绪进行分类。我们使用了一个由 285 个讨论部分组成的语料库(共计 4182 条引文),开发了一种基于规则的方法,以及基于支持向量机(SVM)和两种深度神经网络变体的监督机器学习模型,即卷积神经网络(CNN)和双向长短期记忆网络(BiLSTM)。一个带有手工制作特征的 CNN 模型取得了最好的性能(在验证集上的准确率为 0.882,宏 F 值为 0.721)。我们的结果表明,传统监督学习算法和深度神经网络架构的基线性能相似,并且基于情感词典和修辞结构的手工制作特征允许神经网络方法在这项任务中优于传统机器学习方法。我们将基于规则的方法和表现最好的神经网络模型公开在:https://github.com/kilicogluh/clinical-citation-sentiment。