Suppr超能文献

确认或反驳?:临床研究出版物中引文情绪分类的对比研究。

Confirm or refute?: A comparative study on citation sentiment classification in clinical research publications.

机构信息

Lister Hill National Center for Biomedical Communications, National Library of Medicine, Bethesda, MD 20894, United States.

Lister Hill National Center for Biomedical Communications, National Library of Medicine, Bethesda, MD 20894, United States.

出版信息

J Biomed Inform. 2019 Mar;91:103123. doi: 10.1016/j.jbi.2019.103123. Epub 2019 Feb 10.

Abstract

Quantifying scientific impact of researchers and journals relies largely on citation counts, despite the acknowledged limitations of this approach. The need for more suitable alternatives has prompted research into developing advanced metrics, such as h-index and Relative Citation Ratio (RCR), as well as better citation categorization schemes to capture the various functions that citations serve in a publication. One such scheme involves citation sentiment: whether a reference paper is cited positively (agreement with the findings of the reference paper), negatively (disagreement), or neutrally. The ability to classify citation function in this manner can be viewed as a first step toward a more fine-grained bibliometrics. In this study, we compared several approaches, varying in complexity, for classification of citation sentiment in clinical trial publications. Using a corpus of 285 discussion sections from as many publications (a total of 4,182 citations), we developed a rule-based method as well as supervised machine learning models based on support vector machines (SVM) and two variants of deep neural networks; namely, convolutional neural network (CNN) and bidirectional long short-term memory (BiLSTM). A CNN model augmented with hand-crafted features yielded the best performance (0.882 accuracy and 0.721 macro-F on held-out set). Our results show that baseline performances of traditional supervised learning algorithms and deep neural network architectures are similar and that hand-crafted features based on sentiment dictionaries and rhetorical structure allow neural network approaches to outperform traditional machine learning approaches for this task. We make the rule-based method and the best-performing neural network model publicly available at: https://github.com/kilicogluh/clinical-citation-sentiment.

摘要

尽管这种方法存在公认的局限性,但研究者和期刊的科学影响力的量化在很大程度上仍然依赖于引文数量。因此,需要开发更合适的替代方法,这促使研究人员开发了更先进的指标,如 h 指数和相对引文率(RCR),以及更好的引文分类方案,以捕捉引文在出版物中所起的各种作用。其中一种方案涉及引文情绪:参考论文是被正面引用(与参考论文的发现一致)、负面引用(不一致)还是中性引用。以这种方式对引文功能进行分类的能力可以被视为迈向更细粒度的文献计量学的第一步。在这项研究中,我们比较了几种方法,这些方法在复杂性上有所不同,用于对临床试验出版物中的引文情绪进行分类。我们使用了一个由 285 个讨论部分组成的语料库(共计 4182 条引文),开发了一种基于规则的方法,以及基于支持向量机(SVM)和两种深度神经网络变体的监督机器学习模型,即卷积神经网络(CNN)和双向长短期记忆网络(BiLSTM)。一个带有手工制作特征的 CNN 模型取得了最好的性能(在验证集上的准确率为 0.882,宏 F 值为 0.721)。我们的结果表明,传统监督学习算法和深度神经网络架构的基线性能相似,并且基于情感词典和修辞结构的手工制作特征允许神经网络方法在这项任务中优于传统机器学习方法。我们将基于规则的方法和表现最好的神经网络模型公开在:https://github.com/kilicogluh/clinical-citation-sentiment。

相似文献

1
Confirm or refute?: A comparative study on citation sentiment classification in clinical research publications.
J Biomed Inform. 2019 Mar;91:103123. doi: 10.1016/j.jbi.2019.103123. Epub 2019 Feb 10.
2
Assessing citation integrity in biomedical publications: corpus annotation and NLP models.
Bioinformatics. 2024 Jul 1;40(7). doi: 10.1093/bioinformatics/btae420.
3
Comparing deep learning architectures for sentiment analysis on drug reviews.
J Biomed Inform. 2020 Oct;110:103539. doi: 10.1016/j.jbi.2020.103539. Epub 2020 Aug 17.
4
Optimising window size of semantic of classification model for identification of in-text citations based on context and intent.
PLoS One. 2025 Mar 24;20(3):e0309862. doi: 10.1371/journal.pone.0309862. eCollection 2025.
5
Citation Sentiment Analysis in Clinical Trial Papers.
AMIA Annu Symp Proc. 2015 Nov 5;2015:1334-41. eCollection 2015.
6
Deep Sentiment Analysis of Twitter Data Using a Hybrid Ghost Convolution Neural Network Model.
Comput Intell Neurosci. 2022 Jul 18;2022:6595799. doi: 10.1155/2022/6595799. eCollection 2022.
7
Transformer-based ensemble model for dialectal Arabic sentiment classification.
PeerJ Comput Sci. 2025 Mar 24;11:e2644. doi: 10.7717/peerj-cs.2644. eCollection 2025.
8
Scientific text citation analysis using CNN features and ensemble learning model.
PLoS One. 2024 May 28;19(5):e0302304. doi: 10.1371/journal.pone.0302304. eCollection 2024.
10
A clinical text classification paradigm using weak supervision and deep representation.
BMC Med Inform Decis Mak. 2019 Jan 7;19(1):1. doi: 10.1186/s12911-018-0723-6.

引用本文的文献

1
DistilRoBiLSTMFuse: an efficient hybrid deep learning approach for sentiment analysis.
PeerJ Comput Sci. 2024 Sep 26;10:e2349. doi: 10.7717/peerj-cs.2349. eCollection 2024.
2
Assessing citation integrity in biomedical publications: corpus annotation and NLP models.
Bioinformatics. 2024 Jul 1;40(7). doi: 10.1093/bioinformatics/btae420.
4
Validating GAN-BioBERT: A Methodology for Assessing Reporting Trends in Clinical Trials.
Front Digit Health. 2022 May 24;4:878369. doi: 10.3389/fdgth.2022.878369. eCollection 2022.
5
Machine Learning in Modeling of Mouse Behavior.
Front Neurosci. 2021 Sep 14;15:700253. doi: 10.3389/fnins.2021.700253. eCollection 2021.

本文引用的文献

1
Extracting Drug-Drug Interactions with Word and Character-Level Recurrent Neural Networks.
Proc (IEEE Int Conf Healthc Inform). 2017 Aug;2017:5-12. doi: 10.1109/ICHI.2017.15. Epub 2017 Sep 14.
2
Relative Citation Ratio (RCR): A New Metric That Uses Citation Rates to Measure Influence at the Article Level.
PLoS Biol. 2016 Sep 6;14(9):e1002541. doi: 10.1371/journal.pbio.1002541. eCollection 2016 Sep.
3
Citation Sentiment Analysis in Clinical Trial Papers.
AMIA Annu Symp Proc. 2015 Nov 5;2015:1334-41. eCollection 2015.
6
Exploratory undersampling for class-imbalance learning.
IEEE Trans Syst Man Cybern B Cybern. 2009 Apr;39(2):539-50. doi: 10.1109/TSMCB.2008.2007853. Epub 2008 Dec 16.
7
An index to quantify an individual's scientific research output.
Proc Natl Acad Sci U S A. 2005 Nov 15;102(46):16569-72. doi: 10.1073/pnas.0507655102. Epub 2005 Nov 7.
8
Long short-term memory.
Neural Comput. 1997 Nov 15;9(8):1735-80. doi: 10.1162/neco.1997.9.8.1735.
9
Why the impact factor of journals should not be used for evaluating research.
BMJ. 1997 Feb 15;314(7079):498-502. doi: 10.1136/bmj.314.7079.497.
10
Citation analysis as a tool in journal evaluation.
Science. 1972 Nov 3;178(4060):471-9. doi: 10.1126/science.178.4060.471.

文献AI研究员

20分钟写一篇综述,助力文献阅读效率提升50倍。

立即体验

用中文搜PubMed

大模型驱动的PubMed中文搜索引擎

马上搜索

文档翻译

学术文献翻译模型,支持多种主流文档格式。

立即体验