Neveol Aurélie, Zeng Kelly, Bodenreider Olivier
U.S. National Library of Medicine, Bethesda, Maryland, USA.
AMIA Annu Symp Proc. 2006;2006:589-93.
This paper explores alternative approaches for the evaluation of an automatic indexing tool for MEDLINE, complementing the traditional precision and recall method.
The performance of MTI, the Medical Text Indexer used at NLM to produce MeSH recommendations for biomedical journal articles is evaluated on a random set of MEDLINE citations. The evaluation examines semantic similarity at the term level (indexing terms). In addition, the documents retrieved by queries resulting from MTI index terms for a given document are compared to the PubMed related citations for this document.
Semantic similarity scores between sets of index terms are higher than the corresponding Dice similarity scores. Overall, 75% of the original documents and 58% of the top ten related citations are retrieved by queries based on the automatic indexing.
The alternative measures studied in this paper confirm previous findings and may be used to select particular documents from the test set for a more thorough analysis.
本文探索用于评估MEDLINE自动索引工具的替代方法，以补充传统的精确率和召回率方法。
在一组随机抽取的MEDLINE引文上评估MTI（美国国立医学图书馆用于为生物医学期刊文章生成医学主题词（MeSH）推荐的医学文本索引工具）的性能。该评估在术语层面（索引词）检验语义相似性。此外，将由给定文档的MTI索引词产生的查询所检索到的文档与该文档在PubMed中的相关引文进行比较。
索引词集之间的语义相似性得分高于相应的迪西相似性得分。总体而言，基于自动索引的查询检索到了75%的原始文档和58%的前十篇相关引文。
本文研究的替代度量证实了先前的发现，可用于从测试集中选择特定文档进行更深入的分析。
