Davidson Rory, Hardman Will, Amit Guy, Bilu Yonatan, Della Mea Vincenzo, Galaida Aleksandr, Girshovitz Irena, Kulyabin Mikhail, Horia Popescu Mihai, Roitero Kevin, Sokolov Gleb, Yanover Chen
SNOMED International, London W2 6BD, United Kingdom.
Veratai Ltf, Woking GU22 7QW, United Kingdom.
J Am Med Inform Assoc. 2025 Sep 1;32(9):1397-1406. doi: 10.1093/jamia/ocaf104.
This paper presents the results from a competition challenging participants to develop entity linking models using a subset of annotated MIMIC-IV-Note data and the SNOMED CT Terminology.
As a basis for this work, a large set of 74 808 annotations was curated across 272 discharge notes spanning 6624 unique clinical concepts. Submissions were evaluated using the mean Intersection-over-Union metric, evaluated at the character level with the 3 best performing solutions awarded a cash prize.
The winning solutions employed contrasting approaches: a dictionary-based method, an encoder-based method, and a decoder-based method.
Our analysis reveals that concept frequency in training data significantly impacts model performance, with rare concepts proving particularly challenging. High concept entropy and annotation ambiguity were also associated with decreased performance.
Findings from this work suggest that future projects should focus on improving entity linking for rare concepts and developing methods to better leverage contextual information when training examples are scarce.
本文展示了一场竞赛的结果，该竞赛要求参与者使用带注释的MIMIC-IV-Note数据子集和SNOMED CT术语来开发实体链接模型。
作为这项工作的基础，我们精心整理了一大组74808条注释，涵盖272份出院记录中的6624个独特临床概念。使用平均交并比指标对提交的方案进行评估，在字符级别进行评估，表现最佳的3个解决方案将获得现金奖励。
获胜方案采用了不同的方法：基于字典的方法、基于编码器的方法和基于解码器的方法。
我们的分析表明，训练数据中的概念频率会显著影响模型性能，罕见概念尤其具有挑战性。高概念熵和注释歧义也与性能下降有关。
这项工作的结果表明，未来的项目应专注于改善罕见概念的实体链接，并在训练示例稀缺时开发更好地利用上下文信息的方法。
