Silva João Figueira, Almeida João Rafael, Matos Sérgio
Department of Electronics, Telecommunications and Informatics, Institute of Electronics and Informatics Engineering of Aveiro, University of Aveiro, Aveiro, Portugal.
Department of Information and Communications Technologies, University of A Coruña, A Coruña, Spain.
JMIR Med Inform. 2020 Dec 29;8(12):e22898. doi: 10.2196/22898.
Electronic health records store large amounts of patient clinical data. Despite efforts to structure patient data, clinical notes containing rich patient information remain stored as free text, greatly limiting its exploitation. This includes family history, which is highly relevant for applications such as diagnosis and prognosis.
This study aims to develop automatic strategies for annotating family history information in clinical notes, focusing not only on the extraction of relevant entities such as family members and disease mentions but also on the extraction of relations between the identified entities.
This study extends a previous contribution for the 2019 track on family history extraction from national natural language processing clinical challenges by improving a previously developed rule-based engine, using deep learning (DL) approaches for the extraction of entities from clinical notes, and combining both approaches in a hybrid end-to-end system capable of successfully extracting family member and observation entities and the relations between those entities. Furthermore, this study analyzes the impact of factors such as the use of external resources and different types of embeddings in the performance of DL models.
The approaches developed were evaluated in a first task regarding entity extraction and in a second task concerning relation extraction. The proposed DL approach improved observation extraction, obtaining F scores of 0.8688 and 0.7907 in the training and test sets, respectively. However, DL approaches have limitations in the extraction of family members. The rule-based engine was adjusted to have higher generalizing capability and achieved family member extraction F scores of 0.8823 and 0.8092 in the training and test sets, respectively. The resulting hybrid system obtained F scores of 0.8743 and 0.7979 in the training and test sets, respectively. For the second task, the original evaluator was adjusted to perform a more exact evaluation than the original one, and the hybrid system obtained F scores of 0.6480 and 0.5082 in the training and test sets, respectively.
We evaluated the impact of several factors on the performance of DL models, and we present an end-to-end system for extracting family history information from clinical notes, which can help in the structuring and reuse of this type of information. The final hybrid solution is provided in a publicly available code repository.
电子健康记录存储了大量患者临床数据。尽管人们努力对患者数据进行结构化处理，但包含丰富患者信息的临床记录仍以自由文本形式存储，这极大地限制了其利用。这其中包括家族史，而家族史对于诊断和预后等应用非常重要。
本研究旨在开发用于标注临床记录中家族史信息的自动策略，不仅关注家庭成员和疾病提及等相关实体的提取，还关注已识别实体之间关系的提取。
本研究扩展了之前在2019年全国自然语言处理临床挑战中关于家族史提取赛道的贡献，改进了先前开发的基于规则的引擎，使用深度学习（DL）方法从临床记录中提取实体，并将这两种方法结合在一个能够成功提取家庭成员和观察实体以及这些实体之间关系的混合端到端系统中。此外，本研究分析了外部资源的使用和不同类型嵌入等因素对DL模型性能的影响。
所开发的方法在关于实体提取的第一个任务和关于关系提取的第二个任务中进行了评估。所提出的DL方法改进了观察提取，在训练集和测试集中分别获得了0.8688和0.7907的F分数。然而，DL方法在家庭成员提取方面存在局限性。基于规则的引擎经过调整以具有更高的泛化能力，在训练集和测试集中分别实现了0.8823和0.8092的家庭成员提取F分数。最终的混合系统在训练集和测试集中分别获得了0.8743和0.7979的F分数。对于第二个任务，对原始评估器进行了调整，以进行比原始评估更精确的评估，混合系统在训练集和测试集中分别获得了0.6480和0.5082的F分数。
我们评估了几个因素对DL模型性能的影响，并提出了一个从临床记录中提取家族史信息的端到端系统，这有助于此类信息的结构化和重用。最终的混合解决方案在一个公开可用的代码库中提供。
