School of Medical Education, Guy's, King's and St Thomas' School of Medicine, London, UK
Section of Women's Mental Health, Department of Health Services and Population Research, King's College London, London, UK.
BMJ Open. 2022 Feb 16;12(2):e052911. doi: 10.1136/bmjopen-2021-052911.
This paper evaluates the application of a natural language processing (NLP) model for extracting clinical text referring to interpersonal violence using electronic health records (EHRs) from a large mental healthcare provider.
A multidisciplinary team iteratively developed guidelines for annotating clinical text referring to violence. Keywords were used to generate a dataset which was annotated (ie, classified as affirmed, negated or irrelevant) for: presence of violence, patient status (ie, as perpetrator, witness and/or victim of violence) and violence type (domestic, physical and/or sexual). An NLP approach using a pretrained transformer model, BioBERT (Bidirectional Encoder Representations from Transformers for Biomedical Text Mining) was fine-tuned on the annotated dataset and evaluated using 10-fold cross-validation.
We used the Clinical Records Interactive Search (CRIS) database, comprising over 500 000 de-identified EHRs of patients within the South London and Maudsley NHS Foundation Trust, a specialist mental healthcare provider serving an urban catchment area.
Searches of CRIS were carried out based on 17 predefined keywords. Randomly selected text fragments were taken from the results for each keyword, amounting to 3771 text fragments from the records of 2832 patients.
We estimated precision, recall and F1 score for each NLP model. We examined sociodemographic and clinical variables in patients giving rise to the text data, and frequencies for each annotated violence characteristic.
Binary classification models were developed for six labels (violence presence, perpetrator, victim, domestic, physical and sexual). Among annotations affirmed for the presence of any violence, 78% (1724) referred to physical violence, 61% (1350) referred to patients as perpetrator and 33% (731) to domestic violence. NLP models' precision ranged from 89% (perpetrator) to 98% (sexual); recall ranged from 89% (victim, perpetrator) to 97% (sexual).
State of the art NLP models can extract and classify clinical text on violence from EHRs at acceptable levels of scale, efficiency and accuracy.
本文评估了自然语言处理（NLP）模型在从大型精神保健服务提供商的电子健康记录（EHR）中提取临床文本中涉及人际暴力信息的应用。
一个多学科团队迭代制定了用于注释涉及暴力的临床文本的指南。使用关键词生成了一个数据集，该数据集经过注释（即，分类为肯定、否定或无关），用于：暴力的存在、患者状态（即，作为暴力的实施者、目击者和/或受害者）和暴力类型（家庭、身体和/或性）。使用经过预训练的转换器模型 BioBERT（用于生物医学文本挖掘的双向编码器表示转换器）的 NLP 方法在注释数据集上进行了微调，并使用 10 折交叉验证进行了评估。
我们使用了包含超过 500 000 名来自伦敦南部和莫兹利国民保健信托基金会患者的去识别 EHR 的临床记录交互搜索（CRIS）数据库，这是一家专门的精神保健服务提供商，服务于一个城市集水区。
根据 17 个预定义的关键词进行了 CRIS 搜索。从每个关键词的结果中随机选择文本片段，从 2832 名患者的记录中获得了 3771 个文本片段。
为六个标签（暴力存在、实施者、受害者、家庭、身体和性）开发了二进制分类模型。在肯定存在任何暴力的注释中，78%（1724）涉及身体暴力，61%（1350）涉及患者作为实施者，33%（731）涉及家庭暴力。NLP 模型的精度范围从 89%（实施者）到 98%（性）；召回率范围从 89%（受害者、实施者）到 97%（性）。
最先进的 NLP 模型可以从 EHR 中以可接受的规模、效率和准确性提取和分类关于暴力的临床文本。
