Division of Rheumatology, Immunology, and Allergy, Brigham and Women's Hospital, Boston, MA, 6016BB, 60 Fenwood Road, Boston, 02115, USA.
Harvard Medical School, Boston, MA, USA.
BMC Med Inform Decis Mak. 2019 Nov 15;19(1):226. doi: 10.1186/s12911-019-0970-1.
Electronic medical records (EMR) contain numerical data important for clinical outcomes research, such as vital signs and cardiac ejection fractions (EF), which tend to be embedded in narrative clinical notes. In current practice, this data is often manually extracted for use in research studies. However, due to the large volume of notes in datasets, manually extracting numerical data often becomes infeasible. The objective of this study is to develop and validate a natural language processing (NLP) tool that can efficiently extract numerical clinical data from narrative notes.
To validate the accuracy of the tool EXTraction of EMR Numerical Data (EXTEND), we developed a reference standard by manually extracting vital signs from 285 notes, EF values from 300 notes, glycated hemoglobin (HbA1C), and serum creatinine from 890 notes. For each parameter of interest, we calculated the sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV) and F score of EXTEND using two metrics. (1) completion of data extraction, and (2) accuracy of data extraction compared to the actual values in the note verified by chart review. At the note level, extraction by EXTEND was considered correct only if it accurately detected and extracted all values of interest in a note. Using manually-annotated labels as the gold standard, the note-level accuracy of EXTEND in capturing the numerical vital sign values, EF, HbA1C and creatinine ranged from 0.88 to 0.95 for sensitivity, 0.95 to 1.0 for specificity, 0.95 to 1.0 for PPV, 0.89 to 0.99 for NPV, and 0.92 to 0.96 in F scores. Compared to the actual value level, the sensitivity, PPV, and F score of EXTEND ranged from 0.91 to 0.95, 0.95 to 1.0 and 0.95 to 0.96.
EXTEND is an efficient, flexible tool that uses knowledge-based rules to extract clinical numerical parameters with high accuracy. By increasing dictionary terms and developing new rules, the usage of EXTEND can easily be expanded to extract additional numerical data important in clinical outcomes research.
电子病历（EMR）包含重要的临床结果研究数值数据，例如生命体征和心脏射血分数（EF），这些数据通常嵌入在临床记录的叙述中。在当前实践中，这些数据通常需要手动提取以供研究使用。然而，由于数据集中文本量很大，手动提取数值数据通常变得不可行。本研究的目的是开发和验证一种能够从叙述性文本中高效提取临床数值数据的自然语言处理（NLP）工具。
为了验证工具 EXTraction of EMR Numerical Data（EXTEND）的准确性，我们通过手动从 285 份记录中提取生命体征，从 300 份记录中提取 EF 值，从 890 份记录中提取糖化血红蛋白（HbA1C）和血清肌酐，开发了一个参考标准。对于每个感兴趣的参数，我们使用两种指标计算了 EXTEND 的灵敏度、特异性、阳性预测值（PPV）、阴性预测值（NPV）和 F 分数。（1）数据提取的完成情况，（2）与通过图表审查验证的记录中实际值相比的数据提取准确性。在记录级别，如果 EXTEND 准确地检测并提取了记录中所有感兴趣的值，则认为提取是正确的。使用手动标注的标签作为金标准，EXTEND 在捕捉数值生命体征值、EF、HbA1C 和肌酐的记录级别的准确性为 0.88 到 0.95 的灵敏度，0.95 到 1.0 的特异性，0.95 到 1.0 的 PPV，0.89 到 0.99 的 NPV 和 0.92 到 0.96 的 F 分数。与实际值级别相比，EXTEND 的灵敏度、PPV 和 F 分数范围为 0.91 到 0.95、0.95 到 1.0 和 0.95 到 0.96。
EXTEND 是一种高效、灵活的工具，它使用基于知识的规则以高精度提取临床数值参数。通过增加字典术语和开发新规则，EXTEND 的使用可以很容易地扩展到提取临床结果研究中重要的其他数值数据。
