Hripcsak G, Friedman C, Alderson P O, DuMouchel W, Johnson S B, Clayton P D
Department of Medical Informatics, Columbia-Presbyterian Medical Center, New York, NY 10032, USA.
Ann Intern Med. 1995 May 1;122(9):681-8. doi: 10.7326/0003-4819-122-9-199505010-00007.
To evaluate the automated detection of clinical conditions described in narrative reports.
Automated methods and human experts detected the presence or absence of six clinical conditions in 200 admission chest radiograph reports.
A computerized, general-purpose natural language processor; 6 internists; 6 radiologists; 6 lay persons; and 3 other computer methods.
Intersubject disagreement was quantified by "distance" (the average number of clinical conditions per report on which two subjects disagreed) and by sensitivity and specificity with respect to the physicians.
Using a majority vote, physicians detected 101 conditions in the 200 reports (0.51 per report); the most common condition was acute bacterial pneumonia (prevalence, 0.14), and the least common was chronic obstructive pulmonary disease (prevalence, 0.03). Pairs of physicians disagreed on the presence of at least 1 condition for an average of 20% of reports. The average intersubject distance among physicians was 0.24 (95% Cl, 0.19 to 0.29) out of a maximum possible distance of 6. No physician had a significantly greater distance than the average. The average distance of the natural language processor from the physicians was 0.26 (Cl, 0.21 to 0.32; not significantly greater than the average among physicians). Lay persons and alternative computer methods had significantly greater distance from the physicians (all > 0.5). The natural language processor had a sensitivity of 81% (Cl, 73% to 87%) and a specificity of 98% (Cl, 97% to 99%); physicians had an average sensitivity of 85% and an average specificity of 98%.
Physicians disagreed on the interpretation of narrative reports, but this was not caused by outlier physicians or a consistent difference in the way internists and radiologists read reports. The natural language processor was not distinguishable from the physicians and was superior to all other comparison subjects. Although the domain of this study was restricted (six clinical conditions in chest radiographs), natural language processing seems to have the potential to extract clinical information from narrative reports in a manner that will support automated decision-support and clinical research.
评估对叙述性报告中所描述临床状况的自动检测。
自动方法和人类专家对200份入院胸部X光片报告中的六种临床状况的存在与否进行检测。
一个计算机化的通用自然语言处理器；6名内科医生；6名放射科医生；6名非专业人员；以及其他3种计算机方法。
通过“距离”（每份报告中两名受试者意见不一致的临床状况的平均数量）以及相对于医生的敏感性和特异性来量化受试者间的分歧。
采用多数表决法，医生们在200份报告中检测到101种状况（每份报告0.51种）；最常见的状况是急性细菌性肺炎（患病率0.14），最不常见的是慢性阻塞性肺疾病（患病率0.03）。平均20%的报告中，医生对至少一种状况的存在与否存在分歧。医生之间的平均受试者间距离为0.24（95%可信区间，0.19至0.29），最大可能距离为6。没有医生的距离显著大于平均距离。自然语言处理器与医生的平均距离为0.26（可信区间，0.21至0.32；不显著大于医生之间的平均距离）。非专业人员和其他计算机方法与医生的距离显著更大（均>0.5）。自然语言处理器的敏感性为81%（可信区间，73%至87%），特异性为98%（可信区间，97%至99%）；医生的平均敏感性为85%，平均特异性为98%。
医生对叙述性报告的解读存在分歧，但这并非由异常值医生或内科医生与放射科医生阅读报告方式的持续差异所致。自然语言处理器与医生无明显差异，且优于所有其他比较对象。尽管本研究的领域有限（胸部X光片中的六种临床状况），但自然语言处理似乎有潜力以支持自动决策辅助和临床研究的方式从叙述性报告中提取临床信息。
