Division of Research, Kaiser Permanente Northern California, Oakland, CA, USA.
Department of Gastroenterology, Kaiser Permanente Northern California, Santa Clara, CA, USA.
J Med Syst. 2020 Jul 31;44(9):151. doi: 10.1007/s10916-020-01604-8.
Key variables recorded as text in colonoscopy and pathology reports have been extracted using natural language processing (NLP) tools that were not easily adaptable to new settings. We aimed to develop a reliable NLP tool with broad adaptability. During 1996-2016, Kaiser Permanente Northern California performed 401,566 colonoscopies with linked pathology. We randomly sampled 1000 linked reports into a Training Set and developed an NLP tool using SAS® PERL regular expressions. The NLP tool captured five colonoscopy and pathology variables: type, size, and location of polyps; extent of procedure; and quality of bowel preparation. We used a Validation Set (N = 3000) to confirm the variables' classifications using manual chart review as the reference. Performance of the NLP tool was assessed using the sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and Cohen's κ. Cohen's κ ranged from 93 to 99%. The sensitivity and specificity ranged from 95 to 100% across all categories. For categories with prevalence exceeding 10%, the PPV ranged from 97% to 100% except for adequate quality of preparation (prevalence 92%), for which the PPV was 65%. For categories with prevalence below 10%, the PPVs ranged from 62% to 100%. NPVs ranged from 94% to 100% except for the "complete" extent of procedure, for which the NPV was 73%. Using information from a large community-based population, we developed a transparent and adaptable NLP tool for extracting five colonoscopy and pathology variables. The tool can be readily tested in other healthcare settings.
结肠镜检查和病理报告中以文本形式记录的关键变量已使用自然语言处理 (NLP) 工具提取，这些工具不易适应新环境。我们旨在开发一种具有广泛适应性的可靠 NLP 工具。1996 年至 2016 年间，凯撒永久医疗集团北加州分部进行了 401566 例结肠镜检查，并与病理结果相关联。我们随机抽取 1000 份相关报告作为训练集，并使用 SAS®PERL 正则表达式开发了一种 NLP 工具。该 NLP 工具捕获了五个结肠镜检查和病理变量：息肉的类型、大小和位置；手术范围；以及肠道准备的质量。我们使用 3000 份验证集（N=3000）通过手动图表审查作为参考，确认变量的分类。使用灵敏度、特异性、阳性预测值 (PPV)、阴性预测值 (NPV) 和 Cohen's κ 评估 NLP 工具的性能。Cohen's κ 值范围为 93 至 99%。所有类别中，灵敏度和特异性范围为 95%至 100%。对于流行率超过 10%的类别，PPV 范围为 97%至 100%，除了肠道准备充分（流行率 92%）的情况，PPV 为 65%。对于流行率低于 10%的类别，PPV 范围为 62%至 100%。NPV 范围为 94%至 100%，除了“完全”手术范围，NPV 为 73%。使用来自大型基于社区的人群的信息，我们开发了一种透明且适应性强的 NLP 工具，用于提取五个结肠镜检查和病理变量。该工具可以在其他医疗保健环境中轻松进行测试。
