Department of Population Health Sciences, Geisinger, Danville, PA, United States.
Department of Computational Biomedicine, Cedars-Sinai Medical Center, West Hollywood, CA, United States.
JMIR Ment Health. 2024 Jan 15;11:e53366. doi: 10.2196/53366.
Information regarding opioid use disorder (OUD) status and severity is important for patient care. Clinical notes provide valuable information for detecting and characterizing problematic opioid use, necessitating development of natural language processing (NLP) tools, which in turn requires reliably labeled OUD-relevant text and understanding of documentation patterns.
To inform automated NLP methods, we aimed to develop and evaluate an annotation schema for characterizing OUD and its severity, and to document patterns of OUD-relevant information within clinical notes of heterogeneous patient cohorts.
We developed an annotation schema to characterize OUD severity based on criteria from the Diagnostic and Statistical Manual of Mental Disorders, 5th edition. In total, 2 annotators reviewed clinical notes from key encounters of 100 adult patients with varied evidence of OUD, including patients with and those without chronic pain, with and without medication treatment for OUD, and a control group. We completed annotations at the sentence level. We calculated severity scores based on annotation of note text with 18 classes aligned with criteria for OUD severity and determined positive predictive values for OUD severity.
The annotation schema contained 27 classes. We annotated 1436 sentences from 82 patients; notes of 18 patients (11 of whom were controls) contained no relevant information. Interannotator agreement was above 70% for 11 of 15 batches of reviewed notes. Severity scores for control group patients were all 0. Among noncontrol patients, the mean severity score was 5.1 (SD 3.2), indicating moderate OUD, and the positive predictive value for detecting moderate or severe OUD was 0.71. Progress notes and notes from emergency department and outpatient settings contained the most and greatest diversity of information. Substance misuse and psychiatric classes were most prevalent and highly correlated across note types with high co-occurrence across patients.
Implementation of the annotation schema demonstrated strong potential for inferring OUD severity based on key information in a small set of clinical notes and highlighting where such information is documented. These advancements will facilitate NLP tool development to improve OUD prevention, diagnosis, and treatment.
了解阿片类药物使用障碍（OUD）的状态和严重程度对于患者护理非常重要。临床记录为发现和描述阿片类药物使用问题提供了有价值的信息，这就需要开发自然语言处理（NLP）工具，而这反过来又需要可靠标记的 OUD 相关文本和对文档模式的理解。
为了为自动化 NLP 方法提供信息，我们旨在开发和评估一种用于描述 OUD 及其严重程度的标注方案，并记录异质患者队列的临床记录中与 OUD 相关的信息模式。
我们基于《精神障碍诊断与统计手册》第 5 版的标准开发了一种用于描述 OUD 严重程度的标注方案。共有 2 名注释员审查了 100 名成年患者关键就诊记录中的临床记录，这些患者有或没有 OUD 的证据，包括有或没有慢性疼痛、有或没有 OUD 药物治疗，以及对照组。我们在句子级别上完成了注释。我们根据与 OUD 严重程度标准相匹配的 18 个类别的注释文本计算严重程度得分，并确定 OUD 严重程度的阳性预测值。
标注方案包含 27 个类别。我们对 82 名患者的 1436 个句子进行了注释；18 名患者（其中 11 名为对照组）的记录中没有相关信息。在审查的 15 批记录中，有 11 批的注释员间一致性超过 70%。对照组患者的严重程度评分为 0。在非对照组患者中，平均严重程度评分为 5.1（SD 3.2），表明中重度 OUD，检测中重度或重度 OUD 的阳性预测值为 0.71。进展记录以及急诊科和门诊记录中包含的信息最多，且差异最大。物质使用障碍和精神科类别在所有记录类型中最常见，且在患者之间高度相关，共同出现的频率很高。
标注方案的实施表明，根据少量临床记录中的关键信息推断 OUD 严重程度具有很大潜力，并突出了记录这些信息的位置。这些进展将促进 NLP 工具的开发，以改善 OUD 的预防、诊断和治疗。
