Center for Biostatistics, Informatics, and Data Science, MedStar Health Research Institute, Washington, District of Columbia, USA
Department of Statistics, North Carolina State University, Raleigh, North Carolina, USA.
BMJ Health Care Inform. 2023 May;30(1). doi: 10.1136/bmjhci-2022-100731.
The objective of this study was to explore the use of natural language processing (NLP) algorithm to categorise contributing factors from patient safety event (PSE). Contributing factors are elements in the healthcare process (eg, communication failures) that instigate an event or allow an event to occur. Contributing factors can be used to further investigate why safety events occurred.
We used 10 years of self-reported PSE reports from a multihospital healthcare system in the USA. Reports were first selected by event date. We calculated χ values for each ngram in the bag-of-words then selected N ngrams with the highest χ values. Then, PSE reports were filtered to only include the sentences containing the selected ngrams. Such sentences were called information-rich sentences. We compared two feature extraction techniques from free-text data: (1) baseline bag-of-words features and (2) features from information-rich sentences. Three machine learning algorithms were used to categorise five contributing factors representing sociotechnical errors: communication/hand-off failure, technology issue, policy/procedure issue, distractions/interruptions and lapse/slip. We trained 15 binary classifiers (five contributing factors * three machine learning models). The models' performances were evaluated according to the area under the precision-recall curve (AUPRC), precision, recall, and F1-score.
Applying the information-rich sentence selection algorithm boosted the contributing factor categorisation performance. Comparing the AUPRCs, the proposed NLP approach improved the categorisation performance of two and achieved comparable results with baseline in categorising three contributing factors.
Information-rich sentence selection can be incorporated to extract the sentences in free-text event narratives in which the contributing factor information is embedded.
本研究旨在探索使用自然语言处理（NLP）算法对患者安全事件（PSE）的促成因素进行分类。促成因素是医疗过程中的要素（例如，沟通失败），它们引发或允许事件发生。促成因素可用于进一步调查安全事件发生的原因。
我们使用了美国一家多医院医疗保健系统 10 年来的自我报告 PSE 报告。报告首先按事件日期进行选择。我们计算了词汇袋中每个 ngram 的 χ 值，然后选择 χ 值最高的 N ngrams。然后，PSE 报告被过滤只包含包含所选 ngrams 的句子。这样的句子称为信息丰富的句子。我们比较了两种从自由文本数据中提取特征的技术：（1）基线词汇袋特征和（2）信息丰富句子的特征。我们使用三种机器学习算法来对代表社会技术错误的五个促成因素进行分类：沟通/交接失败、技术问题、政策/程序问题、干扰/中断和失误/疏忽。我们训练了 15 个二进制分类器（五个促成因素*三种机器学习模型）。根据精度-召回曲线下面积（AUPRC）、精度、召回率和 F1 分数来评估模型的性能。
应用信息丰富的句子选择算法提高了促成因素分类性能。通过比较 AUPRC，所提出的 NLP 方法提高了对两个促成因素的分类性能，并在分类三个促成因素方面取得了与基线相当的结果。
可以将信息丰富的句子选择纳入从自由文本事件叙述中提取嵌入促成因素信息的句子。
