Yoon Soo Bin, Lee Jipyeong, Lee Hyung-Chul, Jung Chul-Woo, Lee Hyeonhoon
Department of Anesthesiology and Pain Medicine, Seoul National University College of Medicine, Seoul National University Hospital, Seoul, Republic of Korea.
Department of Anesthesiology and Pain Medicine, Seoul National University Hospital, Seoul, Republic of Korea.
NPJ Digit Med. 2024 Sep 28;7(1):259. doi: 10.1038/s41746-024-01259-6.
The American Society of Anesthesiologist's Physical Status (ASA-PS) classification system assesses comorbidities before sedation and analgesia, but inconsistencies among raters have hindered its objective use. This study aimed to develop natural language processing (NLP) models to classify ASA-PS using pre-anesthesia evaluation summaries, comparing their performance to human physicians. Data from 717,389 surgical cases in a tertiary hospital (October 2004-May 2023) was split into training, tuning, and test datasets. Board-certified anesthesiologists created reference labels for tuning and test datasets. The NLP models, including ClinicalBigBird, BioClinicalBERT, and Generative Pretrained Transformer 4, were validated against anesthesiologists. The ClinicalBigBird model achieved an area under the receiver operating characteristic curve of 0.915. It outperformed board-certified anesthesiologists with a specificity of 0.901 vs. 0.897, precision of 0.732 vs. 0.715, and F1-score of 0.716 vs. 0.713 (all p <0.01). This approach will facilitate automatic and objective ASA-PS classification, thereby streamlining the clinical workflow.
美国麻醉医师协会身体状况（ASA-PS）分类系统在镇静和镇痛前评估合并症，但评估者之间的不一致阻碍了其客观使用。本研究旨在开发自然语言处理（NLP）模型，使用麻醉前评估摘要对ASA-PS进行分类，并将其性能与人类医生进行比较。一家三级医院（2004年10月至2023年5月）717389例手术病例的数据被分为训练、调整和测试数据集。获得委员会认证的麻醉医师为调整和测试数据集创建了参考标签。包括ClinicalBigBird、BioClinicalBERT和生成式预训练变换器4在内的NLP模型与麻醉医师进行了验证。ClinicalBigBird模型的受试者操作特征曲线下面积为0.915。其表现优于获得委员会认证的麻醉医师，特异性分别为0.901和0.897，精确度分别为0.732和0.715，F1分数分别为0.716和0.713（所有p<0.01）。这种方法将有助于自动、客观地进行ASA-PS分类，从而简化临床工作流程。
