Lai Mason, Fenton Cynthia, Rubin Jessica, Huang Chiung-Yu, Pletcher Mark, Lai Jennifer C, Cullaro Giuseppe, Ge Jin
Department of Medicine, University of California, San Francisco.
Division of Hospital Medicine, Department of Medicine, University of California, San Francisco.
medRxiv. 2024 Nov 13:2024.11.13.24317220. doi: 10.1101/2024.11.13.24317220.
Hepatorenal syndrome - Acute Kidney Injury (HRS-AKI) is a severe complication of decompensated cirrhosis that is challenging to predict. Sentiment analysis, a computational process of identifying and categorizing opinions and judgment expressed in text, may enhance traditional prediction methodologies based on structured variables. Large language models (LLMs), such as generative pretrained transformers (GPTs), have demonstrated abilities to perform sentiment analyses on non-clinical texts. We sought to determine if GPT-performed sentiment analysis could improve upon predictions using clinical covariates alone in the prediction of HRS-AKI.
Adult patients admitted to a single academic medical center with decompensated cirrhosis and AKI. We used a protected health information (PHI) compliant version of Microsoft Azure OpenAI GPT-4o to derive a sentiment score ranging from 0 to 1 for HRS-AKI, and conduct natural language processing (NLP) extraction of clinical terms associated with HRS-AKI in clinical notes. The area under the receiver operator curve (AUROC) was compared in logistic regression models incorporating structured variables (socio-demographics, MELD 3.0, hemodynamic parameters) with compared to without sentiment scores and NLP-extracted clinical terms.
In our cohort of 314 participants, higher sentiment score was associated with the diagnosis of HRS-AKI (OR 1.33 per 0.1, 95% CI 1.02-1.79) in multivariate models. AUROC of the baseline model using structured clinical covariates alone was 0.639. With the addition of the GPT-4o derived sentiment score and clinical terms to structured covariates, the final model yielded an improved AUROC of 0.758 (p=0.03).
Clinical texts contain large amounts of data that are currently difficult to extract using standard methodologies. Sentiment analysis and NLP-based variable derivation with GPT-4o in clinical application is feasible and can improve the prediction of HRS-AKI over traditional modeling methodologies alone.
肝肾综合征 - 急性肾损伤（HRS - AKI）是失代偿期肝硬化的一种严重并发症，难以预测。情感分析是一种识别和分类文本中表达的观点和判断的计算过程，可能会增强基于结构化变量的传统预测方法。大型语言模型（LLMs），如生成式预训练变换器（GPTs），已证明能够对非临床文本进行情感分析。我们试图确定GPT执行的情感分析是否能在仅使用临床协变量预测HRS - AKI时改进预测。
纳入一家学术医疗中心收治的失代偿期肝硬化和急性肾损伤的成年患者。我们使用符合受保护健康信息（PHI）的微软Azure OpenAI GPT - 4o版本得出HRS - AKI的情感评分，范围为0至1，并在临床记录中对与HRS - AKI相关的临床术语进行自然语言处理（NLP）提取。在纳入结构化变量（社会人口统计学、MELD 3.0、血流动力学参数）的逻辑回归模型中，比较有和没有情感评分及NLP提取的临床术语时的受试者操作特征曲线下面积（AUROC）。
在我们的314名参与者队列中，多变量模型中较高的情感评分与HRS - AKI的诊断相关（每0.1增加1.33，95%置信区间1.02 - 1.79）。仅使用结构化临床协变量的基线模型的AUROC为0.639。在结构化协变量中加入GPT - 4o得出的情感评分和临床术语后，最终模型的AUROC提高到0.758（p = 0.03）。
临床文本包含大量目前难以用标准方法提取的数据。在临床应用中使用GPT - 4o进行情感分析和基于NLP的变量推导是可行的，并且与单独的传统建模方法相比，可以改善HRS - AKI的预测。
