Laboratory of Biomedical Informatics and Artificial Intelligence, National Medical Research Center for Neurosurgery named after N.N. Burdenko, Moscow, Russian Federation.
Stud Health Technol Inform. 2022 Jun 29;295:555-558. doi: 10.3233/SHTI220788.
In this study, we update the evaluation of the Russian GPT3 model presented in our previous paper in predicting the length of stay (LOS) in neurosurgery. We aimed to assess the performance the Russian GPT-3 (ruGPT-3) language model in LOS prediction using narrative medical records in neurosurgery compared to doctors' and patients' expectations. Doctors appeared to have the most realistic LOS expectations (MAE = 2.54), while the model's predictions (MAE = 3.53) were closest to the patients' (MAE = 3.47) but inferior to them (p = 0.011). A detailed analysis showed a solid quality of ruGPT-3 performance based on narrative clinical texts. Considering our previous findings obtained with recurrent neural networks and FastText vector representation, we estimate the new result as important but probably improveable.
在这项研究中，我们更新了我们之前的论文中对俄罗斯 GPT3 模型的评估，该模型旨在预测神经外科的住院时间（LOS）。我们旨在评估俄罗斯 GPT-3（ruGPT-3）语言模型在神经外科中使用叙事医疗记录预测 LOS 的表现，与医生和患者的预期进行比较。医生的 LOS 预期似乎最接近实际情况（MAE=2.54），而模型的预测（MAE=3.53）则最接近患者的预期（MAE=3.47），但不如患者（p=0.011）。详细分析表明，ruGPT-3 在基于叙事临床文本方面具有出色的性能。考虑到我们之前使用递归神经网络和 FastText 向量表示获得的结果，我们估计这一新结果虽然重要，但可能还有改进空间。
