Clinical Research and Innovation Unit, Xinhua Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai, China.
Shanghai Artificial Intelligence Laboratory, Shanghai, China.
J Med Internet Res. 2024 Oct 30;26:e51711. doi: 10.2196/51711.
Although new technologies have increased the efficiency and convenience of medical care, patients still struggle to identify specialized outpatient departments in Chinese tertiary hospitals due to a lack of medical knowledge.
The objective of our study was to develop a precise and subdividable outpatient triage system to improve the experiences and convenience of patient care.
We collected 395,790 electronic medical records (EMRs) and 500 medical dialogue groups. The EMRs were divided into 3 data sets to design and train the triage model (n=387,876, 98%) and test (n=3957, 1%) and validate (n=3957, 1%) it. The triage system was altered based on the current BERT (Bidirectional Encoder Representations from Transformers) framework and evaluated by recommendation accuracies in Xinhua Hospital using the cancellation rates in 2021 and 2022, from October 29 to December 5. Finally, a prospective observational study containing 306 samples was conducted to compare the system's performance with that of triage nurses, which was evaluated by calculating precision, accuracy, recall of the top 3 recommended departments (recall@3), and time consumption.
With 3957 (1%) records each, the testing and validation data sets achieved an accuracy of 0.8945 and 0.8941, respectively. Implemented in Xinhua Hospital, our triage system could accurately recommend 79 subspecialty departments and reduce the number of registration cancellations from 16,037 (3.83%) of the total 418,714 to 15,338 (3.53%) of the total 434200 (P<.05). In comparison to the triage system, the performance of the triage nurses was more accurate (0.9803 vs 0.9153) and precise (0.9213 vs 0.9049) since the system could identify subspecialty departments, whereas triage nurses or even general physicians can only recommend main departments. In addition, our triage system significantly outperformed triage nurses in recall@3 (0.6230 vs 0.5266; P<.001) and time consumption (10.11 vs 14.33 seconds; P<.001).
The triage system demonstrates high accuracy in outpatient triage of all departments and excels in subspecialty department recommendations, which could decrease the cancellation rate and time consumption. It also improves the efficiency and convenience of clinical care to fulfill better the usage of medical resources, expand hospital effectiveness, and improve patient satisfaction in Chinese tertiary hospitals.
尽管新技术提高了医疗保健的效率和便利性，但由于缺乏医学知识，患者仍然难以在中文三级医院识别专科门诊。
我们的研究旨在开发一个精确和可细分的门诊分诊系统，以改善患者护理的体验和便利性。
我们收集了 395790 份电子病历（EMR）和 500 个医疗对话组。将 EMR 分为 3 个数据集来设计和训练分诊模型（n=387876，98%）并进行测试（n=3957，1%）和验证（n=3957，1%）。分诊系统基于当前的 BERT（Bidirectional Encoder Representations from Transformers）框架进行修改，并使用 2021 年 10 月 29 日至 12 月 5 日的新华医院的取消率来评估其在推荐准确率方面的表现。最后，进行了一项前瞻性观察研究，包含 306 个样本，比较了系统与分诊护士的性能，通过计算前 3 个推荐科室的精度、准确率、召回率（召回率@3）和时间消耗来评估。
使用每个数据集 3957 个记录，测试和验证数据集的准确率分别为 0.8945 和 0.8941。在新华医院实施，我们的分诊系统可以准确推荐 79 个专科，并将总 418714 个挂号取消中的 16037 个（3.83%）减少到总 434200 个中的 15338 个（3.53%）（P<.05）。与分诊系统相比，分诊护士的性能更准确（0.9803 比 0.9153）和精确（0.9213 比 0.9049），因为系统可以识别专科，而分诊护士甚至普通医生只能推荐主要科室。此外，我们的分诊系统在召回率@3（0.6230 比 0.5266；P<.001）和时间消耗（10.11 秒比 14.33 秒；P<.001）方面明显优于分诊护士。
分诊系统在所有科室的门诊分诊中表现出高度准确性，并擅长推荐专科，这可以降低取消率和时间消耗。它还提高了临床护理的效率和便利性，更好地利用了医疗资源，扩大了医院的效率，并提高了中文三级医院患者的满意度。
