Gao Yue, Chen Yuepeng, Wang Minghao, Wu Jinge, Kim Yunsoo, Zhou Kaiyin, Li Miao, Liu Xien, Fu Xiangling, Wu Ji, Wu Honghan
School of Computer Science (National Pilot Software Engineering School), Beijing University of Posts and Telecommunications, Beijing, China.
Key Laboratory of Trustworthy Distributed Computing and Service (BUPT), Ministry of Education, Beijing, China.
NPJ Digit Med. 2024 Dec 19;7(1):368. doi: 10.1038/s41746-024-01363-7.
Automated clinical coding (ACC) has emerged as a promising alternative to manual coding. This study proposes a novel human-in-the-loop (HITL) framework, CliniCoCo. Using deep learning capacities, CliniCoCo focuses on how such ACC systems and human coders can work effectively and efficiently together in real-world settings. Specifically, it implements a series of collaborative strategies at annotation, training and user interaction stages. Extensive experiments are conducted using real-world EMR datasets from Chinese hospitals. With automatically optimised annotation workloads, the model can achieve F1 scores around 0.80-0.84. For an EMR with 30% mistaken codes, CliniCoCo can suggest halving the annotations from 3000 admissions with an ignorable 0.01 F1 decrease. In human evaluations, compared to manual coding, CliniCoCo reduces coding time by 40% on average and significantly improves the correction rates on EMR mistakes (e.g., three times better on missing codes). Senior professional coders' performances can be boosted to more than 0.93 F1 score from 0.72.
自动临床编码（ACC）已成为一种有前途的手动编码替代方案。本研究提出了一种新颖的人在回路（HITL）框架CliniCoCo。利用深度学习能力，CliniCoCo专注于此类ACC系统与人类编码员如何在现实环境中有效且高效地协同工作。具体而言，它在注释、训练和用户交互阶段实施了一系列协作策略。使用来自中国医院的真实电子病历数据集进行了广泛实验。通过自动优化注释工作量，该模型可实现约0.80 - 0.84的F1分数。对于有30%错误代码的电子病历，CliniCoCo可以建议将注释从3000份入院病历减半，同时F1分数仅下降可忽略不计的0.01。在人工评估中，与手动编码相比，CliniCoCo平均将编码时间减少了40%，并显著提高了电子病历错误的校正率（例如，在缺失代码方面提高了三倍）。资深专业编码员的表现可以从0.72提升至超过0.93的F1分数。
