Center for Mathematical Modeling - CNRS UMI2807, Faculty of Physical and Mathematical Sciences, University of Chile, Santiago, Chile.
Center for Medical Informatics and Telemedicine, ICBM, Faculty of Medicine, University of Chile, Santiago, Chile.
BMC Med Inform Decis Mak. 2021 Jul 1;21(1):208. doi: 10.1186/s12911-021-01565-z.
In Chile, a patient needing a specialty consultation or surgery has to first be referred by a general practitioner, then placed on a waiting list. The Explicit Health Guarantees (GES in Spanish) ensures, by law, the maximum time to solve 85 health problems. Usually, a health professional manually verifies if each referral, written in natural language, corresponds or not to a GES-covered disease. An error in this classification is catastrophic for patients, as it puts them on a non-prioritized waiting list, characterized by prolonged waiting times.
To support the manual process, we developed and deployed a system that automatically classifies referrals as GES-covered or not using historical data. Our system is based on word embeddings specially trained for clinical text produced in Chile. We used a vector representation of the reason for referral and patient's age as features for training machine learning models using human-labeled historical data. We constructed a ground truth dataset combining classifications made by three healthcare experts, which was used to validate our results.
The best performing model over ground truth reached an AUC score of 0.94, with a weighted F1-score of 0.85 (0.87 in precision and 0.86 in recall). During seven months of continuous and voluntary use, the system has amended 87 patient misclassifications.
This system is a result of a collaboration between technical and clinical experts, and the design of the classifier was custom-tailored for a hospital's clinical workflow, which encouraged the voluntary use of the platform. Our solution can be easily expanded across other hospitals since the registry is uniform in Chile.
在智利，需要专科咨询或手术的患者必须首先由全科医生转诊，然后再列入等候名单。明确的健康保障（西班牙语中的 GES）通过法律确保解决 85 个健康问题的最长时间。通常，卫生专业人员会手动检查以自然语言书写的每个转诊是否对应 GES 涵盖的疾病。这种分类错误对患者来说是灾难性的，因为它会使他们列入非优先等候名单，其特点是等待时间长。
为了支持手动流程，我们开发并部署了一个系统，该系统使用历史数据自动对转诊进行 GES 涵盖或不涵盖的分类。我们的系统基于专门为智利临床文本训练的词嵌入。我们使用转诊原因和患者年龄的向量表示作为特征，使用人工标记的历史数据训练机器学习模型。我们构建了一个结合了三位医疗保健专家的分类的真实数据集，用于验证我们的结果。
在真实数据上表现最好的模型达到了 0.94 的 AUC 评分，加权 F1 得分为 0.85（精度为 0.87，召回率为 0.86）。在连续七个月的持续自愿使用中，该系统纠正了 87 例患者的错误分类。
该系统是技术和临床专家合作的结果，分类器的设计针对医院的临床工作流程进行了定制，这鼓励了平台的自愿使用。由于智利的注册表是统一的，因此我们的解决方案可以轻松扩展到其他医院。
