School of Life Sciences, Beijing University of Chinese Medicine, Beijing, 100029, China.
School of Traditional Chinese Medicine, Beijing University of Chinese Medicine, Beijing, 100029, China.
J Ethnopharmacol. 2022 Mar 1;285:114905. doi: 10.1016/j.jep.2021.114905. Epub 2021 Dec 8.
Tongue coating has been used as an effective signature of health in traditional Chinese medicine (TCM). The level of greasy coating closely relates to the strength of dampness or pathogenic qi in TCM theory. Previous empirical studies and our systematic review have shown the relation between greasy coating and various diseases, including gastroenteropathy, coronary heart disease, and coronavirus disease 2019 (COVID-19). However, the objective and intelligent greasy coating and related diseases recognition methods are still lacking. The construction of the artificial intelligent tongue recognition models may provide important syndrome diagnosis and efficacy evaluation methods, and contribute to the understanding of ethnopharmacological mechanisms based on TCM theory.
The present study aimed to develop an artificial intelligent model for greasy tongue coating recognition and explore its application in COVID-19.
Herein, we developed greasy tongue coating recognition networks (GreasyCoatNet) using convolutional neural network technique and a relatively large (N = 1486) set of tongue images from standard devices. Tests were performed using both cross-validation procedures and a new dataset (N = 50) captured by common cameras. Besides, the accuracy and time efficiency comparisons between the GreasyCoatNet and doctors were also conducted. Finally, the model was transferred to recognize the greasy coating level of COVID-19.
The overall accuracy in 3-level greasy coating classification with cross-validation was 88.8% and accuracy on new dataset was 82.0%, indicating that GreasyCoatNet can obtain robust greasy coating estimates from diverse datasets. In addition, we conducted user study to confirm that our GreasyCoatNet outperforms TCM practitioners, yet only consuming roughly 1% of doctors' examination time. Critically, we demonstrated that GreasyCoatNet, along with transfer learning, can construct more proper classifier of COVID-19, compared to directly training classifier on patient versus control datasets. We, therefore, derived a disease-specific deep learning network by finetuning the generic GreasyCoatNet.
Our framework may provide an important research paradigm for differentiating tongue characteristics, diagnosing TCM syndrome, tracking disease progression, and evaluating intervention efficacy, exhibiting its unique potential in clinical applications.
舌诊在中医（TCM）中被用作健康的有效特征。舌苔的程度与中医理论中的湿或病气的强弱密切相关。以前的实证研究和我们的系统评价已经表明了油腻舌苔与各种疾病之间的关系，包括胃肠病、冠心病和 2019 年冠状病毒病（COVID-19）。然而，客观和智能的油腻舌苔和相关疾病识别方法仍然缺乏。人工智能舌识别模型的构建可能为中医理论提供重要的证候诊断和疗效评估方法，并有助于理解民族药理学机制。
本研究旨在开发一种用于油腻舌苔识别的人工智能模型，并探索其在 COVID-19 中的应用。
我们使用卷积神经网络技术和一个相对较大的（N=1486）标准设备舌图像集开发了油腻舌苔识别网络（GreasyCoatNet）。测试分别通过交叉验证程序和一个新的（N=50）由普通相机拍摄的数据集进行。此外，还对 GreasyCoatNet 与医生之间的准确性和时间效率进行了比较。最后，将模型转移到识别 COVID-19 的油腻舌苔水平。
在交叉验证的 3 级油腻舌苔分类中，整体准确率为 88.8%，新数据集的准确率为 82.0%，这表明 GreasyCoatNet 可以从不同的数据集获得稳健的油腻舌苔估计。此外，我们进行了用户研究，以确认我们的 GreasyCoatNet 优于中医从业者，而仅消耗医生检查时间的约 1%。至关重要的是，我们证明了 GreasyCoatNet 可以通过使用迁移学习构建更合适的 COVID-19 分类器，而不是直接在患者与对照数据集上进行分类器训练。因此，我们通过微调通用 GreasyCoatNet 来构建疾病特异性深度学习网络。
我们的框架可能为区分舌特征、诊断 TCM 综合征、跟踪疾病进展和评估干预效果提供一个重要的研究范例，在临床应用中展现出其独特的潜力。
