Basic Medical College, Chengdu University of TCM, Chengdu, China.
School of Rehabilitation Medicine, Weifang Medical University, Weifang, China.
J Healthc Eng. 2022 Mar 29;2022:9372807. doi: 10.1155/2022/9372807. eCollection 2022.
The aim of the study is to build a tongue image intelligent analysis "end-to-end" deep learning network based on a tongue diagnosis image of traditional Chinese medicine. The tongue target region in the original image was segmented by the UNet tongue segmentation model at the front end of the network. After segmentation, the feature vector of the tongue target region was extracted by the ResNet network, and then the blood pressure on the day of shooting was fused with the feature vector extracted by the ResNet network through the convolution operation method to complete the extraction of two groups of data of tongue feature and fusion feature. Based on analyzing the data of blood pressure, tongue image, and their fusion at the end of the network, four regression analysis methods were used to predict the stage mean value. After training, the model is tested with the test set data, and the test results are evaluated with mean absolute error (MAE). The prediction error of the model based on the fusion data of tongue image and blood pressure on the day of shooting was lower than that of the other two data modes. The UNet tongue segmentation model combined with the ResNet network can realize the automatic extraction of tongue image features. The extracted features combined with machine learning modeling can be used to explore the complex hierarchical mathematical association between tongue image and clinical data. The experimental results show that the multimodal data fusion method is an important way to mine the clinical value of the TCM tongue image.
本研究旨在构建基于中医舌诊图像的舌象智能分析“端到端”深度学习网络。网络前端采用 UNet 舌分割模型对原始图像中的舌目标区域进行分割,分割后采用 ResNet 网络提取舌目标区域的特征向量,然后通过卷积操作方法将当天拍摄的血压与 ResNet 网络提取的特征向量融合,完成舌特征和融合特征两组数据的提取。基于网络末端对血压、舌象及其融合数据的分析,采用四种回归分析方法对阶段均值进行预测。训练后,使用测试集数据对模型进行测试,并用平均绝对误差(MAE)评估测试结果。基于舌象和当天拍摄的血压融合数据的模型预测误差低于其他两种数据模式。UNet 舌分割模型与 ResNet 网络相结合,实现了舌象特征的自动提取。与机器学习建模相结合提取的特征可用于探索舌象与临床数据之间复杂的层次数学关联。实验结果表明,多模态数据融合方法是挖掘中医舌象临床价值的重要途径。