College of Electronics and Information Engineering, Sichuan University, Chengdu 610041, China.
College of Computer Science, Sichuan University, Chengdu 610041, China.
Sensors (Basel). 2023 Jun 21;23(13):5795. doi: 10.3390/s23135795.
Prosthetic joint infection (PJI) is a prevalent and severe complication characterized by high diagnostic challenges. Currently, a unified diagnostic standard incorporating both computed tomography (CT) images and numerical text data for PJI remains unestablished, owing to the substantial noise in CT images and the disparity in data volume between CT images and text data. This study introduces a diagnostic method, HGT, based on deep learning and multimodal techniques. It effectively merges features from CT scan images and patients' numerical text data via a Unidirectional Selective Attention (USA) mechanism and a graph convolutional network (GCN)-based Feature Fusion network. We evaluated the proposed method on a custom-built multimodal PJI dataset, assessing its performance through ablation experiments and interpretability evaluations. Our method achieved an accuracy (ACC) of 91.4% and an area under the curve (AUC) of 95.9%, outperforming recent multimodal approaches by 2.9% in ACC and 2.2% in AUC, with a parameter count of only 68 M. Notably, the interpretability results highlighted our model's strong focus and localization capabilities at lesion sites. This proposed method could provide clinicians with additional diagnostic tools to enhance accuracy and efficiency in clinical practice.
人工关节感染（PJI）是一种普遍且严重的并发症，其诊断极具挑战性。目前，由于 CT 图像存在大量噪声，且 CT 图像与文本数据在数据量上存在差异，尚未建立一种将 CT 图像与数值文本数据整合的 PJI 统一诊断标准。本研究提出了一种基于深度学习和多模态技术的诊断方法 HGT。它通过单向选择性注意（USA）机制和基于图卷积网络（GCN）的特征融合网络，有效地融合了 CT 扫描图像和患者数值文本数据的特征。我们在一个自建的多模态 PJI 数据集上评估了所提出的方法，通过消融实验和可解释性评估来评估其性能。我们的方法在准确性（ACC）方面达到了 91.4%，在 AUC 方面达到了 95.9%，在 ACC 方面比最近的多模态方法高出 2.9%，在 AUC 方面高出 2.2%，而参数数量仅为 68M。值得注意的是，可解释性结果突出了我们模型在病变部位的强大关注和定位能力。该方法可为临床医生提供额外的诊断工具，以提高临床实践的准确性和效率。
