School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, People's Republic of China.
Department of Radiology, The Third Affiliated Hospital of Sun Yat-sen University, Guangzhou, People's Republic of China.
Phys Med Biol. 2024 Jun 5;69(12). doi: 10.1088/1361-6560/ad4f45.
In oncology, clinical decision-making relies on a multitude of data modalities, including histopathological, radiological, and clinical factors. Despite the emergence of computer-aided multimodal decision-making systems for predicting hepatocellular carcinoma (HCC) recurrence post-hepatectomy, existing models often employ simplistic feature-level concatenation, leading to redundancy and suboptimal performance. Moreover, these models frequently lack effective integration with clinically relevant data and encounter challenges in integrating diverse scales and dimensions, as well as incorporating the liver background, which holds clinical significance but has been previously overlooked.To address these limitations, we propose two approaches. Firstly, we introduce the tensor fusion method to our model, which offers distinct advantages in handling multi-scale and multi-dimensional data fusion, potentially enhancing overall performance. Secondly, we pioneer the consideration of the liver background's impact, integrating it into the feature extraction process using a deep learning segmentation-based algorithm. This innovative inclusion aligns the model more closely with real-world clinical scenarios, as the liver background may contain crucial information related to postoperative recurrence.We collected radiomics (MRI) and histopathological images from 176 cases diagnosed by experienced clinicians across two independent centers. Our proposed network underwent training and 5-fold cross-validation on this dataset before validation on an external test dataset comprising 40 cases. Ultimately, our model demonstrated outstanding performance in predicting early recurrence of HCC postoperatively, achieving an AUC of 0.883.These findings signify significant progress in addressing challenges related to multimodal data fusion and hold promise for more accurate clinical outcome predictions. In this study, we exploited global 3D liver background into modelling which is crucial to to the prognosis assessment and analyzed the whole liver background in addition to the tumor region. Both MRI images and histopathological images of HCC were fused at high-dimensional feature space using tensor techniques to solve cross-scale data integration issue.
在肿瘤学中，临床决策依赖于多种数据模式，包括组织病理学、影像学和临床因素。尽管已经出现了用于预测肝癌（HCC）切除术后复发的计算机辅助多模态决策系统，但现有的模型通常采用简单的特征级联，导致冗余和性能不佳。此外，这些模型通常缺乏与临床相关数据的有效整合，并且在整合不同尺度和维度以及包含具有临床意义但以前被忽视的肝脏背景方面存在挑战。
为了解决这些限制，我们提出了两种方法。首先，我们在模型中引入了张量融合方法，该方法在处理多尺度和多维数据融合方面具有明显的优势，有可能提高整体性能。其次，我们首创地考虑了肝脏背景的影响，使用基于深度学习的分割算法将其纳入特征提取过程。这种创新性的纳入使模型更紧密地与现实世界的临床场景保持一致，因为肝脏背景可能包含与术后复发相关的关键信息。
我们从两个独立中心的 176 名经验丰富的临床医生那里收集了放射组学（MRI）和组织病理学图像。我们提出的网络在这个数据集上进行了训练和 5 折交叉验证，然后在包含 40 个病例的外部测试数据集上进行了验证。最终，我们的模型在预测 HCC 术后早期复发方面表现出色，AUC 达到 0.883。
这些发现标志着在解决多模态数据融合相关挑战方面取得了重大进展，并为更准确的临床结果预测提供了希望。在这项研究中，我们利用全局 3D 肝脏背景进行建模，这对预后评估至关重要，并分析了整个肝脏背景，除了肿瘤区域。使用张量技术在高维特征空间中融合 HCC 的 MRI 图像和组织病理学图像，以解决跨尺度数据集成问题。
