Department of Electrical Engineering and Computer Science, Lassonde School of Engineering, York University, Toronto, Ontario, Canada.
Department of Radiation Oncology, Sunnybrook Health Sciences Center, Toronto, Ontario, Canada.
Med Phys. 2023 Dec;50(12):7852-7864. doi: 10.1002/mp.16574. Epub 2023 Jul 5.
Pathological complete response (pCR) to neoadjuvant chemotherapy (NAC) has demonstrated a strong correlation to improved survival in breast cancer (BC) patients. However, pCR rates to NAC are less than 30%, depending on the BC subtype. Early prediction of NAC response would facilitate therapeutic modifications for individual patients, potentially improving overall treatment outcomes and patient survival.
This study, for the first time, proposes a hierarchical self-attention-guided deep learning framework to predict NAC response in breast cancer patients using digital histopathological images of pre-treatment biopsy specimens.
Digitized hematoxylin and eosin-stained slides of BC core needle biopsies were obtained from 207 patients treated with NAC, followed by surgery. The response to NAC for each patient was determined using the standard clinical and pathological criteria after surgery. The digital pathology images were processed through the proposed hierarchical framework consisting of patch-level and tumor-level processing modules followed by a patient-level response prediction component. A combination of convolutional layers and transformer self-attention blocks were utilized in the patch-level processing architecture to generate optimized feature maps. The feature maps were analyzed through two vision transformer architectures adapted for the tumor-level processing and the patient-level response prediction components. The feature map sequences for these transformer architectures were defined based on the patch positions within the tumor beds and the bed positions within the biopsy slide, respectively. A five-fold cross-validation at the patient level was applied on the training set (144 patients with 9430 annotated tumor beds and 1,559,784 patches) to train the models and optimize the hyperparameters. An unseen independent test set (63 patients with 3574 annotated tumor beds and 173,637 patches) was used to evaluate the framework.
The obtained results on the test set showed an AUC of 0.89 and an F1-score of 90% for predicting pCR to NAC a priori by the proposed hierarchical framework. Similar frameworks with the patch-level, patch-level + tumor-level, and patch-level + patient-level processing components resulted in AUCs of 0.79, 0.81, and 0.84 and F1-scores of 86%, 87%, and 89%, respectively.
The results demonstrate a high potential of the proposed hierarchical deep-learning methodology for analyzing digital pathology images of pre-treatment tumor biopsies to predict the pathological response of breast cancer to NAC.
新辅助化疗（NAC）的病理完全缓解（pCR）已证明与乳腺癌（BC）患者的生存改善密切相关。然而，根据 BC 亚型的不同，pCR 率不到 30%。对 NAC 反应的早期预测将有助于为个体患者进行治疗修改，从而有可能改善整体治疗效果和患者生存。
本研究首次提出了一种分层自注意力引导的深度学习框架，用于使用新辅助化疗前活检标本的数字组织病理学图像预测乳腺癌患者的 NAC 反应。
从 207 名接受 NAC 治疗后接受手术的患者中获取 BC 芯针活检的数字化苏木精和伊红染色幻灯片。每个患者的 NAC 反应使用手术后的标准临床和病理标准确定。通过由斑块级和肿瘤级处理模块组成的分层框架对数字病理学图像进行处理，然后是患者级反应预测组件。在斑块级处理架构中使用卷积层和转换器自注意力块的组合来生成优化的特征图。通过分别适用于肿瘤级处理和患者级反应预测组件的两个视觉转换器架构分析特征图。这些转换器架构的特征图序列是基于肿瘤床内的斑块位置和活检切片内的床位置定义的。在训练集（144 名患者，9430 个标记的肿瘤床和 1559784 个斑块）上应用五重交叉验证来训练模型并优化超参数。一个独立的测试集（63 名患者，3574 个标记的肿瘤床和 173637 个斑块）用于评估框架。
在测试集上获得的结果表明，通过所提出的分层框架预测 NAC 治疗前 pCR 的 AUC 为 0.89，F1 得分为 90%。具有斑块级、斑块级+肿瘤级和斑块级+患者级处理组件的类似框架分别导致 AUC 为 0.79、0.81 和 0.84，F1 得分为 86%、87%和 89%。
结果表明，所提出的分层深度学习方法具有很高的潜力，可以分析新辅助化疗前肿瘤活检的数字病理学图像，预测乳腺癌对 NAC 的病理反应。
