School of Biomedical Engineering, Southern Medical University, Guangzhou, China; Guangdong Provincial Key Laboratory of Medical Image Processing, Southern Medical University, Guangzhou, China; Guangdong Province Engineering Laboratory for Medical Imaging and Diagnostic Technology, Southern Medical University, Guangzhou, China.
Department of Radiology, The Third Affiliated Hospital of Southern Medical University (Academy of Orthopedics, Guangdong Province), Guangzhou, Guangdong, China.
Med Image Anal. 2022 May;78:102415. doi: 10.1016/j.media.2022.102415. Epub 2022 Mar 3.
The morphological evaluation of tumor-infiltrating lymphocytes (TILs) in hematoxylin and eosin (H& E)-stained histopathological images is the key to breast cancer (BCa) diagnosis, prognosis, and therapeutic response prediction. For now, the qualitative assessment of TILs is carried out by pathologists, and computer-aided automatic lymphocyte measurement is still a great challenge because of the small size and complex distribution of lymphocytes. In this paper, we propose a novel dense dual-task network (DDTNet) to simultaneously achieve automatic TIL detection and segmentation in histopathological images. DDTNet consists of a backbone network (i.e., feature pyramid network) for extracting multi-scale morphological characteristics of TILs, a detection module for the localization of TIL centers, and a segmentation module for the delineation of TIL boundaries, where a boundary-aware branch is further used to provide a shape prior to segmentation. An effective feature fusion strategy is utilized to introduce multi-scale features with lymphocyte location information from highly correlated branches for precise segmentation. Experiments on three independent lymphocyte datasets of BCa demonstrate that DDTNet outperforms other advanced methods in detection and segmentation metrics. As part of this work, we also propose a semi-automatic method (TILAnno) to generate high-quality boundary annotations for TILs in H& E-stained histopathological images. TILAnno is used to produce a new lymphocyte dataset that contains 5029 annotated lymphocyte boundaries, which have been released to facilitate computational histopathology in the future.
肿瘤浸润淋巴细胞（TILs）在苏木精和伊红（H&E）染色组织病理学图像中的形态评估是乳腺癌（BCa）诊断、预后和治疗反应预测的关键。目前，TILs 的定性评估由病理学家进行，而由于淋巴细胞体积小且分布复杂，计算机辅助自动淋巴细胞测量仍然是一个巨大的挑战。在本文中，我们提出了一种新颖的密集双任务网络（DDTNet），以同时实现组织病理学图像中自动 TIL 检测和分割。DDTNet 由一个骨干网络（即特征金字塔网络）组成，用于提取 TILs 的多尺度形态特征，一个检测模块用于定位 TIL 中心，一个分割模块用于勾勒 TIL 边界，其中进一步使用边界感知分支为分割提供形状先验。采用有效的特征融合策略，从高度相关的分支中引入具有淋巴细胞位置信息的多尺度特征，以实现精确分割。在三个独立的 BCa 淋巴细胞数据集上的实验表明，DDTNet 在检测和分割指标上优于其他先进方法。作为这项工作的一部分，我们还提出了一种半自动方法（TILAnno），用于生成 H&E 染色组织病理学图像中 TIL 的高质量边界注释。TILAnno 用于生成一个新的淋巴细胞数据集，其中包含 5029 个注释的淋巴细胞边界，这些边界已被释放，以方便未来的计算组织病理学研究。
