IEEE J Biomed Health Inform. 2021 Jun;25(6):2041-2049. doi: 10.1109/JBHI.2020.3036734. Epub 2021 Jun 3.
Modeling variable-sized regions of interest (ROIs) in whole slide images using deep convolutional networks is a challenging task, as these networks typically require fixed-sized inputs that should contain sufficient structural and contextual information for classification. We propose a deep feature extraction framework that builds an ROI-level feature representation via weighted aggregation of the representations of variable numbers of fixed-sized patches sampled from nuclei-dense regions in breast histopathology images.
First, the initial patch-level feature representations are extracted from both fully-connected layer activations and pixel-level convolutional layer activations of a deep network, and the weights are obtained from the class predictions of the same network trained on patch samples. Then, the final patch-level feature representations are computed by concatenation of weighted instances of the extracted feature activations. Finally, the ROI-level representation is obtained by fusion of the patch-level representations by average pooling.
Experiments using a well-characterized data set of 240 slides containing 437 ROIs marked by experienced pathologists with variable sizes and shapes result in an accuracy score of 72.65% in classifying ROIs into four diagnostic categories that cover the whole histologic spectrum.
The results show that the proposed feature representations are superior to existing approaches and provide accuracies that are higher than the average accuracy of another set of pathologists.
The proposed generic representation that can be extracted from any type of deep convolutional architecture combines the patch appearance information captured by the network activations and the diagnostic relevance predicted by the class-specific scoring of patches for effective modeling of variable-sized ROIs.
使用深度卷积网络对全切片图像中的可变大小感兴趣区域（ROI）进行建模是一项具有挑战性的任务，因为这些网络通常需要固定大小的输入，这些输入应该包含足够的结构和上下文信息以进行分类。我们提出了一种深度特征提取框架，通过对从乳腺组织病理学图像中核密集区域采样的可变数量的固定大小的斑块的表示进行加权聚合，构建 ROI 级别的特征表示。
首先，从深度网络的全连接层激活和像素级卷积层激活中提取初始斑块级特征表示，并从在斑块样本上训练的相同网络的类别预测中获得权重。然后，通过对提取的特征激活的加权实例进行串联计算最终的斑块级特征表示。最后，通过平均池化融合斑块级表示来获得 ROI 级表示。
使用由经验丰富的病理学家标记的具有可变大小和形状的 240 张幻灯片组成的特征良好的数据集进行的实验，将 ROI 分为四个诊断类别，涵盖整个组织学谱，分类准确率为 72.65%。
结果表明，所提出的特征表示优于现有方法，并提供了高于另一组病理学家平均准确率的准确率。
可以从任何类型的深度卷积架构中提取的这种通用表示形式结合了网络激活捕获的斑块外观信息和斑块特定评分预测的诊断相关性，可有效建模可变大小的 ROI。
