School of Mathematics, Statistics, and Computer Science, University of KwaZulu-Natal, King Edward Avenue, Pietermaritzburg Campus, Pietermaritzburg, 3201, KwaZulu-Natal, South Africa.
Department of Computer Science, University of Pretoria, Pretoria, 0028, South Africa.
Comput Biol Med. 2022 Oct;149:105943. doi: 10.1016/j.compbiomed.2022.105943. Epub 2022 Aug 12.
The task of classification and localization with detecting abnormalities in medical images is considered very challenging. Computer-aided systems have been widely employed to address this issue, and the proliferation of deep learning network architectures is proof of the outstanding performance reported in the literature. However, localizing abnormalities in regions of images that can support the confidence of classification continues to attract research interest. The difficulty of using digital histopathology images for this task is another drawback, which needs high-level deep learning models to address the situation. Successful pathology localization automation will support automatic acquisition planning and post-imaging analysis. In this paper, we address issues related to the combination of classification with image localization and detection through a dual branch deep learning framework that uses two different configurations of convolutional neural networks (CNN) architectures. Whole-image based CNN (WCNN) and region-based CNN (RCNN) architectures are systematically combined to classify and localize abnormalities in samples. A multi-class classification and localization of abnormalities are achieved using the method with no annotation-dependent images. In addition, seamless confidence and explanation mechanism is provided so that outcomes from WCNN and RCNN are mapped together for further analysis. Using images from both BACH and BreakHis databases, an exhaustive set of experiments was carried out to validate the performance of the proposed method in achieving classification and localization simultaneously. Obtained results showed that the system achieved a classification accuracy of 97.08%, a localization accuracy of 94%, and an area under the curve (AUC) of 0.10 for classification. Further findings from this study revealed that a multi-neural network approach could provide a suitable method for addressing the combinatorial problem of classification and localization anomalies in digital medical images. Lastly, the study's outcome offers means for automating the annotation of histopathology images and the support for human pathologists in locating abnormalities.
在医学图像中进行分类和定位异常的任务被认为极具挑战性。计算机辅助系统已广泛用于解决这一问题，深度学习网络架构的普及证明了文献中报告的出色性能。然而，在能够支持分类置信度的图像区域中定位异常仍然吸引着研究兴趣。使用数字组织病理学图像进行此任务的难度是另一个缺点，需要高级深度学习模型来解决这种情况。成功的病理学定位自动化将支持自动获取计划和成像后分析。在本文中，我们通过使用两种不同配置的卷积神经网络 (CNN) 架构的双分支深度学习框架来解决与分类与图像定位和检测相结合的问题。基于全图像的 CNN (WCNN) 和基于区域的 CNN (RCNN) 架构被系统地结合起来，对样本中的异常进行分类和定位。该方法无需依赖标注的图像即可实现多类分类和异常定位。此外，还提供了无缝的置信度和解释机制，以便将 WCNN 和 RCNN 的结果映射在一起进行进一步分析。使用来自 BACH 和 BreakHis 数据库的图像，进行了详尽的实验，以验证所提出的方法在同时实现分类和定位方面的性能。获得的结果表明，该系统在分类方面的准确率为 97.08%，在定位方面的准确率为 94%，在分类方面的曲线下面积 (AUC) 为 0.10。进一步的研究结果表明，多神经网络方法可以为解决数字医学图像中分类和定位异常的组合问题提供合适的方法。最后，该研究的结果为组织病理学图像的自动化标注提供了手段，并为病理学家定位异常提供了支持。
