Hu Dingyi, Jiang Zhiguo, Shi Jun, Xie Fengying, Wu Kun, Tang Kunming, Cao Ming, Huai Jianguo, Zheng Yushan
Image Processing Center, School of Astronautics, Beihang University, Beijing, 100191, China.
Image Processing Center, School of Astronautics, Beihang University, Beijing, 100191, China; Tianmushan Laboratory, Hangzhou, 311115, Zhejiang Province, China.
Comput Methods Programs Biomed. 2025 May;263:108677. doi: 10.1016/j.cmpb.2025.108677. Epub 2025 Feb 27.
With the development of deep learning techniques, the computer-assisted pathology diagnosis plays a crucial role in clinical diagnosis. An important task within this field is report generation, which provides doctors with text descriptions of whole slide images (WSIs). Report generation from WSIs presents significant challenges due to the structural complexity and pathological diversity of tissues, as well as the large size and high information density of WSIs. The objective of this study is to design a histopathology report generation method that can efficiently generate reports from WSIs and is suitable for clinical practice.
In this paper, we propose a novel approach for generating pathology reports from WSIs, leveraging knowledge retrieval and multi-level regional feature selection. To deal with the uneven distribution of pathological information in WSIs, we introduce a multi-level regional feature encoding network and a feature selection module that extracts multi-level region representations and filters out region features irrelevant to the diagnosis, enabling more efficient report generation. Moreover, we design a knowledge retrieval module to improve the report generation performance that can leverage the diagnostic information from historical cases. Additionally, we propose an out-of-domain application mode based on large language model (LLM). The use of LLM enhances the scalability of the generation model and improves its adaptability to data from different sources.
The proposed method is evaluated on a public datasets and one in-house dataset. On the public GastricADC (991 WSIs), our method outperforms state-of-the-art text generation methods and achieved 0.568 and 0.345 on metric Rouge-L and Bleu-4, respectively. On the in-house Gastric-3300 (3309 WSIs), our method achieved significantly better performance with Rouge-L of 0.690, which surpassed the second-best state-of-the-art method Wcap 6.3%.
We present an advanced method for pathology report generation from WSIs, addressing the key challenges associated with the large size and complex pathological structures of these images. In particular, the multi-level regional feature selection module effectively captures diagnostically significant regions of varying sizes. The knowledge retrieval-based decoder leverages historical diagnostic data to enhance report accuracy. Our method not only improves the informativeness and relevance of the generated pathology reports but also outperforms the state-of-the-art techniques.
随着深度学习技术的发展，计算机辅助病理诊断在临床诊断中发挥着关键作用。该领域的一项重要任务是报告生成，即向医生提供全切片图像（WSI）的文本描述。由于组织的结构复杂性和病理多样性，以及WSI的大尺寸和高信息密度，从WSI生成报告面临重大挑战。本研究的目的是设计一种组织病理学报告生成方法，该方法能够有效地从WSI生成报告并适用于临床实践。
在本文中，我们提出了一种从WSI生成病理报告的新方法，利用知识检索和多级区域特征选择。为了处理WSI中病理信息的不均匀分布，我们引入了一个多级区域特征编码网络和一个特征选择模块，该模块提取多级区域表示并过滤掉与诊断无关的区域特征，从而实现更高效的报告生成。此外，我们设计了一个知识检索模块来提高报告生成性能，该模块可以利用历史病例的诊断信息。此外，我们提出了一种基于大语言模型（LLM）的域外应用模式。LLM的使用增强了生成模型的可扩展性，并提高了其对来自不同来源数据的适应性。
所提出的方法在一个公共数据集和一个内部数据集上进行了评估。在公共的GastricADC（991张WSI）数据集上，我们的方法优于当前最先进的文本生成方法，在Rouge-L和Bleu-4指标上分别达到了0.568和0.345。在内部的Gastric-3300（3309张WSI）数据集上，我们的方法取得了显著更好的性能，Rouge-L为0.690，超过了第二好的当前最先进方法Wcap 6.3%。
我们提出了一种从WSI生成病理报告的先进方法，解决了与这些图像的大尺寸和复杂病理结构相关的关键挑战。特别是，多级区域特征选择模块有效地捕捉了不同大小的具有诊断意义的区域。基于知识检索的解码器利用历史诊断数据提高报告准确性。我们的方法不仅提高了生成的病理报告的信息量和相关性，而且优于当前最先进的技术。
