School of Software, Shandong University, Jinan SD, China.
Center for Medical Artificial Intelligence, Shandong University of Traditional Chinese Medicine, Qingdao SD, China.
Med Image Anal. 2021 Jan;67:101872. doi: 10.1016/j.media.2020.101872. Epub 2020 Oct 21.
Automated medical report generation in spine radiology, i.e., given spinal medical images and directly create radiologist-level diagnosis reports to support clinical decision making, is a novel yet fundamental study in the domain of artificial intelligence in healthcare. However, it is incredibly challenging because it is an extremely complicated task that involves visual perception and high-level reasoning processes. In this paper, we propose the neural-symbolic learning (NSL) framework that performs human-like learning by unifying deep neural learning and symbolic logical reasoning for the spinal medical report generation. Generally speaking, the NSL framework firstly employs deep neural learning to imitate human visual perception for detecting abnormalities of target spinal structures. Concretely, we design an adversarial graph network that interpolates a symbolic graph reasoning module into a generative adversarial network through embedding prior domain knowledge, achieving semantic segmentation of spinal structures with high complexity and variability. NSL secondly conducts human-like symbolic logical reasoning that realizes unsupervised causal effect analysis of detected entities of abnormalities through meta-interpretive learning. NSL finally fills these discoveries of target diseases into a unified template, successfully achieving a comprehensive medical report generation. When employed in a real-world clinical dataset, a series of empirical studies demonstrate its capacity on spinal medical report generation and show that our algorithm remarkably exceeds existing methods in the detection of spinal structures. These indicate its potential as a clinical tool that contributes to computer-aided diagnosis.
脊柱放射学中的自动化医学报告生成，即给定脊柱医学图像，并直接创建放射科医生级别的诊断报告，以支持临床决策，是人工智能在医疗保健领域中的一项新颖而基础的研究。然而，这是一项极具挑战性的任务，因为它涉及到视觉感知和高级推理过程，极其复杂。在本文中，我们提出了神经符号学习（NSL）框架，通过将深度学习和符号逻辑推理相结合，实现了类似于人类的学习，用于脊柱医学报告生成。一般来说，NSL 框架首先利用深度学习来模仿人类的视觉感知，以检测目标脊柱结构的异常。具体来说，我们设计了一个对抗图网络，通过嵌入先验领域知识，将符号图推理模块插入生成对抗网络中，实现了具有高复杂度和可变性的脊柱结构的语义分割。NSL 然后进行类似于人类的符号逻辑推理，通过元解释学习实现对异常检测实体的无监督因果效应分析。NSL 最后将这些目标疾病的发现填入统一模板，成功实现了全面的医学报告生成。当应用于真实的临床数据集时，一系列实证研究证明了其在脊柱医学报告生成中的能力，并表明我们的算法在检测脊柱结构方面明显优于现有方法。这表明它有作为一种临床工具的潜力，可以帮助计算机辅助诊断。
