Su Ke, Su Hang, Li Chongxuan, Zhu Jun, Zhang Bo
IEEE Trans Neural Netw Learn Syst. 2024 Feb;35(2):2667-2679. doi: 10.1109/TNNLS.2022.3190820. Epub 2024 Feb 5.
Neural-symbolic models provide a powerful tool to tackle complex visual reasoning tasks by combining symbolic program execution for reasoning and deep representation learning for visual recognition. A probabilistic formulation of such models with stochastic latent variables can obtain an interpretable and legible reasoning system with less supervision. However, it is still nontrivial to generate reasonable symbolic structures without the guidance of domain knowledge, since it generally involves an optimization problem with both continuous and discrete variables. Despite the challenges, the interpretability of such symbolic structures provides an interface to regularize their generation by domain knowledge. In this article, we propose to incorporate the available domain knowledge into the learning process of probabilistic neural-symbolic (PNS) models via posterior constraints that directly regularize the structure posterior. In this way, our model is able to identify a middle point where the structure generation process mainly learns from data but also selectively borrows information from domain knowledge. We further present inductive reasoning where the posterior constraints can be automatically reweighted to handle noisy annotations. The experimental results show that our method achieves state-of-the-art performance on major abstract reasoning datasets and enjoys good generalization capability and data efficiency.
神经符号模型通过结合用于推理的符号程序执行和用于视觉识别的深度表征学习，提供了一个强大的工具来处理复杂的视觉推理任务。具有随机潜在变量的此类模型的概率公式可以在较少监督的情况下获得一个可解释且清晰的推理系统。然而，在没有领域知识指导的情况下生成合理的符号结构仍然并非易事，因为这通常涉及一个同时包含连续变量和离散变量的优化问题。尽管存在挑战，但此类符号结构的可解释性提供了一个接口，可通过领域知识对其生成进行正则化。在本文中，我们建议通过直接对结构后验进行正则化的后验约束，将可用的领域知识纳入概率神经符号（PNS）模型的学习过程。通过这种方式，我们的模型能够找到一个中间点，在该点结构生成过程主要从数据中学习，但也有选择地从领域知识中借用信息。我们进一步提出归纳推理，其中后验约束可以自动重新加权以处理有噪声的注释。实验结果表明，我们的方法在主要的抽象推理数据集上取得了领先的性能，并具有良好的泛化能力和数据效率。
