Ma Jie, Chai Qi, Liu Jun, Yin Qingyu, Wang Pinghui, Zheng Qinghua
IEEE Trans Neural Netw Learn Syst. 2024 Nov;35(11):16493-16503. doi: 10.1109/TNNLS.2023.3294991. Epub 2024 Oct 29.
Textbook question answering (TQA) is the task of correctly answering diagram or nondiagram (ND) questions given large multimodal contexts consisting of abundant essays and diagrams. In real-world scenarios, an explainable TQA system plays a key role in deepening humans' understanding of learned knowledge. However, there is no work to investigate how to provide explanations currently. To address this issue, we devise a novel architecture toward span-level eXplanations for TQA (XTQA). In this article, spans are the combinations of sentences within a paragraph. The key idea is to consider the entire textual context of a lesson as candidate evidence and then use our proposed coarse-to-fine grained explanation extracting (EE) algorithm to narrow down the evidence scope and extract the span-level explanations with varying lengths for answering different questions. The EE algorithm can also be integrated into other TQA methods to make them explainable and improve the TQA performance. Experimental results show that XTQA obtains the best overall explanation result [mean intersection over union (mIoU)] of 52.38% on the first 300 questions of CK12-QA test splits, demonstrating the explainability of our method (ND: 150 and diagram: 150). The results also show that XTQA achieves the best TQA performance of 36.46% and 36.95% on the aforementioned splits. We have released our code in https://github.com/dr-majie/opentqa.
教科书问答(TQA)是指在由大量文章和图表组成的大型多模态语境下正确回答图表或非图表(ND)问题的任务。在现实场景中,一个可解释的TQA系统在加深人类对所学知识的理解方面起着关键作用。然而,目前尚无研究如何提供解释的工作。为了解决这个问题,我们设计了一种新颖的架构用于TQA的跨度级解释(XTQA)。在本文中,跨度是段落内句子的组合。关键思想是将课程的整个文本语境视为候选证据,然后使用我们提出的从粗到细粒度的解释提取(EE)算法来缩小证据范围,并提取不同长度的跨度级解释以回答不同问题。EE算法还可以集成到其他TQA方法中,使其具有可解释性并提高TQA性能。实验结果表明,XTQA在CK12 - QA测试拆分的前300个问题上获得了最佳的总体解释结果[平均交并比(mIoU)],为52.38%,证明了我们方法的可解释性(ND:150个问题,图表:150个问题)。结果还表明,XTQA在上述拆分上实现了最佳的TQA性能,分别为36.46%和36.95%。我们已将代码发布在https://github.com/dr-majie/opentqa 。