Hao Xubing, Cui Licong, Tao Cui, Roberts Kirk, Amith Muhammad
McWilliams School of Biomedical Informatics, The University of Texas Health Science Center at Houston.
Department of Artificial Intelligence and Informatics, Mayo Clinic.
CEUR Workshop Proc. 2024 Nov;3853.
Ontology development involves a top-down approach where ontology engineers and domain experts collaboratively define and evaluate ontological elements and axioms. Translating ontology axioms into natural language can significantly aid in ontology evaluation by making the content more understandable to subject matter experts who may lack a background in knowledge engineering. In this preliminary study, we investigate the potential of large language models (LLMs) in axiom translation from ontologies to facilitate ontology evaluation. We utilize Llama 3 to translate 1,192 ontology axioms across 19 distinct axiom types from five published ontologies. Results show that 163 (13.67%) of the Llama 3 translation of the axiom are accurately represented, 268 (22.48%) are not accurately represented, and 761 (63.84%) are partially accurate. Our manual evaluation of the Llama 3 translation indicates some competency in producing hierarchical natural language equivalents while revealing some limitations when translating complex axioms. Nonetheless, there are opportunities to improve the results with few-shot training or using LLMs to provide support in knowledge engineering for ontologies.
本体开发涉及一种自上而下的方法，本体工程师和领域专家在其中协作定义和评估本体元素及公理。将本体公理翻译成自然语言，可以使缺乏知识工程背景的主题专家更容易理解内容，从而极大地有助于本体评估。在这项初步研究中，我们探讨了大语言模型（LLMs）在将本体公理翻译以促进本体评估方面的潜力。我们利用Llama 3翻译来自五个已发表本体的19种不同公理类型的1192条本体公理。结果显示，Llama 3对公理的翻译中有163条（13.67%）得到了准确表述，268条（22.48%）未得到准确表述，761条（63.84%）部分准确。我们对Llama 3翻译的人工评估表明，它在生成层次化自然语言等效表述方面有一定能力，但在翻译复杂公理时也存在一些局限性。尽管如此，通过少样本训练或利用大语言模型为本体的知识工程提供支持，仍有机会改进结果。
