Duque-Ramos Astrid, Boeker Martin, Jansen Ludger, Schulz Stefan, Iniesta Miguela, Fernández-Breis Jesualdo Tomás
Departamento de Informática y Sistemas, Facultad de Informática, Universidad de Murcia, Murcia, Spain.
Department of Medical Biometry and Medical Informatics, University of Freiburg, Freiburg, Germany.
PLoS One. 2014 Aug 22;9(8):e104463. doi: 10.1371/journal.pone.0104463. eCollection 2014.
To (1) evaluate the GoodOD guideline for ontology development by applying the OQuaRE evaluation method and metrics to the ontology artefacts that were produced by students in a randomized controlled trial, and (2) informally compare the OQuaRE evaluation method with gold standard and competency questions based evaluation methods, respectively.
In the last decades many methods for ontology construction and ontology evaluation have been proposed. However, none of them has become a standard and there is no empirical evidence of comparative evaluation of such methods. This paper brings together GoodOD and OQuaRE. GoodOD is a guideline for developing robust ontologies. It was previously evaluated in a randomized controlled trial employing metrics based on gold standard ontologies and competency questions as outcome parameters. OQuaRE is a method for ontology quality evaluation which adapts the SQuaRE standard for software product quality to ontologies and has been successfully used for evaluating the quality of ontologies.
In this paper, we evaluate the effect of training in ontology construction based on the GoodOD guideline within the OQuaRE quality evaluation framework and compare the results with those obtained for the previous studies based on the same data.
Our results show a significant effect of the GoodOD training over developed ontologies by topics: (a) a highly significant effect was detected in three topics from the analysis of the ontologies of untrained and trained students; (b) both positive and negative training effects with respect to the gold standard were found for five topics.
The GoodOD guideline had a significant effect over the quality of the ontologies developed. Our results show that GoodOD ontologies can be effectively evaluated using OQuaRE and that OQuaRE is able to provide additional useful information about the quality of the GoodOD ontologies.
（1）通过将OQuaRE评估方法和指标应用于随机对照试验中学生生成的本体工件，评估GoodOD本体开发指南；（2）分别将OQuaRE评估方法与基于金标准和能力问题的评估方法进行非正式比较。
在过去几十年中，已经提出了许多本体构建和本体评估方法。然而，它们都没有成为标准，也没有关于此类方法比较评估的实证证据。本文将GoodOD和OQuaRE结合在一起。GoodOD是一个用于开发健壮本体的指南。它之前在一项随机对照试验中进行了评估，该试验采用基于金标准本体和能力问题的指标作为结果参数。OQuaRE是一种本体质量评估方法，它将软件产品质量的SQuaRE标准应用于本体，并已成功用于评估本体的质量。
在本文中，我们在OQuaRE质量评估框架内评估基于GoodOD指南的本体构建培训的效果，并将结果与基于相同数据的先前研究结果进行比较。
我们的结果表明，GoodOD培训对按主题开发的本体有显著影响：（a）从未经训练和经过训练的学生的本体分析中，在三个主题中检测到高度显著的影响；（b）在五个主题中发现了相对于金标准的正向和负向培训效果。
GoodOD指南对所开发本体的质量有显著影响。我们的结果表明，使用OQuaRE可以有效地评估GoodOD本体，并且OQuaRE能够提供有关GoodOD本体质量的额外有用信息。
