Virginia Modeling, Analysis and Simulation Center, Old Dominion University, Suffolk, VA, United States of America.
Department of Computational and Data Sciences, George Mason University, Fairfax, VA, United States of America.
PLoS One. 2020 May 13;15(5):e0232929. doi: 10.1371/journal.pone.0232929. eCollection 2020.
Verification is a crucial process to facilitate the identification and removal of errors within simulations. This study explores semantic changes to the concept of simulation verification over the past six decades using a data-supported, automated content analysis approach. We collect and utilize a corpus of 4,047 peer-reviewed Modeling and Simulation (M&S) publications dealing with a wide range of studies of simulation verification from 1963 to 2015. We group the selected papers by decade of publication to provide insights and explore the corpus from four perspectives: (i) the positioning of prominent concepts across the corpus as a whole; (ii) a comparison of the prominence of verification, validation, and Verification and Validation (V&V) as separate concepts; (iii) the positioning of the concepts specifically associated with verification; and (iv) an evaluation of verification's defining characteristics within each decade. Our analysis reveals unique characterizations of verification in each decade. The insights gathered helped to identify and discuss three categories of verification challenges as avenues of future research, awareness, and understanding for researchers, students, and practitioners. These categories include conveying confidence and maintaining ease of use; techniques' coverage abilities for handling increasing simulation complexities; and new ways to provide error feedback to model users.
验证是一个至关重要的过程，有助于识别和消除模拟中的错误。本研究使用数据支持的自动化内容分析方法，探索了过去六十年中模拟验证概念的语义变化。我们收集并利用了一个包含 4047 篇同行评审的建模与仿真 (M&S) 出版物的语料库，这些出版物涉及从 1963 年到 2015 年的各种模拟验证研究。我们按出版年代对选定的论文进行分组，以便从四个方面提供见解并探索语料库：（i）整个语料库中突出概念的定位；（ii）验证、确认和验证与确认 (V&V) 作为单独概念的突出程度的比较；（iii）与验证相关的概念的定位；以及（iv）在每个十年内评估验证的定义特征。我们的分析揭示了每个十年验证的独特特征。收集到的见解有助于确定并讨论未来研究、研究人员、学生和从业者的意识和理解的三个验证挑战类别，这些类别包括传达信心和保持易用性；处理不断增加的模拟复杂性的技术的覆盖能力；以及为模型用户提供错误反馈的新方法。
