Technische Hochschule Mittelhessen-THM University of Applied Sciences, Giessen, Germany.
Justus Liebig University Giessen, Giessen, Germany.
PLoS One. 2021 Jul 19;16(7):e0254749. doi: 10.1371/journal.pone.0254749. eCollection 2021.
One should assume that in silico experiments in systems biology are less susceptible to reproducibility issues than their wet-lab counterparts, because they are free from natural biological variations and their environment can be fully controlled. However, recent studies show that only half of the published mathematical models of biological systems can be reproduced without substantial effort. In this article we examine the potential causes for failed or cumbersome reproductions in a case study of a one-dimensional mathematical model of the atrioventricular node, which took us four months to reproduce. The model demonstrates that even otherwise rigorous studies can be hard to reproduce due to missing information, errors in equations and parameters, a lack in available data files, non-executable code, missing or incomplete experiment protocols, and missing rationales behind equations. Many of these issues seem similar to problems that have been solved in software engineering using techniques such as unit testing, regression tests, continuous integration, version control, archival services, and a thorough modular design with extensive documentation. Applying these techniques, we reimplement the examined model using the modeling language Modelica. The resulting workflow is independent of the model and can be translated to SBML, CellML, and other languages. It guarantees methods reproducibility by executing automated tests in a virtual machine on a server that is physically separated from the development environment. Additionally, it facilitates results reproducibility, because the model is more understandable and because the complete model code, experiment protocols, and simulation data are published and can be accessed in the exact version that was used in this article. We found the additional design and documentation effort well justified, even just considering the immediate benefits during development such as easier and faster debugging, increased understandability of equations, and a reduced requirement for looking up details from the literature.
人们认为，系统生物学中的计算机实验比湿实验具有更低的重现性问题，因为它们不受自然生物变异的影响，其环境可以完全控制。然而，最近的研究表明，只有一半已发表的生物系统数学模型可以在不付出大量努力的情况下重现。在本文中，我们通过对房室结一维数学模型的案例研究来检验重现失败或繁琐的潜在原因，我们花了四个月的时间才重现该模型。该模型表明，即使是其他方面严格的研究也可能由于信息缺失、方程和参数错误、可用数据文件缺失、不可执行的代码、缺失或不完整的实验方案以及方程背后的不合理原因而难以重现。其中许多问题似乎与软件工程中使用单元测试、回归测试、持续集成、版本控制、存档服务以及广泛文档记录的详细模块化设计等技术已经解决的问题相似。我们应用这些技术，使用建模语言 Modelica 重新实现了所检查的模型。由此产生的工作流程与模型无关，可以转换为 SBML、CellML 和其他语言。它通过在服务器上的虚拟机中执行自动化测试来保证方法的可重现性，服务器与开发环境物理隔离。此外，它还促进了结果的可重现性，因为模型更加易于理解，并且完整的模型代码、实验方案和模拟数据都已发布并可在本文中使用的精确版本中访问。即使仅考虑开发过程中的直接好处，例如更轻松、更快速的调试、增加对方程的理解以及减少从文献中查找详细信息的需求，我们也认为额外的设计和文档工作是合理的。
