Sinsbeck Michael, Höge Marvin, Nowak Wolfgang
Department of Stochastic Simulation and Safety Research for Hydrosystems (LS3), Institute for Modeling Hydraulic and Environmental Systems, University of Stuttgart, Stuttgart, Germany.
Front Artif Intell. 2020 Aug 13;3:52. doi: 10.3389/frai.2020.00052. eCollection 2020.
Methods for sequential design of computer experiments typically consist of two phases. In the first phase, the exploratory phase, a space-filling initial design is used to estimate hyperparameters of a Gaussian process emulator (GPE) and to provide some initial global exploration of the model function. In the second phase, more design points are added one by one to improve the GPE and to solve the actual problem at hand (e.g., Bayesian optimization, estimation of failure probabilities, solving Bayesian inverse problems). In this article, we investigate whether hyperparameters can be estimated without a separate exploratory phase. Such an approach will leave hyperparameters uncertain in the first iterations, so the acquisition function (which tells where to evaluate the model function next) and the GPE-based estimator need to be adapted to non-Gaussian random fields. Numerical experiments are performed exemplarily on a sequential method for solving Bayesian inverse problems. These experiments show that hyperparameters can indeed be estimated without an exploratory phase and the resulting method works almost as efficient as if the hyperparameters had been known beforehand. This means that the estimation of hyperparameters should not be the reason for including an exploratory phase. Furthermore, we show numerical examples, where these results allow us to eliminate the exploratory phase to make the sequential design method both faster (requiring fewer model evaluations) and easier to use (requiring fewer choices by the user).
计算机实验的序贯设计方法通常由两个阶段组成。在第一阶段，即探索阶段，使用空间填充初始设计来估计高斯过程模拟器（GPE）的超参数，并对模型函数进行一些初始的全局探索。在第二阶段，逐个添加更多的设计点以改进GPE并解决手头的实际问题（例如，贝叶斯优化、失效概率估计、解决贝叶斯逆问题）。在本文中，我们研究了是否可以在没有单独探索阶段的情况下估计超参数。这种方法在最初的迭代中会使超参数不确定，因此获取函数（它告诉接下来在哪里评估模型函数）和基于GPE的估计器需要适应非高斯随机场。以求解贝叶斯逆问题的序贯方法为例进行了数值实验。这些实验表明，确实可以在没有探索阶段的情况下估计超参数，并且所得方法的工作效率几乎与超参数事先已知时一样高。这意味着估计超参数不应成为包含探索阶段的理由。此外，我们展示了一些数值示例，在这些示例中，这些结果使我们能够消除探索阶段，从而使序贯设计方法更快（需要更少的模型评估）且更易于使用（用户需要做出的选择更少）。
