Lee Youngsaeng, Park Jeong-Soo
Data Science Lab, Korea Electric Power Corporation, Seoul 60732, Korea.
Department of Statistics, Chonnam National University, Gwangju 61186, Korea.
Entropy (Basel). 2020 Sep 4;22(9):985. doi: 10.3390/e22090985.
The approximated nonlinear least squares (ALS) method has been used for the estimation of unknown parameters in the complex computer code which is very time-consuming to execute. The ALS calibrates or tunes the computer code by minimizing the squared difference between real observations and computer output using a surrogate such as a Gaussian process model. When the differences (residuals) are correlated or heteroscedastic, the ALS may result in a distorted code tuning with a large variance of estimation. Another potential drawback of the ALS is that it does not take into account the uncertainty in the approximation of the computer model by a surrogate. To address these problems, we propose a generalized ALS (GALS) by constructing the covariance matrix of residuals. The inverse of the covariance matrix is multiplied to the residuals, and it is minimized with respect to the tuning parameters. In addition, we consider an iterative version for the GALS, which is called as the max-minG algorithm. In this algorithm, the parameters are re-estimated and updated by the maximum likelihood estimation and the GALS, by using both computer and experimental data repeatedly until convergence. Moreover, the iteratively re-weighted ALS method (IRWALS) was considered for a comparison purpose. Five test functions in different conditions are examined for a comparative analysis of the four methods. Based on the test function study, we find that both the bias and variance of estimates obtained from the proposed methods (the GALS and the max-minG) are smaller than those from the ALS and the IRWALS methods. Especially, the max-minG works better than others including the GALS for the relatively complex test functions. Lastly, an application to a nuclear fusion simulator is illustrated and it is shown that the abnormal pattern of residuals in the ALS can be resolved by the proposed methods.
近似非线性最小二乘法（ALS）已被用于估计复杂计算机代码中的未知参数，该代码执行起来非常耗时。ALS通过使用高斯过程模型等替代模型，最小化实际观测值与计算机输出之间的平方差来校准或调整计算机代码。当差异（残差）相关或具有异方差时，ALS可能会导致代码调整失真，估计方差较大。ALS的另一个潜在缺点是，它没有考虑替代模型对计算机模型近似中的不确定性。为了解决这些问题，我们通过构造残差协方差矩阵提出了一种广义ALS（GALS）。将协方差矩阵的逆与残差相乘，并相对于调整参数将其最小化。此外，我们考虑了GALS的迭代版本，称为最大-最小G算法。在该算法中，通过最大似然估计和GALS，反复使用计算机和实验数据对参数进行重新估计和更新，直到收敛。此外，为了进行比较，还考虑了迭代加权ALS方法（IRWALS）。对不同条件下的五个测试函数进行了检验，以对这四种方法进行比较分析。基于测试函数研究，我们发现所提出的方法（GALS和最大-最小G）得到的估计值的偏差和方差均小于ALS和IRWALS方法。特别是，对于相对复杂的测试函数，最大-最小G比包括GALS在内的其他方法表现更好。最后，说明了在核聚变模拟器中的应用，结果表明所提出的方法可以解决ALS中残差的异常模式。
