Department of Mathematical and Computational Sciences, National Institute of Technology Karnataka, Mangalore, 575025, India.
Sci Rep. 2023 Mar 23;13(1):4737. doi: 10.1038/s41598-023-32027-3.
For any machine learning model, finding the optimal hyperparameter setting has a direct and significant impact on the model's performance. In this paper, we discuss different types of hyperparameter optimization techniques. We compare the performance of some of the hyperparameter optimization techniques on image classification datasets with the help of AutoML models. In particular, the paper studies Bayesian optimization in depth and proposes the use of genetic algorithm, differential evolution and covariance matrix adaptation-evolutionary strategy for acquisition function optimization. Moreover, we compare these variants of Bayesian optimization with conventional Bayesian optimization and observe that the use of covariance matrix adaptation-evolutionary strategy and differential evolution improves the performance of standard Bayesian optimization. We also notice that Bayesian optimization tends to perform poorly when genetic algorithm is used for acquisition function optimization.
对于任何机器学习模型来说，找到最优的超参数设置对模型的性能都有着直接且显著的影响。在本文中，我们讨论了不同类型的超参数优化技术。我们借助 AutoML 模型，将一些超参数优化技术在图像分类数据集上的性能进行了比较。特别地，本文深入研究了贝叶斯优化，并提出了使用遗传算法、差分进化和协方差矩阵自适应进化策略进行获取函数优化。此外，我们将这些贝叶斯优化变体与传统的贝叶斯优化进行了比较，观察到使用协方差矩阵自适应进化策略和差分进化可以提高标准贝叶斯优化的性能。我们还注意到，当使用遗传算法进行获取函数优化时，贝叶斯优化的性能往往较差。
