Department of Computer Science, Faculty of Computers and Information, Mansoura University, Mansoura, 35516, Egypt.
Department of Information Systems, Faculty of Computers and Information, Mansoura University, Mansoura, 35516, Egypt.
Sci Rep. 2024 Aug 22;14(1):19534. doi: 10.1038/s41598-024-69193-x.
Optimizers are the bottleneck of the training process of any Convolutionolution neural networks (CNN) model. One of the critical steps when work on CNN model is choosing the optimal optimizer to solve a specific problem. Recent challenge in nowadays researches is building new versions of traditional CNN optimizers that can work more efficient than the traditional optimizers. Therefore, this work proposes a novel enhanced version of Adagrad optimizer called SAdagrad that avoids the drawbacks of Adagrad optimizer in dealing with tuning the learning rate value for each step of the training process. In order to evaluate SAdagrad, this paper builds a CNN model that combines a fine- tuning technique and a weight decay technique together. It trains the proposed CNN model on Kather colorectal cancer histology dataset which is one of the most challenging datasets in recent researches of Diagnose of Colorectal Cancer (CRC). In fact, recently, there have been plenty of deep learning models achieving successful results with regard to CRC classification experiments. However, the enhancement of these models remains challenging. To train our proposed model, a learning transfer process, which is adopted from a pre-complicated defined model is applied to the proposed model and combined it with a regularization technique that helps in avoiding overfitting. The experimental results show that SAdagrad reaches a remarkable accuracy (98%), when compared with Adaptive momentum optimizer (Adam) and Adagrad optimizer. The experiments also reveal that the proposed model has a more stable training and testing processes, can reduce the overfitting problem in multiple epochs and can achieve a higher accuracy compared with previous researches on Diagnosis CRC using the same Kather colorectal cancer histology dataset.
优化器是任何卷积神经网络（CNN）模型训练过程的瓶颈。在处理 CNN 模型时，选择最佳优化器来解决特定问题是至关重要的步骤之一。当今研究的最新挑战是构建新版本的传统 CNN 优化器，使其比传统优化器更有效率。因此，这项工作提出了一种名为 SAdagrad 的 Adagrad 优化器的增强版本，该版本避免了 Adagrad 优化器在处理训练过程中每个步骤的学习率值调整方面的缺点。为了评估 SAdagrad，本文构建了一个结合微调技术和权重衰减技术的 CNN 模型。该模型在 Kather 结直肠癌组织学数据集上进行训练，该数据集是最近结直肠癌诊断研究中最具挑战性的数据集之一。事实上，最近有许多深度学习模型在 CRC 分类实验中取得了成功的结果。然而，这些模型的增强仍然具有挑战性。为了训练我们提出的模型，采用了从预先定义的复杂模型中迁移学习的过程，并将其与正则化技术相结合，有助于避免过拟合。实验结果表明，与自适应动量优化器（Adam）和 Adagrad 优化器相比，SAdagrad 达到了显著的准确性（98%）。实验还表明，与使用相同的 Kather 结直肠癌组织学数据集的 CRC 诊断的先前研究相比，所提出的模型具有更稳定的训练和测试过程，可以减少多个时期的过拟合问题，并可以实现更高的准确性。
