Department of Robotics and Automation, Symbiosis Institute of Technology (SIT), Symbiosis International (Deemed University) (SIU), Lavale, Pune, 412115, Maharashtra, India.
Neural Netw. 2024 Dec;180:106738. doi: 10.1016/j.neunet.2024.106738. Epub 2024 Sep 16.
The world today has made prescriptive analytics that uses data-driven insights to guide future actions. The distribution of data, however, differs depending on the scenario, making it difficult to interpret and comprehend the data efficiently. Different neural network models are used to solve this, taking inspiration from the complex network architecture in the human brain. The activation function is crucial in introducing non-linearity to process data gradients effectively. Although popular activation functions such as ReLU, Sigmoid, Swish, and Tanh have advantages and disadvantages, they may struggle to adapt to diverse data characteristics. A generalized activation function named the Generalized Exponential Parametric Activation Function (GEPAF) is proposed to address this issue. This function consists of three parameters expressed: α, which stands for a differencing factor similar to the mean; σ, which stands for a variance to control distribution spread; and p, which is a power factor that improves flexibility; all these parameters are present in the exponent. When p=2, the activation function resembles a Gaussian function. Initially, this paper describes the mathematical derivation and validation of the properties of this function mathematically and graphically. After this, the GEPAF function is practically implemented in real-world supply chain datasets. One dataset features a small sample size but exhibits high variance, while the other shows significant variance with a moderate amount of data. An LSTM network processes the dataset for sales and profit prediction. The suggested function performs better than popular activation functions when a comparative analysis of the activation function is performed, showing at least 30% improvement in regression evaluation metrics and better loss decay characteristics.
当今世界已经实现了预测分析，它利用数据驱动的洞察力来指导未来的行动。然而，数据的分布因场景而异，这使得数据难以有效解释和理解。不同的神经网络模型被用来解决这个问题，从人类大脑的复杂网络结构中汲取灵感。激活函数在引入非线性以有效处理数据梯度方面至关重要。虽然流行的激活函数，如 ReLU、Sigmoid、Swish 和 Tanh 具有优缺点，但它们可能难以适应多样化的数据特征。提出了一种名为广义指数参数激活函数 (GEPAF) 的广义激活函数来解决这个问题。这个函数由三个参数组成：α，代表类似于均值的差分因子；σ，代表控制分布扩散的方差；p，是一个提高灵活性的幂因子；所有这些参数都在指数中。当 p=2 时，激活函数类似于高斯函数。本文首先从数学上和图形上描述了这个函数的数学推导和性质验证。之后，在实际的供应链数据集上实现了 GEPAF 函数。一个数据集样本量小，但方差大，另一个数据集数据量适中，但方差大。LSTM 网络处理数据集以进行销售和利润预测。在对激活函数进行比较分析时，建议的函数表现优于流行的激活函数，回归评估指标至少提高了 30%，并且具有更好的损失衰减特性。
