Li Shiqi, Cui Zhendong, Pei Yongquan
School of Computer and Control Engineering, Yantai University, Yantai 264005, China.
Entropy (Basel). 2022 Dec 15;24(12):1831. doi: 10.3390/e24121831.
Click-through rate (CTR) prediction is crucial for computing advertisement and recommender systems. The key challenge of CTR prediction is to accurately capture user interests and deliver suitable advertisements to the right people. However, there are an immense number of features in CTR prediction datasets, which hardly fit when only using an individual feature. To solve this problem, feature interaction that combines several features via an operation is introduced to enhance prediction performance. Many factorizations machine-based models and deep learning methods have been proposed to capture feature interaction for CTR prediction. They follow an enumeration-filter pattern that could not determine the appropriate order of feature interaction and useful feature interaction. The attention logarithmic network (ALN) is presented in this paper, which uses logarithmic neural networks (LNN) to model feature interactions, and the squeeze excitation (SE) mechanism to adaptively model the importance of higher-order feature interactions. At first, the embedding vector of the input was absolutized and a very small positive number was added to the zeros of the embedding vector, which made the LNN input positive. Then, the adaptive-order feature interactions were learned by logarithmic transformation and exponential transformation in the LNN. Finally, SE was applied to model the importance of high-order feature interactions adaptively for enhancing CTR performance. Based on this, the attention logarithmic interaction network (ALIN) was proposed for the effectiveness and accuracy of CTR, which integrated Newton's identity into ALN. ALIN supplements the loss of information, which is caused by the operation becoming positive and by adding a small positive value to the embedding vector. Experiments are conducted on two datasets, and the results prove that ALIN is efficient and effective.
点击率（CTR）预测对于计算广告和推荐系统至关重要。CTR预测的关键挑战在于准确捕捉用户兴趣并向合适的人投放合适的广告。然而，CTR预测数据集中存在大量特征，仅使用单个特征时很难适用。为了解决这个问题，引入了通过某种操作组合多个特征的特征交互，以提高预测性能。已经提出了许多基于因子分解机的模型和深度学习方法来捕捉CTR预测中的特征交互。它们遵循枚举-过滤模式，无法确定特征交互的合适顺序和有用的特征交互。本文提出了注意力对数网络（ALN），它使用对数神经网络（LNN）对特征交互进行建模，并使用挤压激励（SE）机制自适应地对高阶特征交互的重要性进行建模。首先，将输入的嵌入向量取绝对值，并在嵌入向量的零值上添加一个非常小的正数，这使得LNN输入为正数。然后，通过LNN中的对数变换和指数变换学习自适应顺序的特征交互。最后，应用SE自适应地对高阶特征交互的重要性进行建模，以提高CTR性能。基于此，为了提高CTR的有效性和准确性，提出了注意力对数交互网络（ALIN），它将牛顿恒等式集成到ALN中。ALIN弥补了由于操作变为正数以及向嵌入向量添加小的正值而导致的信息损失。在两个数据集上进行了实验，结果证明ALIN是高效且有效的。
