Hernández Adrián, Amigó José M
Centro de Investigación Operativa, Universidad Miguel Hernández, Av. de la Universidad s/n, 03202 Elche, Spain.
Entropy (Basel). 2021 Feb 26;23(3):283. doi: 10.3390/e23030283.
Deep learning models and graphics processing units have completely transformed the field of machine learning. Recurrent neural networks and long short-term memories have been successfully used to model and predict complex systems. However, these classic models do not perform sequential reasoning, a process that guides a task based on perception and memory. In recent years, attention mechanisms have emerged as a promising solution to these problems. In this review, we describe the key aspects of attention mechanisms and some relevant attention techniques and point out why they are a remarkable advance in machine learning. Then, we illustrate some important applications of these techniques in the modeling of complex systems.
深度学习模型和图形处理单元彻底改变了机器学习领域。循环神经网络和长短期记忆已成功用于对复杂系统进行建模和预测。然而，这些经典模型并不执行序列推理，序列推理是一个基于感知和记忆来指导任务的过程。近年来，注意力机制已成为解决这些问题的一个有前景的方案。在本综述中，我们描述了注意力机制的关键方面以及一些相关的注意力技术，并指出它们为何是机器学习中的一项显著进展。然后，我们阐述了这些技术在复杂系统建模中的一些重要应用。
