Momeni Ali, Rahmani Babak, Malléjac Matthieu, Del Hougne Philipp, Fleury Romain
Laboratory of Wave Engineering, Department of Electrical Engineering, EPFL, Lausanne CH-1015, Switzerland.
Microsoft Research, Cambridge CB4 0AB, UK.
Science. 2023 Dec 15;382(6676):1297-1303. doi: 10.1126/science.adi8474. Epub 2023 Nov 23.
Recent successes in deep learning for vision and natural language processing are attributed to larger models but come with energy consumption and scalability issues. Current training of digital deep-learning models primarily relies on backpropagation that is unsuitable for physical implementation. In this work, we propose a simple deep neural network architecture augmented by a physical local learning (PhyLL) algorithm, which enables supervised and unsupervised training of deep physical neural networks without detailed knowledge of the nonlinear physical layer's properties. We trained diverse wave-based physical neural networks in vowel and image classification experiments, showcasing the universality of our approach. Our method shows advantages over other hardware-aware training schemes by improving training speed, enhancing robustness, and reducing power consumption by eliminating the need for system modeling and thus decreasing digital computation.
深度学习在视觉和自然语言处理方面的近期成功归因于更大的模型，但同时也伴随着能源消耗和可扩展性问题。当前数字深度学习模型的训练主要依赖于反向传播，而这并不适合物理实现。在这项工作中，我们提出了一种简单的深度神经网络架构，通过物理局部学习（PhyLL）算法进行增强，该算法能够在无需详细了解非线性物理层特性的情况下，对深度物理神经网络进行有监督和无监督训练。我们在元音和图像分类实验中训练了多种基于波的物理神经网络，展示了我们方法的通用性。我们的方法通过提高训练速度、增强鲁棒性以及通过消除系统建模需求从而减少数字计算来降低功耗，相比其他硬件感知训练方案具有优势。
