Zhou Shanglin, Li Yingjie, Gao Weilu, Yu Cunxi, Ding Caiwen
School of Computing, University of Connecticut, Storrs, 06269, USA.
A. James Clark School of Engineering, University of Maryland, College Park, 20742, USA.
Sci Rep. 2025 Apr 25;15(1):14419. doi: 10.1038/s41598-025-97262-2.
The democratization of AI encourages multi-task learning (MTL), demanding more parameters and processing time. To achieve highly energy-efficient MTL, Diffractive Optical Neural Networks (DONNs) have garnered attention due to extremely low energy and high computation speed. However, implementing MTL on DONNs requires manually reconfiguring & replacing layers, and rebuilding & duplicating the physical optical systems. To overcome the challenges, we propose LUMEN-PRO, an automated MTL framework using DONNs. We first propose to automate MTL utilizing an arbitrary backbone DONN and a set of tasks, resulting in a high-accuracy multi-task DONN model with small memory footprint that surpasses existing MTL. Second, we leverage the rotability of the physical optical system and replace task-specific layers with rotation of the corresponding shared layers. This replacement eliminates the storage requirement of task-specific layers, further optimizing the memory footprint. LUMEN-PRO provides flexibility in identifying optimal sharing patterns across diverse datasets, facilitating the search for highly energy-efficient DONNs. Experiments show that LUMEN-PRO provides up to 49.58% higher accuracy and 4× better cost efficiency than single-task and existing DONN approaches. It achieves memory lower bound of MTL, with memory efficiency matching single-task models. Compared to IBM-TrueNorth, LUMEN-PRO achieves an [Formula: see text] energy efficiency gain, while it matches Nanophotonic in efficiency but surpasses it in per-operator efficiency due to its larger system.
人工智能的民主化推动了多任务学习（MTL），这需要更多的参数和处理时间。为了实现高能效的多任务学习，衍射光学神经网络（DONNs）因其极低的能耗和高计算速度而受到关注。然而，在衍射光学神经网络上实现多任务学习需要手动重新配置和替换层，以及重建和复制物理光学系统。为了克服这些挑战，我们提出了LUMEN-PRO，这是一个使用衍射光学神经网络的自动化多任务学习框架。我们首先建议利用任意主干衍射光学神经网络和一组任务来自动化多任务学习，从而得到一个高精度、小内存占用的多任务衍射光学神经网络模型，该模型超越了现有的多任务学习方法。其次，我们利用物理光学系统的可旋转性，通过旋转相应的共享层来替换特定任务的层。这种替换消除了特定任务层的存储需求，进一步优化了内存占用。LUMEN-PRO在识别不同数据集的最佳共享模式方面提供了灵活性，有助于寻找高能效的衍射光学神经网络。实验表明，与单任务和现有的衍射光学神经网络方法相比，LUMEN-PRO的准确率提高了49.58%，成本效率提高了4倍。它实现了多任务学习的内存下限，内存效率与单任务模型相当。与IBM-TrueNorth相比，LUMEN-PRO实现了[公式：见原文]的能源效率提升，虽然它在效率上与纳米光子学相当，但由于其系统更大，在每个算子的效率上超过了纳米光子学。
