Feng Zhe, Wu Zuheng, Zou Jianxun, Cheng Lingli, Zhao Xiaolong, Zhang Xumeng, Lu Jian, Wang Cong, Wang Yilin, Wang Haochen, Guo Wenbin, Qian Zhibin, Zhu Yunlai, Xu Zuyu, Dai Yuehua, Liu Qi
School of Integrated Circuits, Anhui University, Hefei, Anhui, China.
Frontier Institute of Chip and System, Fudan University, Shanghai, China.
Nat Commun. 2025 May 27;16(1):4925. doi: 10.1038/s41467-025-60085-w.
The Bellman equation, with a resource-consuming solving process, plays a fundamental role in formulating and solving dynamic optimization problems. The realization of the Bellman solver with memristive computing-in-memory (MCIM) technology, is significant for implementing efficient dynamic decision-making. However, the iterative nature of the Bellman equation solving process poses a challenge for efficient implementation on MCIM systems, which excel at vector-matrix multiplication (VMM) operations but are less suited for iterative algorithms. In this work, by incorporating the temporal dimension and transforming the solution into recurrent dot product operations, a memristive Bellman solver (MBS) is proposed, facilitating the implementation of the Bellman equation solving process with efficient MCIM technology. The MBS effectively reduces the iteration numbers and which further enhanced by approximated solutions leveraging memristor noise. Finally, the path planning tasks are used to verify the feasibility of the proposed MBS. The theoretical derivation and experimental results demonstrate that the MBS effectively reduces the iteration cycles, facilitating the solving efficiency. This work could be a sound of choice for developing high-efficiency decision-making systems.
贝尔曼方程在动态优化问题的制定和求解中起着基础性作用，但其求解过程会消耗资源。利用忆阻存内计算（MCIM）技术实现贝尔曼求解器，对于高效动态决策的实现具有重要意义。然而，贝尔曼方程求解过程的迭代特性给在MCIM系统上的高效实现带来了挑战，MCIM系统擅长向量矩阵乘法（VMM）运算，但不太适合迭代算法。在这项工作中，通过纳入时间维度并将解转换为循环点积运算，提出了一种忆阻贝尔曼求解器（MBS），便于利用高效的MCIM技术实现贝尔曼方程求解过程。MBS有效地减少了迭代次数，并且通过利用忆阻器噪声的近似解进一步增强。最后，使用路径规划任务来验证所提出的MBS的可行性。理论推导和实验结果表明，MBS有效地减少了迭代周期，提高了求解效率。这项工作可能是开发高效决策系统的一个不错选择。
