School of Engineering, University Of Kent, UK.
Faculty Of Mechanical Engineering, University Of Tabriz, Tabriz, Iran.
Neural Netw. 2024 Dec;180:106656. doi: 10.1016/j.neunet.2024.106656. Epub 2024 Aug 22.
This paper presents a new hybrid learning and control method that can tune their parameters based on reinforcement learning. In the new proposed method, nonlinear controllers are considered multi-input multi-output functions and then the functions are replaced with SNNs with reinforcement learning algorithms. Dopamine-modulated spike-timing-dependent plasticity (STDP) is used for reinforcement learning and manipulating the synaptic weights between the input and output of neuronal groups (for parameter adjustment). Details of the method are presented and some case studies are done on nonlinear controllers such as Fractional Order PID (FOPID) and Feedback Linearization. The structure and the dynamic equations for learning are presented, and the proposed algorithm is tested on robots and results are compared with other works. Moreover, to demonstrate the effectiveness of SNNFOPID, we conducted rigorous testing on a variety of systems including a two-wheel mobile robot, a double inverted pendulum, and a four-link manipulator robot. The results revealed impressively low errors of 0.01 m, 0.03 rad, and 0.03 rad for each system, respectively. The method is tested on another controller named Feedback Linearization, which provides acceptable results. Results show that the new method has better performance in terms of Integral Absolute Error (IAE) and is highly useful in hardware implementation due to its low energy consumption, high speed, and accuracy. The duration necessary for achieving full and stable proficiency in the control of various robotic systems using SNNFOPD, and SNNFL on an Asus Core i5 system within Simulink's Simscape environment is as follows: - Two-link robot manipulator with SNNFOPID: 19.85656 hours - Two-link robot manipulator with SNNFL: 0.45828 hours - Double inverted pendulum with SNNFOPID: 3.455 hours - Mobile robot with SNNFOPID: 3.71948 hours - Four-link robot manipulator with SNNFOPID: 16.6789 hours. This method can be generalized to other controllers and systems like robots.
本文提出了一种新的混合学习和控制方法，该方法可以基于强化学习来调整其参数。在新提出的方法中，非线性控制器被视为多输入多输出函数，然后这些函数被用具有强化学习算法的 SNN 替换。多巴胺调制的尖峰时间依赖可塑性（STDP）用于强化学习，并操纵神经元群体的输入和输出之间的突触权重（用于参数调整）。本文介绍了该方法的细节，并对非线性控制器（如分数阶 PID（FOPID）和反馈线性化）进行了一些案例研究。介绍了学习的结构和动态方程，并用机器人对提出的算法进行了测试，并将结果与其他工作进行了比较。此外，为了展示 SNNFOPID 的有效性，我们在各种系统（包括两轮移动机器人、双倒立摆和四连杆机械手机器人）上进行了严格的测试。结果令人印象深刻，每个系统的误差分别为 0.01 米、0.03 弧度和 0.03 弧度。该方法在另一个名为反馈线性化的控制器上进行了测试，结果令人满意。结果表明，该新方法在积分绝对误差（IAE）方面具有更好的性能，并且由于其低能耗、高速和准确性，在硬件实现中非常有用。在华硕 Core i5 系统内的 Simulink 的 Simscape 环境中，使用 SNNFOPD 和 SNNFL 实现对各种机器人系统的完全和稳定控制所需的时间如下：- 带有 SNNFOPID 的两连杆机器人：19.85656 小时- 带有 SNNFL 的两连杆机器人：0.45828 小时- 带有 SNNFOPID 的双倒立摆：3.455 小时- 带有 SNNFOPID 的移动机器人：3.71948 小时- 带有 SNNFOPID 的四连杆机器人：16.6789 小时。该方法可以推广到其他控制器和系统，如机器人。
