Juang Chia-Feng, Hsu Chia-Hung
Department of Electrical Engineering, National Chung-Hsing University, Taichung, Taiwan.
IEEE Trans Syst Man Cybern B Cybern. 2009 Dec;39(6):1528-42. doi: 10.1109/TSMCB.2009.2020569. Epub 2009 May 27.
This paper proposes a new reinforcement-learning method using online rule generation and Q-value-aided ant colony optimization (ORGQACO) for fuzzy controller design. The fuzzy controller is based on an interval type-2 fuzzy system (IT2FS). The antecedent part in the designed IT2FS uses interval type-2 fuzzy sets to improve controller robustness to noise. There are initially no fuzzy rules in the IT2FS. The ORGQACO concurrently designs both the structure and parameters of an IT2FS. We propose an online interval type-2 rule generation method for the evolution of system structure and flexible partitioning of the input space. Consequent part parameters in an IT2FS are designed using Q -values and the reinforcement local-global ant colony optimization algorithm. This algorithm selects the consequent part from a set of candidate actions according to ant pheromone trails and Q-values, both of which are updated using reinforcement signals. The ORGQACO design method is applied to the following three control problems: 1) truck-backing control; 2) magnetic-levitation control; and 3) chaotic-system control. The ORGQACO is compared with other reinforcement-learning methods to verify its efficiency and effectiveness. Comparisons with type-1 fuzzy systems verify the noise robustness property of using an IT2FS.
本文提出了一种新的强化学习方法，即使用在线规则生成和Q值辅助蚁群优化（ORGQACO）来设计模糊控制器。该模糊控制器基于区间二型模糊系统（IT2FS）。所设计的IT2FS的前件部分使用区间二型模糊集来提高控制器对噪声的鲁棒性。IT2FS最初没有模糊规则。ORGQACO同时设计IT2FS的结构和参数。我们提出了一种在线区间二型规则生成方法，用于系统结构的演化和输入空间的灵活划分。IT2FS的后件部分参数使用Q值和强化局部-全局蚁群优化算法进行设计。该算法根据蚂蚁信息素踪迹和Q值从一组候选动作中选择后件部分，二者均使用强化信号进行更新。ORGQACO设计方法应用于以下三个控制问题：1）卡车倒车控制；2）磁悬浮控制；3）混沌系统控制。将ORGQACO与其他强化学习方法进行比较，以验证其效率和有效性。与一型模糊系统的比较验证了使用IT2FS的噪声鲁棒性。
Zotero 插件