Zhang Huifeng, Yue Dong, Dou Chunxia, Hancke Gerhard P
IEEE Trans Neural Netw Learn Syst. 2024 Feb;35(2):1773-1784. doi: 10.1109/TNNLS.2022.3185211. Epub 2024 Feb 5.
The ever-increasing requirements of demand response dynamics, competition among different stakeholders, and information privacy protection intensify the challenge of the optimal operation of microgrids. To tackle the above problems, this article proposes a three-stage optimization strategy with a deep reinforcement learning (DRL)-based distributed privacy optimization. In the upper layer of the model, the rule-based deep deterministic policy gradient (DDPG) algorithm is proposed to optimize the load migration problem with demand response, which enhances dynamic characteristics with the interaction between electricity prices and consumer behavior. Due to the competition among different stakeholders and the information privacy requirement in the middle layer of the model, a potential game-based distributed privacy optimization algorithm is improved to seek Nash equilibriums (NEs) with encoded exchange information by a distributed privacy-preserving optimization algorithm, which can ensure the convergence as well as protect privacy information of each stakeholder. In the lower layer of the model of each stakeholder, economic cost and emission rate are both taken as operation objectives, and a gradient descent-based multiobjective optimization method is employed to approach this objective. The simulation results confirm that the proposed three-stage optimization strategy can be a viable and efficient way for the optimal operation of microgrids.
需求响应动态性要求不断提高、不同利益相关者之间的竞争以及信息隐私保护，加剧了微电网优化运行的挑战。为解决上述问题，本文提出了一种基于深度强化学习（DRL）的分布式隐私优化的三阶段优化策略。在模型的上层，提出了基于规则的深度确定性策略梯度（DDPG）算法来优化具有需求响应的负荷迁移问题，通过电价与用户行为之间的相互作用增强动态特性。由于模型中间层存在不同利益相关者之间的竞争以及信息隐私要求，改进了一种基于势博弈的分布式隐私优化算法，通过分布式隐私保护优化算法寻找具有编码交换信息的纳什均衡（NE），既能保证收敛性，又能保护各利益相关者的隐私信息。在每个利益相关者的模型下层，将经济成本和排放率都作为运行目标，并采用基于梯度下降的多目标优化方法来实现这一目标。仿真结果证实，所提出的三阶段优化策略可以成为微电网优化运行的一种可行且高效的方法。
