Tefera Mulugeta Kassaw, Zhang Shengbing, Jin Zengwang
School of Cybersecurity, Northwestern Polytechnical University, Xi'an 710072, China.
Entropy (Basel). 2023 Feb 24;25(3):413. doi: 10.3390/e25030413.
This paper considers a downlink resource-allocation problem in distributed interference orthogonal frequency-division multiple access (OFDMA) systems under maximal power constraints. As the upcoming fifth-generation (5G) wireless networks are increasingly complex and heterogeneous, it is challenging for resource allocation tasks to optimize the system performance metrics and guarantee user service requests simultaneously. Because of the non-convex optimization problems, using existing approaches to find the optimal resource allocation is computationally expensive. Recently, model-free reinforcement learning (RL) techniques have become alternative approaches in wireless networks to solve non-convex and NP-hard optimization problems. In this paper, we study a deep Q-learning (DQL)-based approach to address the optimization of transmit power control for users in multi-cell interference networks. In particular, we have applied a DQL algorithm for resource allocation to maximize the overall system throughput subject to the maximum power and SINR constraints in a flat frequency channel. We first formulate the optimization problem as a non-cooperative game model, where the multiple BSs compete for spectral efficiencies by improving their achievable utility functions while ensuring the quality of service (QoS) requirements to the corresponding receivers. Then, we develop a DRL-based resource allocation model to maximize the system throughput while satisfying the power and spectral efficiency requirements. In this setting, we define the state-action spaces and the reward function to explore the possible actions and learning outcomes. The numerical simulations demonstrate that the proposed DQL-based scheme outperforms the traditional model-based solution.
本文研究了在最大功率约束下分布式干扰正交频分多址（OFDMA）系统中的下行链路资源分配问题。随着即将到来的第五代（5G）无线网络日益复杂和异构，资源分配任务要同时优化系统性能指标并保证用户服务请求具有挑战性。由于存在非凸优化问题，使用现有方法寻找最优资源分配在计算上代价高昂。近来，无模型强化学习（RL）技术已成为无线网络中解决非凸和NP难优化问题的替代方法。在本文中，我们研究一种基于深度Q学习（DQL）的方法来解决多小区干扰网络中用户的发射功率控制优化问题。具体而言，我们应用一种DQL算法进行资源分配，以在平坦频率信道中的最大功率和信干噪比（SINR）约束下最大化整体系统吞吐量。我们首先将优化问题表述为一个非合作博弈模型，其中多个基站通过改善其可实现的效用函数来竞争频谱效率，同时确保对相应接收机的服务质量（QoS）要求。然后，我们开发一个基于深度强化学习（DRL）的资源分配模型，以在满足功率和频谱效率要求的同时最大化系统吞吐量。在此设置下，我们定义状态 - 动作空间和奖励函数以探索可能的动作和学习结果。数值模拟表明，所提出的基于DQL的方案优于传统的基于模型的解决方案。
