Department of Electronic Engineering, University of York, York YO10 5DD, UK.
Sensors (Basel). 2021 Mar 24;21(7):2284. doi: 10.3390/s21072284.
Medium access control (MAC) is one of the key requirements in underwater acoustic sensor networks (UASNs). For a MAC protocol to provide its basic function of efficient sharing of channel access, the highly dynamic underwater environment demands MAC protocols to be adaptive as well. Q-learning is one of the promising techniques employed in intelligent MAC protocol solutions, however, due to the long propagation delay, the performance of this approach is severely limited by reliance on an explicit reward signal to function. In this paper, we propose a restructured and a modified two stage Q-learning process to extract an implicit reward signal for a novel MAC protocol: Packet flow ALOHA with Q-learning (ALOHA-QUPAF). Based on a simulated pipeline monitoring chain network, results show that the protocol outperforms both ALOHA-Q and framed ALOHA by at least 13% and 148% in all simulated scenarios, respectively.
媒体访问控制（MAC）是水声传感器网络（UASN）的关键要求之一。对于一个 MAC 协议来说，要想有效地共享信道访问，其基本功能就是提供高效的信道访问，而高度动态的水下环境要求 MAC 协议具有自适应性。在智能 MAC 协议解决方案中，Q-learning 是一种很有前途的技术，然而，由于传播延迟较长，这种方法的性能严重受到依赖显式奖励信号的限制。在本文中，我们提出了一种重构和修改的两阶段 Q-learning 过程，以提取一种新的 MAC 协议的隐式奖励信号：基于 Q-learning 的分组流 ALOHA（ALOHA-QUPAF）。基于模拟的管道监测链网络，结果表明，该协议在所有模拟场景中分别比 ALOHA-Q 和有帧 ALOHA 至少提高了 13%和 148%。
