Guo Jia, Gong Xiangyang, Wang Wendong, Que Xirong, Liu Jingyu
State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing 100876, China.
Sensors (Basel). 2019 Jul 15;19(14):3121. doi: 10.3390/s19143121.
There are few network resources in wireless multimedia sensor networks (WMSNs). Compressing media data can reduce the reliance of user's Quality of Experience (QoE) on network resources. Existing video coding software, such as H.264 and H.265, focuses only on spatial and short-term information redundancy. However, video usually contains redundancy over a long period of time. Therefore, compressing video information redundancy with a long period of time without compromising the user experience and adaptive delivery is a challenge in WMSNs. In this paper, a semantic-aware super-resolution transmission for adaptive video streaming system (SASRT) for WMSNs is presented. In the SASRT, some deep learning algorithms are used to extract video semantic information and enrich the video quality. On the multimedia sensor, different bit-rate semantic information and video data are encoded and uploaded to user. Semantic information can also be identified on the user side, further reducing the amount of data that needs to be transferred. However, identifying semantic information on the user side may increase the computational cost of the user side. On the user side, video quality is enriched with super-resolution technologies. The major challenges faced by SASRT include where the semantic information is identified, how to choose the bit rates of semantic and video information, and how network resources should be allocated to video and semantic information. The optimization problem is formulated as a complexity-constrained nonlinear NP-hard problem. Three adaptive strategies and a heuristic algorithm are proposed to solve the optimization problem. Simulation results demonstrate that SASRT can compress video information redundancy with a long period of time effectively and enrich the user experience with limited network resources while simultaneously improving the utilization of these network resources.
无线多媒体传感器网络（WMSN）中的网络资源很少。压缩媒体数据可以减少用户体验质量（QoE）对网络资源的依赖。现有的视频编码软件，如H.264和H.265，仅关注空间和短期信息冗余。然而，视频通常在很长一段时间内包含冗余信息。因此，在不影响用户体验和自适应传输的前提下，压缩长时间的视频信息冗余是WMSN面临的一项挑战。本文提出了一种用于WMSN的自适应视频流系统的语义感知超分辨率传输（SASRT）。在SASRT中，使用一些深度学习算法来提取视频语义信息并提高视频质量。在多媒体传感器上，对不同比特率的语义信息和视频数据进行编码并上传给用户。在用户端也可以识别语义信息，进一步减少需要传输的数据量。然而，在用户端识别语义信息可能会增加用户端的计算成本。在用户端，利用超分辨率技术提高视频质量。SASRT面临的主要挑战包括语义信息的识别位置、如何选择语义和视频信息的比特率，以及如何将网络资源分配给视频和语义信息。该优化问题被表述为一个复杂度受限的非线性NP难问题。提出了三种自适应策略和一种启发式算法来解决该优化问题。仿真结果表明，SASRT能够有效地压缩长时间的视频信息冗余，并在有限的网络资源下丰富用户体验，同时提高这些网络资源的利用率。
