IEEE Trans Image Process. 2018 Jul;27(7):3316-3331. doi: 10.1109/TIP.2018.2815842.
Streaming video services represent a very large fraction of global bandwidth consumption. Due to the exploding demands of mobile video streaming services, coupled with limited bandwidth availability, video streams are often transmitted through unreliable, low-bandwidth networks. This unavoidably leads to two types of major streaming-related impairments: compression artifacts and/or rebuffering events. In streaming video applications, the end-user is a human observer; hence being able to predict the subjective Quality of Experience (QoE) associated with streamed videos could lead to the creation of perceptually optimized resource allocation strategies driving higher quality video streaming services. We propose a variety of recurrent dynamic neural networks that conduct continuous-time subjective QoE prediction. By formulating the problem as one of time-series forecasting, we train a variety of recurrent neural networks and non-linear autoregressive models to predict QoE using several recently developed subjective QoE databases. These models combine multiple, diverse neural network inputs, such as predicted video quality scores, rebuffering measurements, and data related to memory and its effects on human behavioral responses, using them to predict QoE on video streams impaired by both compression artifacts and rebuffering events. Instead of finding a single time-series prediction model, we propose and evaluate ways of aggregating different models into a forecasting ensemble that delivers improved results with reduced forecasting variance. We also deploy appropriate new evaluation metrics for comparing time-series predictions in streaming applications. Our experimental results demonstrate improved prediction performance that approaches human performance. An implementation of this work can be found at https://github.com/christosbampis/NARX_QoE_release.
流媒体服务占据了全球带宽消耗的很大一部分。由于移动视频流媒体服务的需求不断增长，再加上带宽有限，视频流通常通过不可靠、低带宽的网络传输。这不可避免地导致了两种主要的与流媒体相关的干扰：压缩伪影和/或缓冲事件。在流媒体视频应用中，最终用户是人类观察者；因此，能够预测与流媒体相关的主观体验质量（QoE），可以创建感知优化的资源分配策略，从而提供更高质量的视频流媒体服务。我们提出了各种递归动态神经网络，用于进行连续时间主观 QoE 预测。通过将问题表述为时间序列预测问题，我们使用几种新开发的主观 QoE 数据库来训练各种递归神经网络和非线性自回归模型，以预测 QoE。这些模型结合了多种不同的神经网络输入，例如预测的视频质量分数、缓冲测量值以及与记忆及其对人类行为反应的影响相关的数据，以预测同时受到压缩伪影和缓冲事件影响的视频流的 QoE。我们没有找到单一的时间序列预测模型，而是提出并评估了将不同模型聚合到预测集成中的方法，从而在降低预测方差的同时提供更好的结果。我们还为流媒体应用中的时间序列预测比较部署了适当的新评估指标。我们的实验结果表明，预测性能得到了提高，接近人类表现。这项工作的实现可以在 https://github.com/christosbampis/NARX_QoE_release 上找到。
