Accuracy of no-reference quality metrics in network-impaired video streams

The Video Quality Metric (VQM) is nowadays one of the most used objective methods to assess video quality, thanks to its high correlation with both the human visual system (HVS) and subjective methods. VQM is, however, not viable in real-time deployments such as mobile streaming, not only due to its high computational demands but, specifically, because it is a Full-Reference (FR) metric, which requires as input both the original video and its impaired counterpart. On the other hand, No-Reference (NR) objective algorithms operate directly on the impaired video and are considerably faster, but loose out when it comes to accuracy. In this research, we assess a range of NR metrics, alongside a lightweight FR metric, using VQM as benchmark. Our study covers a range of methods, a diverse set of video types and encoding conditions, and a range of network impairment test-cases. We show the extent by which packet loss affects different video types, correlating the accuracy of NR metrics to the FR benchmark. Our study helps identifying the conditions under which simple metrics may be used effectively and indicates an avenue to control the quality of streaming systems in line with human perception.