On the Number of Participants Needed for Subjective Quality Assessment of 360° Videos
2019 (English)In: 13th International Conference on Signal Processing and Communication Systems, ICSPCS 2019 - Proceedings, Institute of Electrical and Electronics Engineers Inc. , 2019Conference paper, Published paper (Refereed)
Abstract [en]
Immersive multimedia applications such as 360° videos and virtual reality (VR) live broadcasting are expected to receive increased popularity in the consumer markets. These applications provide three-dimensional spatial information and require to develop the related advanced wired and wireless ecosystems ranging from head-mounted displays (HMDs) and processors to network connectivity. To develop immersive multimedia systems that provide acceptable user experiences, it is essential to conduct subjective experiments that support assessing subjective video quality as perceived by the users. In this paper, we focus on the question of how many participants are needed in subjective experiments for 360° videos such that a statistically meaningful subjective quality assessment is supported. For this purpose, a subjective experiment for 360° videos has been conducted using the modified absolute category rating with hidden reference (M-ACR-HR) method. A statistical analysis of the opinion scores given by the participants to the visual stimuli shown on an HMD is provided. In particular, mean opinion scores (MOS), differential MOS (DMOS), and the associated margin of error (MoE) and differential MoE are obtained. It is shown that 21 to 26 participants would be sufficient to drive the differential MoE of the DMOS below the given thresholds of ϵ =1% and ϵ=0.75%, respectively. © 2019 IEEE.
Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers Inc. , 2019.
Keywords [en]
360° videos, differential margin of error, experimental design, M-ACR-HR, number of participants, statistical analysis, subjective experiments, Design of experiments, Helmet mounted displays, Multimedia systems, Statistical methods, User experience, Virtual reality, Absolute category ratings, Margin of error, Multimedia applications, Subjective quality assessments, Subjective video quality, Signal processing
National Category
Telecommunications
Identifiers
URN: urn:nbn:se:bth-19370DOI: 10.1109/ICSPCS47537.2019.9008748ISI: 000613248200084Scopus ID: 2-s2.0-85082024269ISBN: 9781728121949 (print)OAI: oai:DiVA.org:bth-19370DiVA, id: diva2:1422849
Conference
13th International Conference on Signal Processing and Communication Systems, ICSPCS 2019; Gould Coast; Australia; 16 December 2019 through 18 December 2019
Part of project
VIATECH- Human-Centered Computing for Novel Visual and Interactive Applications, Knowledge Foundation
Funder
Knowledge Foundation, 201700562020-04-092020-04-092021-12-22Bibliographically approved