Rating Duration Analysis for Subjective Quality Assessment of 360° Videos
2020 (English)In: Proceedings - 2020 International Conference on Virtual Reality and Visualization, ICVRV 2020, Institute of Electrical and Electronics Engineers Inc. , 2020, p. 42-46Conference paper, Published paper (Refereed)
Abstract [en]
Due to large advances in immersive media systems offering increased resolutions, frame rates, and dynamic ranges, applications such as watching 360° videos on head-mounted displays (HMDs) have become more popular. To assist the development of immersive media applications, subjective tests of 360° videos on HMDs are needed to obtain a ground truth on the quality as perceived by the end-users. The absolute category rating (ACR) method, standardized for subjective quality assessment of conventional videos, has also been used for assessing the subjective quality of 360° videos. Recently, the modified ACR (M-ACR) method has been proposed aiming at subjective quality assessment of 360° videos. Several subjective test campaigns have been conducted to assess the ACR and M-ACR method for evaluating 360° videos covering a wide range of resolutions, bit rates, and quantization parameters. The opinion scores given by the participants in these subjective tests are accumulated to mean option scores (MOSs) and confidence intervals. However, to the best of our knowledge, a detailed statistical analysis and comparison of the rating durations associated with casting opinion scores for 360° test videos in the ACR and M-ACR method has not been reported in the literature. In this paper, we therefore provide a rating duration analysis for subjective quality assessment of 360° videos for the ACR and M-ACR method. The results support experimental designs for 360° video quality assessment and include the following findings: 1) The mean rating duration is shorter for the ACR method, 2) The difference of the standard deviation of rating durations versus MOS between the ACR and M-ACR method is insignificant, 3) For a given mean rating duration, however, the standard deviation is smaller for the M-ACR method indicating that the quality rating task is easier to execute than with ACR. 4) The period to be allocated in subjective tests for rating 360° videos should exceed 5 s but can be kept below or set equal to 10 s. © 2020 IEEE.
Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers Inc. , 2020. p. 42-46
Keywords [en]
Human-centered computing-Computing methodology-Perception, Human-centered computing-Computing methodology-Virtual reality, Helmet mounted displays, Statistics, Virtual reality, Visualization, Absolute category ratings, Confidence interval, Head mounted displays, Quantization parameters, Standard deviation, Subjective quality, Subjective quality assessments, Video quality assessment, Quality control
National Category
Telecommunications
Identifiers
URN: urn:nbn:se:bth-22012DOI: 10.1109/ICVRV51359.2020.00020Scopus ID: 2-s2.0-85111461406ISBN: 9780738142524 (print)OAI: oai:DiVA.org:bth-22012DiVA, id: diva2:1584693
Conference
2020 International Conference on Virtual Reality and Visualization, ICVRV 2020, Recife, 13 November 2020 through 14 November 2020
Part of project
VIATECH- Human-Centered Computing for Novel Visual and Interactive Applications, Knowledge Foundation
Funder
Knowledge Foundation, 201700562021-08-132021-08-132021-08-13Bibliographically approved