Change search
Link to record
Permanent link

Direct link
Publications (10 of 13) Show all publications
Elwardy, M., Zepernick, H.-J., Hu, Y. & Chu, T. M. (2023). ACR360: A Dataset on Subjective 360° Video Quality Assessment Using ACR Methods. In: Wysocki B.J., Wysocki T.A. (Ed.), 2023 16th International Conference on Signal Processing and Communication System, ICSPCS 2023 - Proceedings: . Paper presented at 16th International Conference on Signal Processing and Communication System, ICSPCS 2023, Bydgoszcz, 6 Sept - 8 Sept 2023. Institute of Electrical and Electronics Engineers (IEEE)
Open this publication in new window or tab >>ACR360: A Dataset on Subjective 360° Video Quality Assessment Using ACR Methods
2023 (English)In: 2023 16th International Conference on Signal Processing and Communication System, ICSPCS 2023 - Proceedings / [ed] Wysocki B.J., Wysocki T.A., Institute of Electrical and Electronics Engineers (IEEE), 2023Conference paper, Published paper (Refereed)
Abstract [en]

The recent advances in immersive technologies have been essential in the development of a wide range of novel standalone and networked immersive media applications. The concepts of virtual reality, augmented reality, and mixed reality relate to different compositions of real and computer-generated virtual objects. In this context, 360° video streaming has become increasingly popular offering improved immersive experiences when viewed on a head-mounted display (HMD). An important component in the development of novel immersive media systems are subjective tests in which participants assess the quality of experience of representative test stimuli. In this paper, the annotated ACR360 dataset is presented which is publicly available on GitHub. The ACR360 dataset contains a wide range of psychophysical and psychophysiological data that was collected in Subjective tests on 360° video quality. The test stimuli were shown on an HMD and rated according to the absolute category rating (ACR) and modified ACR (MACR) methods. To support an easy exploration and utilization of the ACR360 dataset by the research community, its structure on GitHub is described and a comprehensive illustration of analysis options are provided for each data category. The ACR360 dataset may be used for conducting meta-analysis in combination with other datasets to improve precision and to pursue research questions that cannot be answered by an individual study. © 2023 IEEE.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2023
Keywords
360° video, absolute category rating, annotated dataset, Immersive media, quality assessment, Subjective test, Helmet mounted displays, Image quality, Mixed reality, Quality of service, Statistical tests, Subjective testing, Video streaming, Absolute category ratings, Annotated datasets, Head-mounted-displays, Immersive technologies, Media application, Video quality, Augmented reality
National Category
Telecommunications
Identifiers
urn:nbn:se:bth-25548 (URN)10.1109/ICSPCS58109.2023.10261151 (DOI)2-s2.0-85174510305 (Scopus ID)9798350333510 (ISBN)
Conference
16th International Conference on Signal Processing and Communication System, ICSPCS 2023, Bydgoszcz, 6 Sept - 8 Sept 2023
Funder
Knowledge Foundation, 20220068Knowledge Foundation, 20170056
Note

  

Available from: 2023-11-07 Created: 2023-11-07 Last updated: 2023-11-08Bibliographically approved
Elwardy, M., Zepernick, H.-J. & Chu, T. M. (2022). On the Number of Subjects Needed for 360° Video Quality Experiments: An SOS Based Analysis. In: International Conference on Quality of Multimedia Experience, QoMEX 2022: . Paper presented at 14th International Conference on Quality of Multimedia Experience, QoMEX 2022, Lippstadt, 5-7 September, 2022.. Institute of Electrical and Electronics Engineers (IEEE)
Open this publication in new window or tab >>On the Number of Subjects Needed for 360° Video Quality Experiments: An SOS Based Analysis
2022 (English)In: International Conference on Quality of Multimedia Experience, QoMEX 2022, Institute of Electrical and Electronics Engineers (IEEE), 2022Conference paper, Published paper (Refereed)
Abstract [en]

Subjective 360° video quality experiments provide the ground truth for developing related systems such that acceptable quality of experience is offered to the users. To obtain statistically meaningful insights from the data recorded in quality experiments, engaging a sufficiently large number of subjects is essential. In contrast to conventional videos, little has been reported on statistical measures that may guide the choice of the number of subjects required for 360° video quality experiments. In this paper, the standard deviation of opinion score (SOS) hypothesis, measuring the diversity among the opinion scores given by a panel of subjects, is used to reveal the number of subjects needed. It is shown that the number of subjects needed varies depending on a threshold on SOS characteristics in terms of diversity among opinion scores. © 2022 IEEE.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2022
Series
International Workshop on Quality of Multimedia Experience (QoMEX), ISSN 2372-7179
Keywords
Systems engineering, Ground truth, Related systems, Standard deviation, Statistical measures, Video quality, Quality of service
National Category
Telecommunications
Identifiers
urn:nbn:se:bth-24015 (URN)10.1109/QoMEX55416.2022.9900890 (DOI)2-s2.0-85141699143 (Scopus ID)9781665487948 (ISBN)
Conference
14th International Conference on Quality of Multimedia Experience, QoMEX 2022, Lippstadt, 5-7 September, 2022.
Funder
Knowledge Foundation, 20170056
Available from: 2022-12-01 Created: 2022-12-01 Last updated: 2023-09-15Bibliographically approved
Elwardy, M., Zepernick, H.-J. & Hu, Y. (2022). SSV360: A Dataset on Subjetive Quality Assessment of 360° Videos for Standing and Seated Viewing on an HMD. In: Proceedings - 2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops, VRW 2022: . Paper presented at 2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops, VRW 2022, Virtual, Online, 12 March 2022 through 16 March 2022 (pp. 189-194). IEEE Computer Society
Open this publication in new window or tab >>SSV360: A Dataset on Subjetive Quality Assessment of 360° Videos for Standing and Seated Viewing on an HMD
2022 (English)In: Proceedings - 2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops, VRW 2022, IEEE Computer Society, 2022, p. 189-194Conference paper, Published paper (Refereed)
Abstract [en]

In this paper, we introduce the SSV360 dataset containing psy-chophysical and psychophysiological data gathered in a pilot study on the effect of standing and seated viewing of 360° videos on subjective quality assessment. The stimuli were shown to the participants on a head-mounted display for both viewing conditions. The ground truth contained in the SSV360 dataset may support, e.g., benchmarking of immersive media processing algorithms and developing of objective perceptual quality metrics. © 2022 IEEE.

Place, publisher, year, edition, pages
IEEE Computer Society, 2022
Keywords
Human-centered computing-Computing methodologies-Virtual reality-Dataset, Benchmarking, Helmet mounted displays, Computing methodologies, Ground truth, Head-mounted-displays, Human-centered computing, Human-centered computing-computing methodology-virtual reality-dataset, Immersive media, Pilot studies, Quality assessment, Subjective quality assessments, Viewing conditions, Virtual reality
National Category
Computer Sciences
Identifiers
urn:nbn:se:bth-22955 (URN)10.1109/VRW55335.2022.00047 (DOI)000808111800038 ()2-s2.0-85129677249 (Scopus ID)9781665484022 (ISBN)
Conference
2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops, VRW 2022, Virtual, Online, 12 March 2022 through 16 March 2022
Available from: 2022-05-20 Created: 2022-05-20 Last updated: 2022-08-11Bibliographically approved
Elwardy, M., Zepernick, H.-J. & Hu, Y. (2021). On Head Movements in Repeated 360 degrees Video Quality Assessment for Standing and Seated Viewing on Head Mounted Displays. In: 2021 IEEE CONFERENCE ON VIRTUAL REALITY AND 3D USER INTERFACES ABSTRACTS AND WORKSHOPS (VRW 2021): . Paper presented at 28th IEEE Conference on Virtual Reality and 3D User Interfaces (IEEE VR) (pp. 71-74). IEEE (28th IEEE Conference on Virtual Reality and 3D User Interfaces (IEEE VR))
Open this publication in new window or tab >>On Head Movements in Repeated 360 degrees Video Quality Assessment for Standing and Seated Viewing on Head Mounted Displays
2021 (English)In: 2021 IEEE CONFERENCE ON VIRTUAL REALITY AND 3D USER INTERFACES ABSTRACTS AND WORKSHOPS (VRW 2021), IEEE, 2021, no 28th IEEE Conference on Virtual Reality and 3D User Interfaces (IEEE VR), p. 71-74Conference paper, Published paper (Refereed)
Abstract [en]

Watching 360 degrees videos on head mounted displays (HMDs) allows viewers to explore scenes in all directions. In this paper, we focus on investigating the head movements of two participants for standing and seated viewing of a total of 720 360 degrees videos on HMDs. The head movements were recorded in a 360 degrees video quality assessment experiment which was repeated after a long and short break between sessions to study changes in viewing behavior over time. The analysis of the head movement data is provided as histograms of head rotations, head speed, head turns, and head trajectories. It is shown that the participants have their own distinct exploration behavior for standing viewing which becomes less different for seated viewing.

Place, publisher, year, edition, pages
IEEE, 2021
Keywords
Human-centered computing, Computing methodologies, Virtual reality, Head Movements
National Category
Telecommunications
Identifiers
urn:nbn:se:bth-22101 (URN)10.1109/VRW52623.2021.00020 (DOI)000676157600013 ()978-1-6654-4057-8 (ISBN)
Conference
28th IEEE Conference on Virtual Reality and 3D User Interfaces (IEEE VR)
Note

open access

Available from: 2021-09-03 Created: 2021-09-03 Last updated: 2021-11-25Bibliographically approved
Hu, Y., Elwardy, M. & Zepernick, H.-J. (2021). On the Effect of Standing and Seated Viewing of 360 degrees Videos on Subjective Quality Assessment: A Pilot Study. Computers, 10(6)
Open this publication in new window or tab >>On the Effect of Standing and Seated Viewing of 360 degrees Videos on Subjective Quality Assessment: A Pilot Study
2021 (English)In: Computers, E-ISSN 2073-431X, Vol. 10, no 6Article in journal (Refereed) Published
Abstract [en]

Due to the advances in head-mounted displays (HMDs), hardware and software technologies, and mobile connectivity, virtual reality (VR) applications such as viewing 360 degrees videos on HMDs have seen an increased interest in a wide range of consumer and vertical markets. Quality assessment of digital media systems and services related to immersive visual stimuli has been one of the challenging problems of multimedia signal processing. Specifically, subjective quality assessment of 360 degrees videos presented on HMDs is needed to obtain a ground truth on the visual quality as perceived by humans. Standardized test methodologies to assess the subjective quality of 360 degrees videos on HMDs are currently not as developed as for conventional videos and are subject to further study. In addition, subjective tests related to quality assessment of 360 degrees videos are commonly conducted with participants seated on a chair but neglect other options of consumption such as standing viewing. In this paper, we compare the effect that standing and seated viewing of 360 degrees videos on an HMD has on subjective quality assessment. A pilot study was conducted to obtain psychophysical and psychophysiological data that covers explicit and implicit responses of the participants to the shown 360 degrees video stimuli with different quality levels. The statistical analysis of the data gathered in the pilot study is reported in terms of average rating times, mean opinion scores, standard deviation of opinion scores, head movements, pupil diameter, galvanic skin response (GSR), and simulator sickness scores. The results indicate that the average rating times consumed for 360 degrees video quality assessment are similar for standing and seated viewing. Further, the participants showed higher resolving power among different 360 degrees video quality levels and were more confident about the given opinion scores for seated viewing. On the other hand, a larger scene exploration of 360 degrees videos was observed for standing viewing which appears to distract from the quality assessment task. A slightly higher pupil dilation was recorded for standing viewing which suggests a slightly more immersed experience compared to seated viewing. GSR data indicate a lower degree of emotional arousal in seated viewing which seems to allow the participants to better conduct the quality assessment task. Similarly, simulator sickness symptoms are kept significantly lower when seated. The pilot study also contributes to a holistic view of subjective quality assessment and provides indicative ground truth that can guide the design of large-scale subjective tests.

Place, publisher, year, edition, pages
MDPI, 2021
Keywords
human-centered computing, computing methodologies, virtual reality, 360 degrees videos, standing viewing, seated viewing, perception, subjective tests, quality assessment
National Category
Telecommunications
Identifiers
urn:nbn:se:bth-21983 (URN)10.3390/computers10060080 (DOI)000665437300001 ()
Funder
Knowledge Foundation, 20170056
Available from: 2021-07-08 Created: 2021-07-08 Last updated: 2023-01-06Bibliographically approved
Elwardy, M., Zepernick, H.-J., Chu, T. M. & Hu, Y. (2021). On the Opinion Score Consistency in Repeated 360° Video Quality Assessment for Standing and Seated Viewing on Head-Mounted Displays. In: Wysocki T.A., Wysocki B.J. (Ed.), 2021 15th International Conference on Signal Processing and Communication Systems, ICSPCS 2021 - Proceedings: . Paper presented at 15th International Conference on Signal Processing and Communication Systems, ICSPCS 2021, Virtual, Online, 13 December 2021 through 15 December 2021. Institute of Electrical and Electronics Engineers Inc.
Open this publication in new window or tab >>On the Opinion Score Consistency in Repeated 360° Video Quality Assessment for Standing and Seated Viewing on Head-Mounted Displays
2021 (English)In: 2021 15th International Conference on Signal Processing and Communication Systems, ICSPCS 2021 - Proceedings / [ed] Wysocki T.A., Wysocki B.J., Institute of Electrical and Electronics Engineers Inc. , 2021Conference paper, Published paper (Refereed)
Abstract [en]

The development of immersive media systems and services relies on subjective tests that provide ground truth on the quality of experience as perceived by humans. In this paper, we investigate the opinion score consistency through a repeated subjective 360° video quality assessment experiment. The test stimuli are presented on a head-mounted display with participants given the task to rate the video quality in three repeated sessions for both standing and seated viewing. The statistical analysis of the data from the subjective tests aims at revealing if participants change their rating behavior over time or keep their opinion scores given to the test stimuli in standing and seated viewing consistent in a statistical sense. The experimental results are reported in terms of histograms of opinion scores, skewness and kurtosis of opinion scores, mean opinion scores, standard deviation of opinion scores (SOS), SOS fitting functions, and analysis of variance tests. The statistical analysis supports the conjecture that each participant has its own but consistent rating behavior throughout the respective sessions for standing and seated viewing. The finding that the quality rating behavior of the individual participant does not fundamentally change over time may, e.g., assist in scheduling subjective tests under pandemic conditions where experimental campaigns may need to be stalled for an unknown period of time. © 2021 IEEE.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers Inc., 2021
Keywords
360° videos, head-mounted display, mean opinion score, seated viewing, standing viewing, subjective tests, Helmet mounted displays, Quality of service, Systems engineering, 360° video, Head-mounted-displays, Immersive media, Mean opinion scores, Media systems, Quality assessment, Subjective test, Video quality, Higher order statistics
National Category
Telecommunications
Identifiers
urn:nbn:se:bth-22674 (URN)10.1109/ICSPCS53099.2021.9660331 (DOI)000802223300039 ()2-s2.0-85124628339 (Scopus ID)9781665436991 (ISBN)
Conference
15th International Conference on Signal Processing and Communication Systems, ICSPCS 2021, Virtual, Online, 13 December 2021 through 15 December 2021
Funder
Knowledge Foundation, 20170056
Note

open access

Available from: 2022-02-25 Created: 2022-02-25 Last updated: 2022-07-01Bibliographically approved
Elwardy, M., Hu, Y., Zepernick, H.-J., Chu, T. M. & Sundstedt, V. (2020). Comparison of ACR Methods for 360° Video Quality Assessment Subject to Participants' Experience with Immersive Media. In: Wysocki T.A.,Wysocki B.J. (Ed.), 2020 14th International Conference on Signal Processing and Communication Systems, ICSPCS 2020 - Proceedings: . Paper presented at 14th International Conference on Signal Processing and Communication Systems, ICSPCS 2020, Virtual, Adelaide, Australia, 14 December 2020 through 16 December 2020. Institute of Electrical and Electronics Engineers Inc., Article ID 9310071.
Open this publication in new window or tab >>Comparison of ACR Methods for 360° Video Quality Assessment Subject to Participants' Experience with Immersive Media
Show others...
2020 (English)In: 2020 14th International Conference on Signal Processing and Communication Systems, ICSPCS 2020 - Proceedings / [ed] Wysocki T.A.,Wysocki B.J., Institute of Electrical and Electronics Engineers Inc. , 2020, article id 9310071Conference paper, Published paper (Refereed)
Abstract [en]

Watching 360° videos on head-mounted displays (HMDs) has become more popular in recent years. As such, subjective quality assessment of 360° videos on HMDs is needed to obtain a ground truth on the quality as perceived by users. Due to the lack of standardized methods, the absolute category rating (ACR) with hidden reference (ACR-HR) method for conventional videos has also been used for subjective quality assessment of 360° videos on HMDs. A modified ACR (M-ACR) method tailored for assessing 360° videos on HMDs has recently been proposed. In this paper, we compare the ACR-HR and M-ACR with hidden reference (M-ACR-HR) method regarding subjective quality assessment of 360° videos on HMDs. The comparison is conducted subject to participants' experience with watching immersive media on HMDs. The findings include: 1) Average rating times to cast opinion scores is lower for the ACR-HR method irrespective of participants' experience, 2) Mean opinion scores are similar for both methods irrespective of participants' experience, 3) The M-ACR-HR method is more reliable, especially with experts and participants with no experience, 4) Discomfort is significantly lower for the ACR-HR method, especially, for participants with no experience. © 2020 IEEE.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers Inc., 2020
Keywords
Helmet mounted displays, Absolute category ratings, Ground truth, Head mounted displays, Immersive media, Mean opinion scores, Standardized methods, Subjective quality assessments, Video quality assessment, Signal processing
National Category
Telecommunications
Identifiers
urn:nbn:se:bth-21062 (URN)10.1109/ICSPCS50536.2020.9310071 (DOI)2-s2.0-85100031773 (Scopus ID)9781728199726 (ISBN)
Conference
14th International Conference on Signal Processing and Communication Systems, ICSPCS 2020, Virtual, Adelaide, Australia, 14 December 2020 through 16 December 2020
Funder
Knowledge Foundation, 2018/863,20170056
Note

open access

Available from: 2021-02-12 Created: 2021-02-12 Last updated: 2021-07-02Bibliographically approved
Elwardy, M., Zepernick, H.-J., Hu, Y., Chu, T. M. & Sundstedt, V. (2020). Evaluation of Simulator Sickness for 360 degrees Videos on an HMD Subject to Participants' Experience with Virtual Reality. In: IEEE CONFERENCE ON VIRTUAL REALITY AND 3D USER INTERFACES WORKSHOPS (VRW 2020): . Paper presented at 27th IEEE Conference on Virtual Reality and 3D User Interfaces, VRW 2020, Atlanta, United States, 22 March 2020 through 26 March 2020 (pp. 477-484). Institute of Electrical and Electronics Engineers Inc., Article ID 9090490.
Open this publication in new window or tab >>Evaluation of Simulator Sickness for 360 degrees Videos on an HMD Subject to Participants' Experience with Virtual Reality
Show others...
2020 (English)In: IEEE CONFERENCE ON VIRTUAL REALITY AND 3D USER INTERFACES WORKSHOPS (VRW 2020), Institute of Electrical and Electronics Engineers Inc. , 2020, p. 477-484, article id 9090490Conference paper, Published paper (Refereed)
Abstract [en]

Virtual reality (VR) has seen tremendous advances in head-mounted displays (HMDs), optics, media quality, and other improvements that facilitate immersive experiences. With the occurrence of new technologies like Cloud VR and networked VR video services, applications such as 360° video streaming are becoming more popular within the broader consumer markets. As a result, VR content is accessible to customers with rather different levels of experiences with immersive media, i.e., never, sometimes, or often use of VR. The question, therefore, arises to which degree simulator sickness is induced to viewers depending on their experiences with VR on HMDs. In this paper, simulator sickness is evaluated for 360° videos that were shown on an HTC Vive Pro HMD to participants having different levels of experience with VR on HMDs. The modified absolute category rating with hidden reference (M-ACR-HR) method was used in a subjective experiment for video quality assessment within two subsequent sessions along with a simulator sickness questionnaire (SSQ). A statistical analysis of the SSQ scores is performed to reveal the relationship between simulator sickness and participants' experiences with VR regarding: (1) Individual symptoms, (2) Pairwise comparison of symptoms, and (3) Symptom clusters of nausea, oculomotor, disorientation, and total score. It is shown that the simulator sickness symptoms, in general, are slightly or rarely perceived across the different experience levels for the selected 360° videos. The results indicate that the reported simulator sickness increases in the second session for participants that never used VR on HMDs. Sufficiently long breaks between sessions should therefore be accounted for in the M-ACR-HR method to avoid that simulator sickness influences quality rating. © 2020 IEEE.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers Inc., 2020
Keywords
360 videos, 360° videos, Immersive media, M-ACR-HR method, simulator sickness questionnaire, subjective experiments, Diseases, Helmet mounted displays, Simulators, User interfaces, Absolute category ratings, Head mounted displays, Pair-wise comparison, Quality ratings, Simulator sickness, Video quality assessment, Virtual reality
National Category
Human Computer Interaction Telecommunications
Identifiers
urn:nbn:se:bth-19572 (URN)10.1109/VRW50115.2020.00100 (DOI)000578054300093 ()2-s2.0-85085391320 (Scopus ID)9781728165325 (ISBN)
Conference
27th IEEE Conference on Virtual Reality and 3D User Interfaces, VRW 2020, Atlanta, United States, 22 March 2020 through 26 March 2020
Funder
Knowledge Foundation, 20170056
Available from: 2020-06-05 Created: 2020-06-05 Last updated: 2021-07-02Bibliographically approved
Hu, Y., Elwardy, M. & Zepernick, H.-J. (2020). On the Effect of Standing and Seated Viewing of 360 degrees Videos on Subjective Quality Assessment. In: 2020 IEEE CONFERENCE ON VIRTUAL REALITY AND 3D USER INTERFACES WORKSHOPS (VRW 2020): . Paper presented at 27th IEEE Conference on Virtual Reality and 3D User Interfaces (IEEE VR), Atlanta, MAR 22-26 (pp. 285-286). IEEE COMPUTER SOC
Open this publication in new window or tab >>On the Effect of Standing and Seated Viewing of 360 degrees Videos on Subjective Quality Assessment
2020 (English)In: 2020 IEEE CONFERENCE ON VIRTUAL REALITY AND 3D USER INTERFACES WORKSHOPS (VRW 2020), IEEE COMPUTER SOC , 2020, p. 285-286Conference paper, Published paper (Refereed)
Abstract [en]

In this paper, we compare the impact that standing and seated viewing of 360 degrees videos on head-mounted displays has on subjective quality assessment. The statistical analysis of the data gathered in a pilot study is reported in terms of average rating times, mean opinion scores, and simulator sickness scores. The results indicate: (1) Average rating times consumed for 360 degrees video quality assessment are similar for standing and seated viewing, (2) Higher resolving power among different quality levels is obtained for seated viewing, (3) Simulator sickness is kept significantly lower when seated.

Place, publisher, year, edition, pages
IEEE COMPUTER SOC, 2020
Keywords
Human-centered computing, Computing methodologies, Virtual reality, Perception
National Category
Telecommunications Communication Systems
Identifiers
urn:nbn:se:bth-20718 (URN)10.1109/VRW50115.2020.00061 (DOI)000578054300054 ()
Conference
27th IEEE Conference on Virtual Reality and 3D User Interfaces (IEEE VR), Atlanta, MAR 22-26
Available from: 2020-11-13 Created: 2020-11-13 Last updated: 2021-05-21Bibliographically approved
Chu, T. M., Zepernick, H.-J. & Elwardy, M. (2020). Rating Duration Analysis for Subjective Quality Assessment of 360° Videos. In: Proceedings - 2020 International Conference on Virtual Reality and Visualization, ICVRV 2020: . Paper presented at 2020 International Conference on Virtual Reality and Visualization, ICVRV 2020, Recife, 13 November 2020 through 14 November 2020 (pp. 42-46). Institute of Electrical and Electronics Engineers Inc.
Open this publication in new window or tab >>Rating Duration Analysis for Subjective Quality Assessment of 360° Videos
2020 (English)In: Proceedings - 2020 International Conference on Virtual Reality and Visualization, ICVRV 2020, Institute of Electrical and Electronics Engineers Inc. , 2020, p. 42-46Conference paper, Published paper (Refereed)
Abstract [en]

Due to large advances in immersive media systems offering increased resolutions, frame rates, and dynamic ranges, applications such as watching 360° videos on head-mounted displays (HMDs) have become more popular. To assist the development of immersive media applications, subjective tests of 360° videos on HMDs are needed to obtain a ground truth on the quality as perceived by the end-users. The absolute category rating (ACR) method, standardized for subjective quality assessment of conventional videos, has also been used for assessing the subjective quality of 360° videos. Recently, the modified ACR (M-ACR) method has been proposed aiming at subjective quality assessment of 360° videos. Several subjective test campaigns have been conducted to assess the ACR and M-ACR method for evaluating 360° videos covering a wide range of resolutions, bit rates, and quantization parameters. The opinion scores given by the participants in these subjective tests are accumulated to mean option scores (MOSs) and confidence intervals. However, to the best of our knowledge, a detailed statistical analysis and comparison of the rating durations associated with casting opinion scores for 360° test videos in the ACR and M-ACR method has not been reported in the literature. In this paper, we therefore provide a rating duration analysis for subjective quality assessment of 360° videos for the ACR and M-ACR method. The results support experimental designs for 360° video quality assessment and include the following findings: 1) The mean rating duration is shorter for the ACR method, 2) The difference of the standard deviation of rating durations versus MOS between the ACR and M-ACR method is insignificant, 3) For a given mean rating duration, however, the standard deviation is smaller for the M-ACR method indicating that the quality rating task is easier to execute than with ACR. 4) The period to be allocated in subjective tests for rating 360° videos should exceed 5 s but can be kept below or set equal to 10 s. © 2020 IEEE.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers Inc., 2020
Keywords
Human-centered computing-Computing methodology-Perception, Human-centered computing-Computing methodology-Virtual reality, Helmet mounted displays, Statistics, Virtual reality, Visualization, Absolute category ratings, Confidence interval, Head mounted displays, Quantization parameters, Standard deviation, Subjective quality, Subjective quality assessments, Video quality assessment, Quality control
National Category
Telecommunications
Identifiers
urn:nbn:se:bth-22012 (URN)10.1109/ICVRV51359.2020.00020 (DOI)2-s2.0-85111461406 (Scopus ID)9780738142524 (ISBN)
Conference
2020 International Conference on Virtual Reality and Visualization, ICVRV 2020, Recife, 13 November 2020 through 14 November 2020
Funder
Knowledge Foundation, 20170056
Available from: 2021-08-13 Created: 2021-08-13 Last updated: 2021-08-13Bibliographically approved
Organisations
Identifiers
ORCID iD: ORCID iD iconorcid.org/0000-0002-7550-5818

Search in DiVA

Show all publications