|
Work item:
|
P.MOSQUITO
|
|
Subject/title:
|
Subjective test methodology for assessing impact of adaptive streaming effects on longer-term Quality of Experience (MOS) and quitting (for objective model development)
|
|
Status:
|
[Carried to next study period]
|
|
Approval process:
|
AAP
|
|
Type of work item:
|
Recommendation
|
|
Version:
|
New
|
|
Equivalent number:
|
-
|
|
Timing:
|
-
|
|
Liaison:
|
-
|
|
Supporting members:
|
TU Ilmenau, NTT, Ericsson, OPTICOM, Rohde & Schwarz
|
|
Summary:
|
Initial loading delay, stalling, video- or audio-quality issues or switches during video streaming are known to impact the QoE of end-users. However, the specific relationship between these effects and the users’ likelihood to abort a playback – or the perceived overall quality of the entire media session – vary for different services. Furthermore, QoE and a corresponding aborting behavior depend on the expectations of the users. It is important for Internet Service Providers to be able to assess the impact of these factors on user QoE and aborting, as numbers quantifying these relationships in a standardized manner are not readily available.
In this recommendation, a procedure for conducting studies with human test subjects is described. These studies are called behavioral, since interaction with a (simulated) video streaming service is the main focus. Results from a study conducted following this recommendation may be used for several purposes, including but not limited to developing prediction models that estimate:
a. the likelihood of users abandoning a video playback, and
b. the perceived quality of the viewing session, and optionally
c. the likely reasons for users to abandon the viewing session.
The test procedure specified in this recommendation aims to bridge the gap between traditional video quality assessment methods such as specified in ITU-T Rec. P.910 and large-scale data collections that Over-the-top video providers may conduct. The method is based on ITU-T Rec. P.917, which is extended beyond initial loading delay and amended by a modified way of rating subjectively perceived integral quality at the end of a sequence.
First, background and related work on tests aimed at assessing the impact of adaptive-streaming-type effects on longer-sequence (up to 5min) viewing experience are described. A general overview of factors that influence user behavior and subsequent ratings is given. Then, the document describes a test procedure, which is to be conducted in a controlled environment, be it in a lab or in a crowdsourcing context with subjects in an appropriate environment.
|
|
Comment:
|
-
|
|
Reference(s):
|
|
|
Historic references:
|
|
Contact(s):
|
|
| ITU-T A.5 justification(s): |
|
|
|
|
First registration in the WP:
2021-05-18 09:59:36
|
|
Last update:
2021-05-18 10:27:05
|
|
