​​Executive summary: Meeting of ITU-T Study Group 12 (Performance, Quality of Service, Quality of Experience), Geneva, 19-28 September 2017

Note: This is not an official record and is subject to correction or modification

Overview

ITU-T Study Group 12 held its second meeting of the study period in Geneva, 19-28 September 2017. The meeting was attended by 119 participants from 43 countries (see list of participants​​). 14 delegates used the remote participation facilities.

The meeting received 81 Contributions and handled 217 TDs. Delegates reviewed 60 incoming Liaison statements and adopted 23 outgoing Liaison statements​​. 

Ten Recommendations were consented (3 new, 6 revised, 1 Corrigendum), and two Supplements were agreed (1 new, 1 revised) (see below for details).

A webinar (announcement | recording | slides) was held on 28 September 2017 following the closing plenary, summarizing key meeting results. Photos of the SG12 opening plenary can be found here​.

Major accomplishments

Completed work

The meeting consented new ITU-T Recommendations on

(1) ​ Subjective diagnostic test method for conversational speech quality analysis (will be known as ITU-T P.804); 
​(2)  Customer experience management index for popular services in operators' network to score service quality that customers experience in terms of key network performance parameters (ITU-T E.831);
(3)  Influence factors on gaming quality of experience (ITU-T G.1032).

E.831 was initially introduced to SG12's Quality of Service Development Group (QSDG) before being elevated to Study Group level and includes an example of the use of the index in an operator's network.

The meeting consented revisions of existing ITU-T Recommendations on

(4)  Vocabulary for performance and quality of service (ITU-T P.10 / G.100);
(5)  Subjective quality evaluation of audio and audiovisual multiparty telemeetings (ITU-T P.1301);
(6)  Parametric bitstream-based quality assessment of progressive download and adaptive audiovisual streaming services over reliable transport (ITU-T P.1203);
(7)  Parametric bitstream-based quality assessment of progressive download and adaptive audiovisual streaming services over reliable transport - Video quality estimation module (ITU-T P.1203.1);
(8)  Parametric bitstream-based quality assessment of progressive download and adaptive audiovisual streaming services over reliable transport - Audio quality estimation module (ITU-T P.1203.2);
(9)  Parametric bitstream-based quality assessment of progressive download and adaptive audiovisual streaming services over reliable transport - Quality integration module (ITU-T P.1203.3).​​

​The revisions to the Recommendations in the P.1203 family fix ambiguities in descriptions that prevent operators from correctly implementing the respective Recommendations, and increase readability.

SG12 also consented a Corrigendum to an existing ITU-T Recommendation on

(10)  Wideband extension to Recommendation P.862 for the assessment of wideband telephone networks and speech codecs (ITU-T P.862.2).

​The Corrigendum addresses a systematic under-prediction of P.862.2 caused by the incorrect transformation of the sound pressure level.

SG12 agreed on two non-normative texts:​
(1) ​ Revised Supplement 26 to ITU-T P series of Recommendations: Scenarios for the subjective quality evaluation of audio and audiovisual multiparty telemeetings;
(2)​ ​ New Supplement to ITU-T G series of Recommendations: IP aware QoS management.

New work items

SG12 established new work items on

1. Voice service quality monitoring and troubleshooting framework for intrusive parametric voice QoE prediction (P.VSQMTF);
2. Considerations for realizing virtual measurement systems (Y.cvms);
3. The effect of SIM-boxing on QoS and QoE (E.QSIMbox);
4. Amendment 2 to Recommendation ITU-T E.802 (E.802Amd2);
5. QoS management model for bridging the gaps between QoS and QoE (E.QoSMgtMod);
6. Amendment 1 to Recommendation ITU-T Y.1545.1 (Y.1545.1Amd1);
7. Assessment of the LTE circuit switched fall back  - impact on QoE and QoS (G.CSFB);
8. Measurement scenarios, advanced measurement systems and sampling methodologies to monitor the QoS in mobile networks (E.MTSM);
9. QoE-based network capacity planning (G.NCP);
10. Objective model for assessing impact of initial loading delay on user experience (P.QUIT);
11. Subjective test methodology for assessing impact of initial loading delay on user experience (P.QUITS); and
12. QoS and QoE aspects of digital financial services (G.DFS).

Work items E.802Amd2, E.QoSMgtMod, G.CSFB, E.MTSM and Y.1545.1 are related to the mandate given to SG12 by WTSA Resolution 95. E.QSIMbox is related to WTSA Resolution 29, and G.DFS related to WTSA Resolution 89.

WTSA matters

SG12's operational plan for implementation of WTSA Resolution 95 (ITU-T initiatives to raise awareness on best practices and policies related to service quality) was approved, as updated in TD380. The meeting requested TSB to disseminate a questionnaire on service quality regulatory frameworks contained in TD347R1 via TSB Circular. SG12 approved a liaison to ITU-T Study Groups 2 and 3 on matters related to alternative calling procedures (WTSA Resolution 29) and SIM-boxing.

Other activities

SG12 released the ITU-T Software Tool Library (Recommendation ITU-T G.191) on GitHub, enabling the wider community to use, improve and contribute to the code.

A computer lab was set up in the basement of the ITU Montbrillant building to advance the practical work on P.NATS phase 2: Parametric bitstream-based and pixel-based quality assessment of progressive download and adaptive audiovisual streaming services over reliable transport for HEVC and UHD.

SG12 approved working methods enabling it to collaborate with ETSI TC STQ WG MOBILE in selected areas of interest, e.g., update and maintenance of the ETSI TS 102 250 series and Recommendation ITU-T E.804; QoS and QoE aspects of digital financial services.

ITU/BDT colleagues joined SG12 on two occasions to introduce the QoS Regulation Manual and the QoS Training Programme, both developed with the support of SG12 delegates.
​
Nineteen delegates from developing countries participated in a bridging the standardization gap hands-on training session.

SG12 in the news

• ITU News: Contribute to open-source ITU speech coding tools on GitHub
• ITU News: Webinar, Thursday 28 September, 15:45 CEST: New ITU standards for quality of video gaming, streamed media and speech quality
• Flickr: ITU-T SG12 Opening​

Results by group directly reporting to Plenary

Q1/12

• Coordination on WTSA Resolution 95 (Hammamet, 2016)
• Approved working methods to collaborate with ETSI TC STQ WG Mobile
• Approved eight outgoing liaison statements

Q2/12

• Launch of STL on GitHub
• Produced the following Recommendations for consent in plenary:
• Revised Recommendation ITU-T P.10 / G.100: Vocabulary for performance and quality of service
• Adopted one outgoing liaison statement

SG12 Regional Group on QoS for the Africa Region (SG12RG-AFR)

• Reviewed its action plan
• Coordinated African contributions to SG12

Quality of Service Development Group (QSDG)

• Reviewed and updated its work programme for 2017-18

Results by working party

Working Party 1/12

• Produced the following Recommendations for consent in plenary:
• New Recommendation ITU-T P.804: Subjective diagnostic test method for conversational speech quality analysis;
• Revised Recommendation ITU-T P.1301: Subjective quality evaluation of audio and audiovisual multiparty telemeetings.
• Produced the following non-normative texts for agreement in plenary:
• Revised Supplement 26 to ITU-T P series of Recommendations: Scenarios for the subjective quality evaluation of audio and audiovisual multiparty telemeetings
• Adopted one outgoing liaison statement

Working Party 2/12

• Produced the following Recommendations for consent in plenary:
• Corrigendum 1 to Recommendation ITU-T P.862.2: Wideband extension to Recommendation P.862 for the assessment of wideband telephone networks and speech codecs;
• Revised Recommendation ITU-T P.1203: Parametric bitstream-based quality assessment of progressive download and adaptive audiovisual streaming services over reliable transport;
• Revised Recommendation ITU-T P.1203.1: Parametric bitstream-based quality assessment of progressive download and adaptive audiovisual streaming services over reliable transport - Video quality estimation module;
• Revised Recommendation ITU-T P.1203.2: Parametric bitstream-based quality assessment of progressive download and adaptive audiovisual streaming services over reliable transport - Audio quality estimation module;
• Revised Recommendation ITU-T P.1203.3: Parametric bitstream-based quality assessment of progressive download and adaptive audiovisual streaming services over reliable transport - Quality integration module.
• Established new work item P.VSQMTF
• Adopted five outgoing liaison statements

Working Party 3/12

• Produced the following Recommendations for consent in plenary:
• New Recommendation ITU-T E.831: Customer experience management index for popular services in operators' network to score service quality that customers experience in terms of key network performance parameters;
• New Recommendation ITU-T G.1032: Influence factors on gaming quality of experience.
• Produced the following non-normative texts for agreement in plenary:
• New Supplement to ITU-T G series of Recommendations: IP aware QoS management
• Established new work items Y.cvms, E.QSIMBox, E.802Amd2, E.QoSMgtMod, Y.1545.1Amd1, G.CSFB, E.MTSM, G.NCP, P.QUIT, P.QUITS, G.DFS
• Adopted eight outgoing liaison statements
• Adopted a Questionnaire on service quality regulatory frameworks

Future meetings and workshops

Meetings of SG12

• 1-10 May 2018, Geneva, Switzerland
• 27 November - 6 December 2018, Geneva, Switzerland

Meetings of Working parties under SG12

• WP2/12: 15 February 2018, Geneva, Switzerland (ITU)

Meetings of Rapporteur groups

• Q4/12: 23-24 January 2018, Novi, Michigan, United States (BlackBerry)
• Q9/12: 14-15 February 2018, Geneva, Switzerland (ITU)
• Q12/12: TBC 12-16 March 2018, Dakar, Senegal (jointly with SG12RG-AFR)
• Qs 13 and 14/12: 27-29 November 2017, Krakow, Poland (AGH University) (jointly with VQEG)
• Qs 13 and 14/12: 26-28 February 2018, TBC: Stockholm/Berlin (Ericsson/T-Labs)
• Q17/12: TBC 27-28 February 2018, TBC: Stockholm/Berlin (Ericsson/T-Labs) (with Qs 13 and 14/12)

Meeting of SG12 Regional Group on QoS for the Africa Region (SG12RG-AFR)

• TBC: 12-16 March 2018, Dakar, Senegal, with workshop and Q12/12 rapporteur group meeting

Meeting of Quality of Service Development Group (QSDG)

• TBC: September 2018, hosted by Türk Telekom

SG12 Workshops

• ITU Workshop on Telecommunications Service Quality, 27-29 November 2017, Rio de Janeiro, Brazil​​
• TBC: 12-16 March 2018, Dakar, Senegal (with SG12RG-AFR and Q12/12)
• TBC: September 2018, Istanbul, Turkey (with QSDG)