Overview of ITU-T Study Group 12 Activities on QoS and QoE

1. Overview of ITU-T Study Group 12 Activities on QoS and QoE Joachim Pomy, SG 12 Rapporteur Consultant, Opticom GmbH Consultant@joachimpomy.de CITEL (PCC.I)/ ITU Forum on Information and Communication Technology Service: Quality, Control and Surveillance (Cartagena de Indias, Colombia, 23-24 September 2013)

2. Where it All Begins:Real Communication Situation

3. ... and where Technology comes to Play:Employing a Telecommunication System

4. SG 12 Key History Events (1) • Established in 1957 • Separate SG 16 for Transmission Planning • Old SG 12 just did subjective testing & telephone standards • At CCITT Plenary Assembly in 1984, these SGs were merged to form the present-day SG 12 • During same time SG 12 had a Laboratory WP • Subjective & objective testing for loudness of telephones • In the CCITT Laboratory in Geneva - closed in 1988 • Since 1986 Speech Quality Experts Group (SQEG) • To better provide the quality requirements and subjective testing for speech coding algorithms standardized by SG 18 (later SG 15, now SG 16) • Key role in the standardization of all of the G.711-G.729 speech and audio codecs, and still exists as Q7/12.

5. SG 12 Key History Events (2) • Since 1997 Video Quality Experts Group (VQEG) • With a nucleus of subjective and objective video quality expertise from SG12 • SG12 still maintains close contact with VQEG • Since the early 1990s work in SG 12 has expanded • Into many new technology areas • ATM • VoIP • Multimedia • Digital wireless • IP QoS classes • Ethernet • IPTV, etc. • Network Performance • Resource Management. • Since 2008 SG12 was given responsibility for • Operational aspects of telecommunication network service quality • Responsibility for the QoS Development Group (QSDG).

6. SG 12 Mandate • Performance, QoS and QoE • Responsible for Recommendations on performance, quality of service (QoS) and quality of experience (QoE) for • full spectrum of terminals, networks and services • ranging from speech over fixed circuit-based networks • to multimedia applications over networks that are mobile and packet based • Included are • operational aspects of performance, QoS and QoE • end-to-end quality aspects of interoperability • development of multimedia quality assessment methodologies, both subjective and objective. • SG 12 is the Lead SG on • quality of service and quality of experience • driver distraction and voice aspects of car communications • http://www.itu.int/en/ITU-T/studygroups/2013-2016/12/Pages/default.aspx

7. SG 12 Recommendations • E.420-E.479, E.800-E.859 • G.100-series, except G.160-, G.180- and G.190-series • G.1000-series • I.350-series (including Y.1501/G.820/I.351), I.371, I.378, I.381 • P-series, except P.900-series • Y.1220-, Y.1530-, Y.1540-, Y.1560-serieshttp://www.itu.int/ITU-T/recommendations/index_sg.aspx?sg=12

8. SG 12 Leading Team • Chairman • Kwame Baah-Acheamfuor (Ghana) • Vice Chairmen • Paul Barrett (United States) • Vinvent Barriac (France) • Gamal Amin Elsayed (Sudan) • Hyung-Soo Kim (Republic of Korea) • Al Morton (United States) • Qi Feng (China) • José Guadalupe Rojas Ramírez (Mexico) • Akira Takahashi (Japan) • Hassan Talib (Morocco) • TSB Support • Hiroshi Ota, Counsellor • Sarah Scott, Assistant

9. Working Parties • WP 1Terminals and multimedia subjective assessment • CHAIR: Lars Birger Nielsen(Denmark) VICE CHAIR: Gunilla Berndtsson (Sweden) • WP 2Objective models and tools for multimedia quality • CHAIR: Paul Barrett (United States) VICE CHAIR: Vincent Barriac (France) • WP 3Multimedia QoS and QoE • CHAIR: Paul Coverdale (China) VICE CHAIR: Akira Takahashi (Japan)

10. Groups overseen by SG12 • CarCom Focus Group (closed in 2013) • Hans Gierlich, Chair • Driver Distraction Focus Group (closed in 2013) • Scott Pennock, Chair • QoS Development Group • Yvonne Umutoni, Chair • Regional Group for Africa • Gamal Amin Elsayed, Chair

11. CarCom Focus Group • In car communication • Quality parameters & testing methods • Interaction of car hands free systems with radio • Requirements on sub-system level • Requirements and testing procedures for super-wideband and fullband systems • Interaction with other audio components and systems in the car • Special requirements/testing procedures for speech recognition systems in cars • Quality models: what and how can be applied for the car environment?

12. QoS Development Group • Encourages wider participation in international QoS activities • Identifies and develop performance monitoring activities for QoS • Identifies procedures & practices for inclusion in the ITU-T Rec.s • Disseminates information relating to QoS techniques & procedures • Development of coordinated approach to study of QoS • Pursues other activities which may improve international QoS

13. Work Programme (1/8) • Coordination of QoS/performance studies • Communications Quality of Service: A framework and definitions • Vocabulary for performance and quality of service • Mean Opinion Score (MOS) terminology • Transmission characteristics for wideband digital handset and headset telephones - Inclusion of extended wideband terminals • Transmission characteristics for cordless and moblie digital terminals • Transmission characteristics for wideband digital loudspeaking and hands-free telephony terminals - Inclusion of extended wideband terminals • Technical requirements and test methods for the universal wired headset or headphone interface of digital mobile terminals • Narrowband hands-free communication in motor vehicles • Wideband hands-free communication in motor vehicles

14. Work Programme (2/8) • User interface requirements for automotive applications • Subsystem requirements for automotive speech services • Artificial ears • Calculation of loudness ratings for telephone sets • Use of head and torso simulator (HATS) for hands-free and handset terminal testing • Bone conduction testing setup • Multiple test positions setup • Setups and testing techniques for terminal performance measurements with background noise • Artificial voices • Speech processing devices for acoustic enhancement • Transmission characteristics and speech quality parameters of hands-free terminals

15. Work Programme (3/8) • Test signals for use in telephonometry • Objective test methods for speech communication systems using complex test signals • Methods for determining the ‘Objective Quality Number’ as overall quality value for terminals • Applicability of psycho-acoustic models to HATS based setups for the benefit of terminal testing • Setup signals for terminal with background noise • Subjective quality evaluation of telephone services based on spoken dialogue systems • Subjective performance of active signal processing devices • Cultural/language/nationality dependence of subjective quality • Subjective evaluation of generic sound activity detectors • Multidimensional scaling quality assessment • Subjective assessment methods for music quality of narrowband and wideband telephony

16. Work Programme (4/8) • Test methodology for web browsing and browser based applications • Subjective usability evaluation of speech-based or multimodal interactive services • Q.7/12 defines test plans for tests of speech coders, and reports the test results and analysis • E-model extension for non-handset user interfaces • E-model extension for speech processing devices • E-model update for conversational quality features • Mixed-band and beyond-wideband E-model • E-model for monitoring • Operational quality estimator • Perceptual approaches for multi-dimensional analysis • Perceptual objective noise reduction

17. Work Programme (5/8) • Perceptual Objective Prediction of Speech Intelligibility • No-reference models for quality prediction • Subjective method for simulated conversation tests addressing audio and audiovisual call quality • Effect of delays on the telemeeting quality • Use of auditory and visual cues for high-quality telemeetings • Overall telemeeting quality value (long term goal) • Quality implications and requirements for telemeeting and conferencing services • Spatial audio meetings quality evaluation • Frequent updates of Appendices • Quality of Experience aspects of Multi Connection • End-to-end QoE aspects of tandemmed speech processing devices

18. Work Programme (6/8) • End-to-end performance for managed voice over LTE networks This Recommendation describes the key aspects impacting end-to-end performance of managed voice applications over LTE networks. • Definitions and associated measurement methods for user-centric parameters for call handling in cellular mobile voice service • Mobile QoS • Supplement XX to ITU-T E.800-series Recommendations (Guidelines on Regulatory Aspects of QoS) • End-user multimedia QoS categories • Network model for evaluating multimedia transmission performance over Internet Protocol • Opinion model for video and audio streaming applications • QoE of Web-browsing • QoE requirements for multicast video streaming services • QoE requirements for telepresence services • Guidance for the use of P.1201 and P.1202 in operational contexts

19. Work Programme (7/8) • QoE-diagnostic information from P.120X.Y type models on technical causes of QoE impairments • Integral index of quality for general service monitoring (per user-session); KQI definitions • Media-session QoE model (temporal pooling, long sequences; UDP- and TCP-based streaming) • Parametric non-intrusive assessment of audiovisual progressive download, considering stalling/re-buffering & quality integration • Parametric non-intrusive assessment of TCP-based multimedia streaming quality, considering adaptive streaming • In-service non-intrusive measurement device – Voice service measurements • Analysis and interpretation of INMD voice-service measurements • Conformance testing for voice over IP transmission quality assessment models • Conversational model • Framework for invoking diagnostic functions • Voice Service Diagnosis Framework

20. Work Programme (8/8) • Technical cause analysis • Buffer models for development of client performance metrics • Hand-over performance among multiple access networks • Internet protocol data communication service – IP packet transfer and availability performance parameters • Network performance objectives for IP-based services • Measurements in IP networks for inter-domain performance assessment • Ethernet frame transfer and availability performance • Ethernet service activation test methodology • Home network performance parameters • Proposed Appendices for Y.1566 : QoS mapping and interworking between Ethernet, IP and MPLS

21. Cannot find your Topic ? • Work in ITU-T is contribution driven ! • Next Meeting of Study Group 12 • 3 – 12 December 2013 in Geneva • Deadline for Contributions • 20 November 2013 • http://www.itu.int/net/ITU-T/ddp/Default.aspx?groupid=T13-SG12

22. e-Learning Course on Mobile QoS -1 • Currently under Development • based on draft new Rec. ITU-T E.MQoS • Under the Umbrella of ITU Academyhttp://academy.itu.int/ • Other Courses available. e.g. • Working Methods of ITU-T Study Groupshttp://academy.itu.int/index.php/component/k2/item/1115

23. e-Learning Course on Mobile QoS-2 • Objectives • Better understanding of QoS seen from the end user perspective • Target Audience • Telecommunication Engineers • Telecoms Managers • Decision Makers engaged with relevant stakeholders: • Mobile Operators • National Regulators • Customer Organizations • Industry Organization • (regional) SDOs. • Required Knowledge Level before Course • Basic QoS concepts • Perceptual models • Skills gained by attending the Course • Ability to understand Mobile QoS from an end user perspective superior to the operation and maintanence parameters of mobile networks

24. e-Learning Course on Mobile QoS-3 • Module 1: Assessment of Quality of Service • Introduction to the relation between performance, QoS and QoE and their assessment and standardization • Module 2: Definition of Quality of Service parameters and their computation • Introduction to the four-layer model for mobile QoS parameters • Module 3: Typical procedures for Quality of Service measurement equipment

25. e-Learning Course on Mobile QoS-4 • Module 4: Requirements for Quality of Service measurement equipment • Test equipment for mobile QoS cannot be standardized. Therefore, this module provides an introduction to tests which are based on emulation of a typical user • Introduction to different Test scenarios: • User-to-user services • Store-and-forward services • Information services • Push services • Module 5: Definition of typical measurement profiles • Introduction of measurement profiles • One focus is on main factors affecting comparability

26. e-Learning Course on Mobile QoS-5 • Module 6: Post processing and statistical methods • Introduction to appropriate data analysis methods • Different data types are introduced • Four general categories of measurement results: • Data with binary values • Data out of time-interval measurements • Measurement of data throughput • Data concerning quality measures • Module 7: Network based Quality of Service measurements • This final module introduces network based QoS measurement

27. e-Learning Course on Mobile QoS-6 • Verification of the Learning Progress • Number of Questions • at intermediate Stages • at the end of Modules • at the end of the Course • Automatic Generation of Certificate • upon successful completion of Course

28. Any questions ? Joachim Pomy Telecommunications & Int'l Standards Germany Tel.: +49 177 78 71958 Email: consultant@joachimpomy.de