Country example of QoS measurement of voice service Case of Morocco

1. ITU Workshop on “Workshop on Practical measurement of QoS/QoE Parameters for Regulatory Compliance”(Cotonou, Benin, 16-17 July 2012) Country example of QoS measurement of voice serviceCase of Morocco Hassan TALIB, Vice-Chair ITU-T SG 12, talib@anrt.ma // htalib@ties.itu.int

2. Presentation Scope • The QoS for Morocco telecommunications A protocol for voice QoS measurements: • Choice of a subjective method, • List and definitions of indicators, • Measurements Methodology: • Definition of voice QoS measurement, • Sample and Statistic considerations, • Organization and Operation, • Process of Outcomes. • Recommendations to regulators/Op.

3. The QoS for Morocco Telecommunications Fixed and mobile Operators VSAT Operators Fixed Operator with narrow mobility GMPCS Operators ISP Radio Trunk Operators MAROC CONNECT (WANA) MAROC TELECOM MEDITELECOM Open Competition Fixed and mobile: oligopoly oligopoly Oligopoly Duopoly SPACECOM GULFSAT CIMECOM EUROPEAN DATACOM GLOBALSTAR ORBCOMM SOREMAR THURAYA MAROC TELECOM MAROC CONNECT (WANA) MEDITELECOM ISPs INQUAM MORATEL • Every operators are responsible for QoS

4. The QoS for telecommunications in Morocco • International regulatory framework (standards) : • IUT-T : series E, G, P, Y, QoS handbook, ... • Regional : ETSI (series EG), IEEE,… • A national regulatory Framework: • Allocations • QoS Obligations for operators A benchmark of best practices and technological monitoring The ANRT regularly monitors the QoS • Use feedback : • Complaints, • Press, • End Users Associations, • … Analysis of data received from the operators in terms of performances, KPI and QoE measurements Ground Measurements (campaigns) by ANRT

5. The QoS for telecommunications in Morocco • Development of voice mobile telephony service in Morocco

6. Presentation Scope • The QoS for Morocco telecommunications A protocol for voice QoS measurements: • Choice of a subjective method, • List and definitions of indicators, • Measurements Methodology: • Definition of voice QoS measurement, • Sample and Statistic considerations, • Organization and Operation, • Process of Outcomes. • Recommendations to regulators/Op.

7. Protocol for voice QoS measurements Choice of a subjective method • The quality of service sets a premium on models for reference (IUT-T standards : E, I, G, P, Y… ou ETSI : ETSI TS 102 250-x), on the levels of performance for service provided and the measurement methods. The models and methods are supposed to be coherent with the users expectations. • For voice service, QoS requirements could be related to many aspects sumed up in its availability, accessibility and continuity and reliability.

8. Protocol for voice QoS measurements Choice of a subjective method Necessity to harmonize the definitions of indicators between operators and the regulator.

9. Presentation Scope • The QoS for Morocco telecommunications A protocol for voice QoS measurements: • Choice of a subjective method, • List and definitions of indicators, • Measurements Methodology: • Definition of voice QoS measurement, • Sample and Statistic considerations, • Organization and Operation, • Process of Outcomes. • Recommendations to regulators/Op.

10. Protocol for Voice QoS measurements List and definitions of indicators TR + TE + TC = 100% and TNR = TE + TC

11. Protocol for voice QoS measurements List and definitions of indicators Evaluation subjective de la qualité auditive : définitions des défauts de communication vocale

12. Presentation Scope • The QoS for Morocco telecommunications A protocol for voice QoS measurements: • Choice of a subjective method, • List and definitions of indicators, • Measurements Methodology: • Definition of voice QoS measurement, • Sample and Statistic considerations, • Organization and Operation, • Process of Outcomes. • Recommendations to regulators/Op.

13. Protocol for voice QoS measurements Measurements Methodology A QoS measurement shall necessarily be done within the coverage area by all concerned mobile networks. The QoS measurement is different from the coverage measurement. A voice QoS measurement consists in trying to establish a communication from the mobile investigator to the fixed investigator to verify the reception of the call (no failure) and the continuation of the communication for two minutes (no cut off) and to assess the audio quality of the established communication. Each network is tested by a “couple” of investigators, on mobile and the other fixed.

14. Protocol for voice QoS measurements Measurements Methodology

15. Protocol for voice QoS measurements Measurements Methodology • A communication is considered failed if: • at the first attempt (maximum delay time is about 15 seconds), it is not established, • or has not been maintained more than 5 seconds. • A communication is considered cut off if, at the first attempt: • it is established and maintained more than 5 seconds, • but cut off before 2 minutes.

16. Protocol for voice QoS measurements Measurements Methodology Precautions to be taken: Outdoor : avoid measuring under trees. Indoor : very inside buildings and till 8th floor. Incar : use terminals without additional antenna and drive at a speed less than 80km/h. Force terminals on 2G in case of use of bi-mode posts. Every points for measurements shall be covered.

17. Protocol for voice QoS measurements Measurements Methodology • Sample and statistic considerations: • Average duration of a measurement: 7 mn, • Average duration for measurements takes place everyday: 8:00 to 10:00 (Example : 09:00-13:00 and 14:00 to 20:00), • Everyday (per team) : 85 measurements x Number of operators; For 3 operators : 255 measurements per day,

18. Protocol for voice QoS measurements Measurements Methodology • Sample and statistic considerations: • Breakdown Element: Per operator, Per site (cities, motorways, national highways, railways), per configuration (Indoor, Outdoor, Incar), per traffic destination (OnNet and OffNet). • Important numbers to provide a significant sample (statistic error about 2%) : elementary values after any breakdown shall be higher than 33.

19. Presentation Scope • The QoS for Morocco telecommunications A protocol for voice QoS measurements: • Choice of a subjective method, • List and definitions of indicators, • Measurements Methodology: • Definition of voice QoS measurement, • Sample and Statistic considerations, • Organization and Operation, • Process of Outcomes. • Recommendations to regulators/Op.

20. Protocol for voice QoS measurements Measurements Methodology • Organization and Operation: • Decider to achieve based on own resources or controlled subcontracting, • Select effectively investigators (mobiles and Fixed) and conduct a targeted training, • Consult with operators on the methodology but not the sample and the planning of each campaign,

21. Protocol for voice QoS measurements Measurements Methodology • Organization and operation: • Develop a guide for the investigator with some templates to be used by the investigator: Ref., delay, GPS Information, recipient, outcomes,… • Use the principle for codificationprincipe de codification : opérateur, site, configuration, trafic,…

22. Protocol for voice QoS measurements MeasurementsMethodology • Organization and operation: • Guarantee any conditions so that each measurement is exactly conducted in the same conditions for every operators: delay, space, method,… • Proceed to a number of important blank tests calibrate any test system.

23. Presentation Scope • The QoS for Morocco telecommunications A protocol for voice QoS measurements: • Choice of a subjective method, • List and definitions of indicators, • Measurements Methodology: • Definition of voice QoS measurement, • Sample and Statistic considerations, • Organization and Operation, • Process of Outcomes. • Recommendations to regulators/Op.

24. Protocol for voice QoS measurements MeasurementsMethodology • Outcomes Processing: (Scheduled to be Delivered) • Trend in indicator “positive” : successful rate. • Trend in indicator “negative” : unsuccessful rate = TE+TC. • Reports: • Global Report: reference, • Specific Report to each operator (request for actions plan for corrections), • Report for publication.

25. Presentation Scope • The QoS for Morocco telecommunications A protocol for voice QoS measurements: • Choice of a subjective method, • List and definitions of indicators, • Measurements Methodology: • Definition of voice QoS measurement, • Sample and Statistic considerations, • Organization and Operation, • Process of Outcomes. • Recommendations to regulators/Op.

26. Conclusions and Recommendations • Recommendation n°1 • Permanent Consultation with the upstream operators upon the methodology: financing of campaigns. Adoption of an agreed protocol of references for measurements. • Recommendation n°2 • Positive use of outcomes by the operators (consider sanctions as last resort).

27. Conclusions and Recommendations 1stvoicecampaign on basedupon a large sample (thirtycities, highways, national motorways and railways). 1st quarter of the year 2ndsimilarcampaignpracticallybased on the first upon the samesample. 4th quarter of the year (Publication of results) • Recommendation n°3 • Publication of measurements outcomes, • Proposal of a scenario : Communication of operators’ results Delay of 5 to 6 months for correction of discrepancies identified by the ANRT

28. Conclusions and Recommendations • Recommendation n°3 (bis) • Publication of outcome measurements The TMR is equal to the successfuk average rates (TR) recorded by each of the three operators: TMR

29. Thanks for your attention Questions/Answers talib@anrt.ma // htalib@ties.itu.int