1 / 39

QoS Measurement and Management for Multimedia Services

QoS Measurement and Management for Multimedia Services. Thesis Proposal Wenyu Jiang April 29, 2002. Topics Covered. Objective QoS metrics for real-time multimedia Subjective/Perceived quality Objective perceptual quality estimation algorithms Quality enhancement for real-time multimedia

clementine
Download Presentation

QoS Measurement and Management for Multimedia Services

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. QoS Measurement and Management for Multimedia Services Thesis Proposal Wenyu Jiang April 29, 2002

  2. Topics Covered • Objective QoS metrics for real-time multimedia • Subjective/Perceived quality • Objective perceptual quality estimation algorithms • Quality enhancement for real-time multimedia • IP telephony deployment • VoIP quality in the current Internet

  3. Backgrounds and Motivations • The Internet is still best-effort • Needs QoS monitoring • What to measure/monitor? • Loss, delay, jitter • Must map to perceived quality • What to do if quality is not good? • End-to-End: FEC, LBR • Network provisioning: voice traffic aggregation • IP telephony service deployment • Current ITSPs are not doing well • Lack of study on localized deployment • What is the status of the current Internet?

  4. How Real-time Multimedia Works • A/D conversion; Encoding; Packet transmission; Decoding; Playout; D/A conversion • Dominant QoS factors: • Loss  clipping/distortion in audio • Delay  lower interactivity • Jitter  late loss

  5. Delay and Loss Measurement • Solutions for clock synchronization • Telephone-based synchronization • RTT-based, assume symmetric delays • GPS-based • Dealing with Clock drift • De-skewing by linear regression • One-way vs. round-trip measurement • Internet load often asymmetric • One-way loss and delay are more relevant to real-time multimedia

  6. Loss and Delay Models • Loss Models • Gilbert model • Extended Gilbert model • Others • Delay Models • More difficult to construct • No universal distribution function • Temporal correlation between delays

  7. Perceived Quality Estimation • Mean Opinion Score (MOS) • Requires human listeners • Labor and time intensive • Reflective of real quality • Objective perceptual quality estimation algorithms • PESQ, PSQM/PSQM+, MNB, EMBSD • Speech recognition based (new)

  8. Network Provisioning for VoIP • Silence suppression • Saves bandwidthstatistical multiplexing • The on/off patterns in human voice depend on the voice codec or the silence detector • Voice traffic aggregation • Multiplexing by token bucket filtering • The on/off patterns in human voice directly affects aggregation performance • Past study assumes exponential distribution

  9. IP Telephony Deployment • Localized deployment • More practical than a grand-scale Internet deployment • Can still interoperate with an IP telephony carrier • Issues • PSTN interoperability • Security • Scalability • Billing

  10. Research Objectives • Objective QoS metrics • Modeling • Their relationship to perceived quality • Objective perceptual quality estimation algorithms vs. perceived quality (MOS) • Quality improvement measures • End-to-End: FEC vs. LBR • Network-based: voice traffic aggregation • IP telephony deployment issues • VoIP quality measurement over the Internet

  11. Completed Work: QoS Measurement Tools • UDP packet trace generator • Clock synchronization and de-skewing tool • Loss and delay modeling tools • By examining a packet trace • Outputs Gilbert and extended Gilbert model parameters • Outputs conditional delay CCDF • Playout simulator • Simulates several common playout algorithms • FEC is also supported

  12. Completed Work: Comparison of Loss Models • Loss burst distribution • Roughly, but not exactly exponential • Inter-loss distance • Clustering between adjacent loss bursts

  13. Loss Model Comparison, contd. • Loss burstiness on FEC performance • FEC less efficient under bursty loss • Final loss pattern (after playout, FEC) • Generally also bursty

  14. Mapping from Loss Model to Perceived Quality • Random vs. bursty loss • Bursty  lower MOS • Effect of loss burstiness • Sometimes very bursty loss does not lead to lower quality

  15. A New Delay Model • Conditional CCDF (C3DF) • Allows estimation of burstiness in the late losses introduced by (fixed) playout algorithm

  16. Objective vs. Subjective MOS • Algorithms: PESQ, PSQM, PSQM+, MNB, EMBSD Using Original Linear 16 samples as reference signal Using G.729 no loss clip as reference signal

  17. Objective MOS Correlation, contd. • Second test set • Stronger “saturation” effect observed for MNB1 and MNB2, but not for PESQ Linear-16 reference signal G.729 reference signal

  18. Auditory Distance vs. MOS • EMBSD and PSQM+ appear to have the largest spread, i.e., least correlation w. MOS • PSQM seems to be similar to MNB in terms of correlation

  19. Auditory Distance vs. MOS, contd. • Second test set • Similar behaviors observed Linear-16 reference signal G.729 reference signal

  20. Analysis of Objective MOS Correlation • Quantitative metric • Correlation coefficient  • But it does not tell everything!

  21. Speech Recognition Performance as a MOS predictor • Evaluation of automatic speech recognition (ASR) based MOS prediction • IBM ViaVoice Linux version • Codec used: G.729 • Performance metric • absolute word recognition ratio • relative word recognition ratio

  22. Recognition Ratio vs. MOS • Both MOS and Rabs decrease w.r.t loss • Then, eliminate middle variable p

  23. Speaker Dependency Check • Absolute performance is speaker-dependent • But relative word recognition ratio is not

  24. Speech Intelligibility Results • Human listeners are asked to do transcription • Human recognition result curves are less “smooth” than MOS curves.

  25. Analysis of Voice On-Off Patterns • Past study finds spurt & gap distributions to be exponential • Modern voice codecs and silence detectors have different behaviors

  26. Voice Traffic Aggregation • Simulation environment • DiffServ token bucket filter • Exponential, CDF and trace-based model simulations • N voice sources • Token buffer size B (packets) • R: ratio of reserved vs. peak bandwidth • Key performance figure • Probability of out-of-profile packet

  27. Aggregation Simulation Results • Results based on G.729 VAD • CDF model resembles trace model in most cases • Exponential (traditional) model • Under-predicts out-of-profile packet probability; • The under-prediction ratio increases as token buffer size B increases

  28. Simulation Results, contd. • Results based on NeVoT SD (default parameters: high threshold, long hangover) • Similar behavior, although the gap between exponential and CDF model is smaller for NeVoT case

  29. Comparisons of FEC and LBR • Forward error correction • Bit-exact recovery • No decoder state drift upon recovery • Low bit-rate redundancy (LBR) • Just the opposite to FEC • Design of an optimal LBR algorithm • State repair via redundant codec • Optimal packet alignment • MOS quality verified to be better than the rat LBR • Allows a more “fair” comparison with FEC

  30. MOS Quality of FEC vs. LBR • FEC shows a substantial and consistent advantage over LBR • This is true for all LBR configurations we tested • Main codec is G.729 except for AMR LBR DoD-CELP LBR DoD-LPC LBR

  31. MOS of FEC vs. LBR, contd. • AMR LBR: narrowest gap with FEC • (Not shown here) FEC out-performs LBR under random loss as well G.723.1 LBR AMR LBR

  32. Optimizing FEC Quality • Packet interval  loss burstiness FEC efficiency  • Result: FEC MOS performance also improves

  33. Optimizing Conversational MOS for FEC • A larger packet interval  more delay • Trade-off between quality and delay • The E-model • Considers both delay and loss (and many other transmission quality factors) • Optimizing FEC MOS with the E-model

  34. Optimizing FEC MOS, contd. • Validating E-model based prediction with real MOS test results

  35. Localized IP Telephony Deployment: Architecture • Component based and distributed architecture • Allows easy integration of all SIP-compliant devices and programs

  36. Deployment Issues • PSTN interoperability • T1 configuration and PBX integration • T1 line type (Channelized vs. ISDN PRI) • Line coding and framing (layer 2) • Trunk type: Direct-inward-dialing (DID) • Access permission on the PBX side • SIP/PSTN gateway configuration • Dial-peer: locates the proper SIP server or PSTN trunk • Dial-plan (translating calls from/to PSTN)

  37. Deployment Issues, contd. • Security • Issue: gateway has no authentication feature • Solution: • Use gateway’s access control lists to block direct calls • SIP proxy server handles authentication using record-route • Allows easier change in authentication module (software-based) • Certain users can only make certain gateway calls • Scalability • SIP server (DNS SRV scaling) • Gateway; voice-mail server; conference server • Billing • Initial implementation via transaction logging

  38. On-going Research • Measurement of the current Internet • How well can it support VoIP? • Or, how easy can VoIP applications adapt to (unfavorable) network conditions? • How fast does network condition change? • Can network redundancy help improve VoIP quality? • Physical redundancy (access links) • Virtual redundancy (overlay networking)

  39. Conclusions • Completed research relating to many aspects of real-time multimedia, in particular VoIP • On-going work calls for: • A comprehensive measurement of the Internet • Analysis of the to-be measurement data • An answer to the question: how good is it today, and, how much better can we do?

More Related