1 / 21

New Methods for Voice Quality Evaluation for IP Networks

New Methods for Voice Quality Evaluation for IP Networks. Outline. Introduction Subjective MOS test from traditional to Internet based Intrusive voice quality measurement from listening-only to conversational Non-intrusive measurement from subjective based to objective based Conclusions.

bree-gilbert
Download Presentation

New Methods for Voice Quality Evaluation for IP Networks

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. New Methods for Voice Quality Evaluation for IP Networks

  2. Outline • Introduction • Subjective MOS test from traditional to Internet based • Intrusive voice quality measurement from listening-only to conversational • Non-intrusive measurement from subjective based to objective based • Conclusions ITC-18, Berlin, Germany

  3. Introduction • Aims: to investigate new subjective and objective measurement methods for VoIP applications • Subjective tests • MOS (Mean Opinion Score), user perceived quality • Benchmarking objective methods • Slow, time consuming and expensive • Objective measurements • Intrusive methods (e.g. PESQ, only listening quality) • Non-intrusive methods (e.g. E-model, only for limited codec/applications) ITC-18, Berlin, Germany

  4. Traditional MOS test • Strict test requirement (e.g. sound proof room) • Guarantee consistent testing environment • Slow, time-consuming and expensive • Test environment far to reality • Originated from codec quality assessment (the quality difference is subtle). ITC-18, Berlin, Germany

  5. Internet-based MOS test • Based on Internet, carried out at office, project room or classroom, …, close to reality • Easier access to large number of subjects (e.g. 39 tests at the same time) • Save time and money • Lack of a controlled testing environment (e.g. background noise). ITC-18, Berlin, Germany

  6. Online MOS test ITC-18, Berlin, Germany

  7. Online MOS test ITC-18, Berlin, Germany

  8. Online MOS test results ITC-18, Berlin, Germany

  9. MOS (PESQ) PESQ Jitter buffer decoder encoder Trace data Degraded speech MOS (Web or Room) Subjective test E-model MOS (E-model) Voice Quality Evaluation Reference speech ITC-18, Berlin, Germany

  10. Performance Comparison ITC-18, Berlin, Germany

  11. Performance Comparison Correlation coefficients for MOS comparison • Results show that Internet-based MOS test compares well with traditional MOS test. • Two objective test methods (PESQ and E-model) can both predict subjective MOS score well. ITC-18, Berlin, Germany

  12. Intrusive conversational quality measurement • PESQ is the latest ITU standard for intrusive voice quality measurement. • It can only predict one-way listening voice quality. • How to extend it to predict conversational quality? • One solution: to combine with E-model ITC-18, Berlin, Germany

  13. MOS (PESQ) Reference speech Ie PESQ MOS  R  Ie E-model MOSc Degraded speech Delay model End-to-end delay Id Intrusive conversationalmeasurement • Procedures: • Obtain MOS from PESQ based on a comparison of reference and degraded speech • Convert it back to R-factor, then to Ie • Obtain Id from end-to-end delay • Calculate MOSc from Id and Ie from E-model ITC-18, Berlin, Germany

  14. R-factor with MOS ( From R to MOS, from G.107) (1) ( From MOS to R, by 3rd order polynomial) (2) ITC-18, Berlin, Germany

  15. Combined with E-model (3) (4) (5) Convert R back to MOS to obtain conversational voice quality ITC-18, Berlin, Germany

  16. Non-intrusiveobjective measurement • E-model is the latest ITU standard for non-intrusive voice quality prediction. • Its parameters/equations (e.g. Ie) are still based on subjective tests. • Difficult to apply to new codecs/applications • How to improve it ? • Based on objective methods (e.g. PESQ) to derive parameters/equations for new codecs and applications. ITC-18, Berlin, Germany

  17. Procedure 1:obtain MOS (PESQ) • Obtain MOS (PESQ) vs. packet loss rate (random) for the codec (e.g. AMR 12.2 Kb/s) • MOS (PESQ) value was obtained by averaging over different speech samples (male and female) in ITU data base and different loss locations/patterns. ITC-18, Berlin, Germany

  18. Procedure 2:Obtain Ie equation • Convert MOS vs. packet loss to Ie vs. packet loss using Equations (2) and (3). • Curve fitting to obtain equation of Ie vs. loss. (6) ITC-18, Berlin, Germany

  19. Procedure 3: Obtain MOS (loss, delay) • From Ie and Id, R can be derived, then MOS can be obtained directly from packet loss and end-to-end delay. • This can be used for non-intrusive quality monitoring and perceptual buffer optimization and perceived QoS control. ITC-18, Berlin, Germany

  20. Conclusions/Future work • Investigated novel subjective and objective voice quality evaluation methods. • Future work • More extensive Internet-based MOS test and comparison with P.800 MOS test • New applications for predicted perceived voice quality. ITC-18, Berlin, Germany

  21. Contact • MOS Test Website: http://www.tech.plymouth.ac.uk/spmc/people/lfsun/mos/ • Contact • Lingfen Sun: L.Sun@plymouth.ac.uk • Emmanuel Ifeachor: E.Ifeachor@plymouth.ac.uk Thank you! ITC-18, Berlin, Germany

More Related