Voice Quality and Testing EIT-03 Alan Percy Director of Business Development AudioCodes

1. Voice Quality and TestingEIT-03Alan PercyDirector of Business DevelopmentAudioCodes

2. Who is AudioCodes? • Leading OEM Manufacturer of Enabling VoIP equipment • Gateways • Media Servers • Boards, Chips and Modules • Key IPR holder for G.723.1 • 11th Year in Business • $82.8M Revenue for FY04 • NASDAQ: AUDC • Over 440 Employees • Worldwide Company: • Headquarters in Israel • US offices in San Jose, Boston, Chicago, Dallas, RTP, and Somerset NJ • Offices in Mexico, France, UK, China, and Japan

3. Background • Customer demand is building • VoIP carriers are proliferating • What will differentiate the carriers? Source: In-Stat

4. New Carriers, New Challenges A few years ago, we equipment providers thought we had this figured out. Then: • Private networks • Intra-Enterprise • Carefully controlled (over-engineered) networks

5. New Carriers, New Challenges Today: New Frontier • Depending on existing broadband • Cable • DSL • Fixed wireless • New Wireless Services • 3G • WiMax • WiFi • SAT

6. What have we learned? You can’t always control the network • To make the business model work, you sometimes need to leverage the existing infrastructure

7. What have we learned? The last mile is the hardest mile!

8. What have we learned? • Murphy’s law is true! (Poor voice quality will only affect the CEO during an important investor call) • Poor voice quality will get you kicked out • Huge variation in voice quality between equipment vendors (Especially when network conditions are not “ideal”)

9. Is voice quality important? “Voice Quality is the most sensitive issue for business customers” Greg Schreiber, VP of Product Development - Vaspian

10. What affects voice quality? Packet Loss Echo Jitter Latency Quality Coder Comfort Noise Voice Activity Detection

11. Choosing the Right Coder

12. Vocoder Factors • Single speaker, no packet loss, no acoustic noise • Vocoder quality given non speech audio (music, background noise) • Bit rate • Algorithmic delay = Packet size + look ahead • Audio bandwidth • Low bit rate algorithm type • Complexity (# MIPS) • Intrinsic robustness to packet loss

13. Vocoder Factors Vocoder implementation issues: • Not all vocoders standards have bit exact test vectors • And test vectors don’t cover every angle … • Little known fact - even long approved and deployed vocoder standards still have bugs – let alone new ones … • Audiocodes has found and reported bugs in numerous standard vocoders • Hence the importance of Speech Quality Testing and deployment experience

14. Vocoder Tradeoffs • Complexity is always a constraint • Bit rate and algorithm type affect speech and non-speech vocoding quality • Larger packet size enables lower bit rate but increases delay (e.g. G.723.1 vs. G.729A) • Higher bit rate enables better intrinsic robustness to packet loss • Higher audio bandwidth requires higher bit rate

15. Echo Canceller Factors • Imperfect EC performance could result in: • Residual echo • Clipping • Distorted speech • Annoying variation in background noise • Longer round trip delay makes echo cancellation more critical

16. Echo Canceller Factors • ECs are challenged most during Double Talk (DT) • Double Talk happen occasionally during full duplex conversations • During Double Talk • EC may diverge • The Non Linear Processor (NLP) could clip, attenuate or distort the near-end speech • Therefore it is crucial to disable the NLP during Double Talk • Timely and reliable detection of Double Talk could be tricky

17. Echo Canceller Factors • Background noise continuity (level, spectrum) should be maintained • The variety of real life echo paths poses an algorithmic and testing challenge • field experience crucial • EC standards (e.g. G.168) specify testing methods - not standard algorithms • proprietary EC algorithms could vary significantly in quality • Quality testing is essential

18. Echo Canceller Tradeoffs • Attenuation vs. tail length vs. adaptation speed vs. steady state coefficients noise and EC stability • Echo Attenuation in Single Talk vs. Double Talk performance • Complexity (# MIPS)

19. Packet Loss Concealment • Vocoder type (CELP vs. ADPCM vs. PCM) • All vocoders (including G.711) have PLC standardized to some extent • Vendor distinction depends on Jitter Buffer quality and PLC enhancements • Trade offs • Latency vs. Jitter buffer size vs. % frame erasure rate • Redundancy schemes vs. bit rate

20. Condition Interaction Effect Effect: • Customers tell us that G.723.1 many times sounds better than other’s G.711 Why? • Lower bit rate reduces congestion and results in fewer dropped packets. • G.723.1 also includes packet loss concealment, smoothing over the gaps.

21. Improving VoIP Quality Techniques vendors use to maximize quality: • Dynamic Jitter Buffer • Adjusting to ever-changing network conditions • Packet loss concealment • All coders, including G.711 • Echo cancellation algorithms • VAD and CNG algorithms • Extensive field and lab testing

22. Speech Quality Testing • Methods • Events • Results

23. Speech Quality Testing Methods • MOS • Traditional subjective method, uses a panel of listeners, accurate but expensive • PESQ - Perceptual Evaluation of Speech Quality • objective method based on human hearing model, ITU P.862 standard • Computer driven • G.168 • ITU standard for echo canceller testing, uses artificial speech signals • P.501 / P.502 • ITU standard for complex test signals used to test echo cancellers and other modules. • These signals were designed to provide more information to the designer about the weakness points.

24. ETSI Speech Quality Test Event • Sponsored by ETSI - Done by HEAD Acoustics Labs • Has been held several times over the past three years • VoIP equipment testing in controlled conditions • Receive an objective and comparative analysis • The tests include a range of aspects, including: • Speech sound quality • Echo measurements • Double Talk performance • Transmission quality in the presence of background noise • Anonymous - Only your results are identified

25. ETSI Test Setup

26. ETSI results “Pie Chart” • Six conversational and six listening tests • Ten gateways participated in 3rd SQTE • The results for each gateway are summarized in a Pie Chart with a slice per each category • The performance in each category was represented by the radius of the slice and by a color code • Red – the amount of which the results were under the requirement • Yellow – result was below the requirement if accompanied with red or OK if no red • Green – result was above the requirement

27. ETSI SQTE Test Results

28. ETSI SQTE Test Results AudioCodes

29. How can you use this information? • Ask for ETSI results when shopping • Read and evaluate performance needs against test results. • Justify partner selection • Help sell your solution

30. Where to learn more: Full ETSI SQTE Test Report: http://www.etsi.org/plugtests/History/2004SQTE.htm AudioCodes ETSI SQTE Results: http://www.audiocodes.com/Content.aspx?voip=2189

31. Let’s talk! Stop by Booth #405