Comparison of Narrowband and Wideband Speech codecs in noisy environnement

1. Workshop on Wideband Speech Quality in Terminals and Networks: Assessment and Prediction8th and 9th June 2004 - Mainz, Germany Comparison of Narrowband and Wideband Speech codecs in noisy environnement Noel Chateau noel.chateau@francetelecom.com, Laetitia Gros laetitia.gros@francetelecom.com, Catherine Quinquis catherine.quinquis@francetelecom.com, Jean Yves Monfort jeanyves.monfort@francetelecom.com

2. Content • 3Gpp/VoIP context • Methodology • Description of Experiment • Results • Conclusion

3. 3GPP/VoIP Context • Rel 99 : March 2000 • only narrow band speech and Circuit switched communications • Rel4: March 2001 • Packet switched for download • Selection of wide band speech codec • Rel5: March 2002 • Packet switched conversationnal communications • narrow band and wide band codecs • Rel6: September 2004 • Characterisation of default codecs for Packet switched conversationnal communications

4. 3GPP/ Characterisation of default codecs for Packet switched conversationnal communications • 1st phase • AMR codec in quiet and noisy environment • AMRWB codec in quiet and noisy environment • 2nd phase • AMR and AMRWB versus legacy codecs in quiet environment • Further experiment • AMR and AMRWB versus legacy codecs in noisy environment : car noise

5. Methodology • VoIP in UMTS • delay is one major issue • Interleaving due to radio environment • Buffering in decoder side to cope with jitter • IP transport • Packetisation in IP as well as on radio leg • Not only quality of codecs but acceptability of service • Conversation test • More accurate to assess acceptability of service

6. Experiment: Description • Noisy environment : car noise @ 60dB Pa • Car noise is in one of the room, the other room is quiet • Codecs under test • AMR (6.7 and 12.2 modes) • AMRWB (12.65 and 15.85 modes) • G.723.1@ 6.3 kb/s • G.729@ 8 kb/s • G.711@ 64 kb/s • G.722 @ 64 kb/s • Environmental conditions • IP packet loss : 0% or 3%

7. PC 1 : VOIP Terminal Simulator PC 3 : Network Simulator PC 2 : VOIP Terminal Simulator Network Board A Network Board B Hub 1 Hub 2 Experiment: Description

8. Results on voice quality criterion • ANOVA • significant main effects: noise, PLP & codec • significant interactions: codec x PLP • Planned comparisons • Interaction bandth x noise for PLP=0%: NS • Interaction Bandth x PLP for Noise: NS • Tuckey tests show a superiority of some codecs on others for noisy and/or lossy conditions, but not systematic. Without noise and packet losses: no significant differences. • The superiority of WB Codecs is higher for noisy and packet-loss conditions, but there is no systematic benefit of WB in noise. A N O V A

9. Results on understanding criterion • ANOVA • significant main effects: noise, PLP & codec • significant interactions: codec x PLP • Planned comparisons • Interaction bandth x noise for PLP = 0%: Signif. • Interaction Bandth x PLP for Noise: NS • Tuckey tests show an almost systematic superiority of WB and G.711 codecs on NB codecs (except AMR NB 12.2 kbs) for noisy and/or lossy conditions. Without noise and packet losses: no significant differences. • Considered globally, WB and G.711 codecs better resist to noise for PLP=0%. A N O V A

10. Results on interaction criterion • ANOVA • significant main effects: noise, PLP & codec • significant interactions: codec x noise • Planned comparisons • Interaction bandth x noise for PLP = 0%: Signif. • Interaction Bandth x PLP for Noise: NS • Tuckey tests show only a few significant differences for noisy conditions only. • As for the understandy criterion, considered globally, WB codecs better resist to noise for PLP=0%. A N O V A

11. Results on default perception criterion • ANOVA • significant main effects: noise, PLP & codec • significant interactions: codec x PLP • Planned comparisons • Interaction bandth x noise for PLP = 0%: NS • Interaction Bandth x PLP for Noise: NS • Tuckey tests show no significant differences between codecs for any condition. • The default perception criterion does not seem relevant to detect a possible advantage of WB codecs in noise. A N O V A

12. Results on global quality criterion • ANOVA • significant main effects: noise, PLP & codec • significant interactions: none • Planned comparisons • Interaction bandth x noise for PLP = 0%: NS • Interaction Bandth x PLP for Noise: NS • Tuckey tests show an almost systematic superiority of WB and G.711 codecs on NB codecs (except AMR NB 12.2 kbs) for noisy and/or lossy conditions. Without noise and packet losses: no significant differences. • As for the voice-quality criterion, the superiority of WB Codecs is higher for noisy and packet-loss conditions, but there is no systematic benefit of WB in noise. A N O V A

13. Summary of subjective tests results • ANOVA is not useful for determining if WB codecs are significantly better than NB codecs in noisy environment but show that noise was the strongest experimental factor that affected the subjective data. • Planned comparisons and Tuckey tests show that: • the MOS differences between WB and NB are significantly larger in noisy conditions than in silent conditions for no packet losses, for "Understanding" and "Interactivity" criteria. • the MOS differences between WB and NB are often but not systematically larger in noisy and/or 3% packet-loss conditions than in the silent and 0 % packet-loss condition, for the "Voice quality" and "Global quality" criteria. • the differences between WB and NB is independant on the presence of noise and packet loss for the "Default perception" criterion. • It can be concluded that without packet loss, in noise, the advantage of WB compared to NB is that it enhances the comprehension and the interaction with the other partner, but not voice and global quality, since these criteria are highly affected by the presence of noise.