1 / 33

Outline

Measuring Interaction QoE in Internet Videoconferencing Prasad Calyam (Presenter) Ohio Supercomputer Center, The Ohio State University Mark Haffner, Prof. Eylem Ekici Prof. Chang-Gun Lee The Ohio State University Seoul National University MMNS, November 1 st 2007. Outline.

ramya
Download Presentation

Outline

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Measuring Interaction QoE in Internet VideoconferencingPrasad Calyam (Presenter)Ohio Supercomputer Center, The Ohio State UniversityMark Haffner, Prof. Eylem Ekici Prof. Chang-Gun LeeThe Ohio State UniversitySeoul National UniversityMMNS, November 1st 2007

  2. Outline • Background • Voice and Video over IP (VVoIP) Overview • Network QoS and End-user QoE in VVoIP • Streaming QoE versus Interaction QoE • Network Fault Events • Multi-Activity Packet Trains (MAPTs) methodology • Participant Interaction Patterns • Traffic Model for MAPTs emulation • Vperf tool implementation of MAPTs • Performance Evaluation • Concluding Remarks and Future Work

  3. Outline • Background • Voice and Video over IP (VVoIP) Overview • Network QoS and End-user QoE in VVoIP • Streaming QoE versus Interaction QoE • Network Fault Events • Multi-Activity Packet Trains (MAPTs) methodology • Participant Interaction Patterns • Traffic Model for MAPTs emulation • Vperf tool implementation of MAPTs • Performance Evaluation • Concluding Remarks and Future Work

  4. Voice and Video over IP (VVoIP) Overview • Large-scale deployments of VVoIP are on the rise • Video streaming (one-way voice and video) • MySpace, Google Video, YouTube, IPTV, … • Video conferencing (two-way voice and video) • Polycom, MSN Messenger, WebEx, Acrobat Connect, … • Challenges for large-scale VVoIP deployment • Real-time or online monitoring of end-user Quality of Experience (QoE) • Traditional network Quality of Service (QoS) monitoring not adequate • Network QoS metrics: bandwidth, delay, jitter, loss • Need objective techniques for automated network-wide monitoring • Cannot rely on end-users to provide subjective rankings – expensive and time consuming

  5. Network QoS and End-user QoE • End-user QoE is mainly dependent on the combined impact of network factors • Device factors such as voice/video codecs, peak video bit rate (a.k.a. dialing speed) also matter • Our study maps the network QoS to end-user QoE for a given set of commonly used device factors • H.263 video codec, G.711 voice codec, 256/384/768 Kbps dialing speeds Network QoS End-user QoE

  6. Voice and Video Packet Streams • Total packet size (tps) – sum of payload (ps), IP/UDP/RTP header (40 bytes), and Ethernet header (14 bytes) • Dialing speed is ; = 64 Kbps fixed for G.711 voice codec • Voice has fixed packet sizes (tpsvoice ≤ 534 bytes) • Video packet sizes are dependent on alevin the content

  7. Video alev • Low alev • Slow body movements and constant background; E.g. Claire video sequence • High alev • Rapid body movements and/or quick scene changes; E.g. Foreman video sequence • ‘Listening’ versus ‘Talking’ • Talking video alev(i.e., High) consumes more bandwidth than Listening video alev (i.e., Low) Foreman Claire

  8. End-user QoE Types • Streaming QoE • End-user QoE affected just by voice and video impairments • Video frame freezing • Voice drop-outs • Lack of lip sync between voice and video • Interaction QoE • End-user QoE also affected by additional interaction effort in a conversation • “Can you repeat what you just said?” • “This line is noisy, lets hang-up and reconnect…” • QoE is measured using “Mean Opinion Score” (MOS) rankings

  9. Network Fault Events • “Best-effort service” of Internet causes network fault events that impact application performance • Cross traffic congestion, routing instabilities, physical link failures, DDoS attacks • Our definition of network fault events is based on the “Good”, “Acceptable” and “Poor” (GAP) performance levels for QoS metrics causing GAP QoE • Type-I: Performance of any network factor changes from Good grade to Acceptable grade over a 5 second duration • Type-II: Performance of any network factor changes from Good grade to Poor grade over a 10 second duration

  10. Related Work • Characteristics of network fault events well understood • Bursts, spikes, complex patterns – lasting few seconds to a few minutes (Markopoulou et. al., Ciavattone et. al.) • Measuring Streaming QoE impact due to network fault events has been well studied • ITU-T E-Model is a success story for VoIP QoE estimation • Designed for CBR voice traffic and handles only voice related impairments • ITU-T J.144 developed for VVoIP QoE measurement • “PSNR-based MOS” – Requires original and reconstructed video frames for frame-by-frame comparisons • Offline method - PSNR calculation is a time consuming and computationally intensive process • Online VVoIP QoE measurement proposals • PSQA (G. Rubino, et. al.), rPSNR (S. Tao, et. al.) • Measuring Interaction QoE impact due to network fault events has NOT received due attention • Need for schemes to measure interaction difficulties in voice and video conferences presented by A. Rix, et. al.

  11. Problem Summary • Given: • Voice/video codecs used in a videoconference • Dialing speed of the videoconference • Network fault event types to monitor • Develop: • An objective technique that can measure Interactive VVoIP QoE • Real-time measurement without involving actual end-users, video sequences and VVoIP appliances • An active measurement tool that can: (a) emulate VVoIP traffic on a network path, and (b) use the objective technique to produce Interaction QoE measurements Multi-Activity Packet Trains Methodology Vperf Tool

  12. Outline • Background • Voice and Video over IP (VVoIP) Overview • Network QoS and End-user QoE in VVoIP • Streaming QoE versus Interaction QoE • Network Fault Events • Multi-Activity Packet Trains (MAPTs) methodology • Participant Interaction Patterns • Traffic Model for MAPTs emulation • Vperf tool implementation of MAPTs • Performance Evaluation • Concluding Remarks and Future Work

  13. Proposed Solution Methodology • “Multi-Activity Packet Trains” (MAPTs) measure Interaction QoE in an automated manner • They mimic participant interaction patterns and video activity levels as affected by network fault events • Given a session-agenda, excessive talking than normal due to unwanted participant interaction patterns impacts Interaction QoE • “Unwanted Agenda-bandwidth” measurement and compare with baseline (consumption during normal conditions) • Higher values indicate poor interaction QoE and caution about potential increase in Internet traffic congestion levels • Measurements serve as an input for ISPs to improve network performance using suitable traffic engineering techniques

  14. Proposed Solution Methodology (2) ‘repeat’ ‘disconnect’ ‘reconnect’ ‘reorient’ Type-I and Type-II fault detection

  15. Participant Interaction Patterns • Assumption: Question (Request) and Answer (Response) items in a session agenda • Side-A listening when Side-B talking, and vice versa Normal – PIP1

  16. Participant Interaction Patterns (2) • Participant Interaction Patterns (PIPs) using MAPTs for a “Type-I” network fault event Repeat – PIP2

  17. Participant Interaction Patterns (3) • Participant Interaction Patterns (PIPs) using MAPTs for a “Type-II” network fault event Disconnect/Reconnect/Reorient – PIP3

  18. Participant Interaction Patterns (4) • Our goal is to measure the UnwantedAgenda-bandwidth and UnwantedAgenda-timemeasurements after MAPTs emulation of the Q & A session agenda

  19. Traffic Model for MAPTs Emulation • Traffic Model for probing packet trains obtained from trace-analysis • Combine popularly used low and high alev video sequences and model them at 256/384/768 Kbps dialing speeds for H.263 video codec • Low – Grandma, Kelly, Claire, Mother/Daughter, Salesman • High – Foreman, Car Phone, Tempete, Mobile, Park Run • Modeling • Video Encoding Rates (bsnd) time series • Packet Size (tps) distribution • Derived instantaneous inter-packet times (tps) by dividing instantaneous packet sizes by video encoding rates

  20. Video Encoding Rates (bsnd) Modeling • Time-series modeling of the bsnd data using the classical decomposition method • We find a Second order moving average [MA(2)] process model fit • θ1 – MA(1) parameter • θ2 – MA(2) parameter High alev Low alev

  21. Video Packet Size (tps) Distribution Modeling • Distribution-fit analysis on the tps data • We find a Gamma distribution fit • α – shape parameter • β – scale parameter For High alev at 256 Kbps

  22. Traffic Model Parameters for MAPTs Emulation

  23. Outline • Background • Voice and Video over IP (VVoIP) Overview • Network QoS and End-user QoE in VVoIP • Streaming QoE versus Interaction QoE • Network Fault Events • Multi-Activity Packet Trains (MAPTs) methodology • Participant Interaction Patterns • Traffic Model for MAPTs emulation • Vperf tool implementation of MAPTs • Performance Evaluation • Concluding Remarks and Future Work

  24. Vperf Tool Implementation of MAPTs • Per-second frequency of “Interim Test Report” generation • Interaction QoE reported by Vperf tool - based on the progress of the session-agenda

  25. Example – Session Agenda and Network Factor Limits File

  26. Outline • Background • Voice and Video over IP (VVoIP) Overview • Network QoS and End-user QoE in VVoIP • Streaming QoE versus Interaction QoE • Network Fault Events • Multi-Activity Packet Trains (MAPTs) methodology • Participant Interaction Patterns • Traffic Model for MAPTs emulation • Vperf tool implementation of MAPTs • Performance Evaluation • Concluding Remarks and Future Work

  27. MAPTs Emulation at different Dialing Speeds 256 Kbps 384 Kbps 768 Kbps

  28. MAPTs Measurements Evaluation • Increased the number of Type-I and Type-II network fault events in a controlled LAN testbed for a fixed session-agenda • NISTnet network emulator for network fault generation • Recorded Unwanted Agenda-Bandwidth and Unwanted Agenda-Timemeasured by Vperf tool (b) Impact of Type-II Network Fault Events on Unwanted Agenda-Bandwidth • Impact of Type-I Network Fault Events on Unwanted Agenda-Bandwidth

  29. MAPTs Measurements Evaluation (2) (c) Impact of Type-I and Type-II Network Fault Events on Unwanted Agenda-Time

  30. Outline • Background • Voice and Video over IP (VVoIP) Overview • Network QoS and End-user QoE in VVoIP • Streaming QoE versus Interaction QoE • Network Fault Events • Multi-Activity Packet Trains (MAPTs) methodology • Participant Interaction Patterns • Traffic Model for MAPTs emulation • Vperf tool implementation of MAPTs • Performance Evaluation • Concluding Remarks and Future Work

  31. Conclusion • We proposed a Multi-Activity Packet Trains methodology • Mimic participant interaction patterns and video activity levels as affected by network fault events • MAPTs provide real-time objective measurements of Interaction QoE in a large-scale videoconferencing system • Without requiring end-users, actual video sequences, VVoIP appliances • Defined new Interaction QoE Metrics • Unwanted Agenda-Bandwidth, Unwanted Agenda-Time • Implemented MAPTs in an active measurement tool called Vperf and evaluated Interaction QoE measurements on a network testbed

  32. Future Work • Our work is a first-step towards measuring how network fault events impact Interaction QoE in videoconferencing sessions • We considered basic participant interaction patterns and network fault event types • Future scope could include several other participant interaction patterns and network fault event types • E.g. MAPTs for network fault events that cause lack of lip-sync • Human subject testing to more accurately map and validate network fault event types and participant interaction patterns

  33. Thank you for your attention!☺Any Questions?

More Related