1 / 40

“Evalvid-RA” Simulation of rate adaptive video

“Evalvid-RA” Simulation of rate adaptive video. TTM4142 Networked Multimedia Systems Arne Lie, SINTEF ICT November 6, 2008. Overview. Why Evalvid-RA How to compress video How to simulate video transmission How to simulate rate adaptive video Evalvid-RA architecture How to use Evalvid-RA.

lola
Download Presentation

“Evalvid-RA” Simulation of rate adaptive video

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. “Evalvid-RA”Simulation of rate adaptive video TTM4142 Networked Multimedia Systems Arne Lie, SINTEF ICT November 6, 2008

  2. Overview • Why Evalvid-RA • How to compress video • How to simulate video transmission • How to simulate rate adaptive video • Evalvid-RA architecture • How to use Evalvid-RA TTM4142:: Arne Lie, Evalvid-RA

  3. Objectives • Congestion control for media:: highest possible perceived quality! • Avoid persistent long queues • low latency (media sender, network queues, media receiver) • low drop probability • Bandwidth: • Fair bandwidth • Avoid unnecessary large rate reduction • Grab available excess bandwidth • Network simulation of media:: requirements • Run traffic with the right characteristics • Use source models, or • Use trace driven simulation (i.e. genuine video traffic) • Perceived quality: need real media! • Evalvid tool-set • But we need “online” rate adaptive trace simulations TTM4142:: Arne Lie, Evalvid-RA

  4. Example with congestion Link utilization ~6Mbps 100% Link capacity 15Mbps ~6Mbps ~6Mbps 2s 4s 6s 8s 10s time MPEG-4 “Foreman” ~700kbps TTM4142:: Arne Lie, Evalvid-RA

  5. The throughput using best effort Internet Only 15/18.7=80.2% of the packets can survive after congestion takes place: 20% packet loss for all flows! Or adapt the rate with 20% Link utilization 100% Link capacity 15Mbps ~4.8Mbps ~6Mbps ~4.8Mbps ~6Mbps ~4.8Mbps 2s 4s 6s 8s 10s time MPEG-4 “Foreman” still ~700kbps TTM4142:: Arne Lie, Evalvid-RA

  6. Comparison of TailDrop, P-AQM and P-AQM with “ECF CC” TTM4142:: Arne Lie, Evalvid-RA

  7. Main challenges • Encoding/decoding of video is CPU demanding • We want to be able to simulate multiple video traffic flows in mixed traffic scenarios on a single computer! • How to keep complexity low? • We want to be able to play resulting video so that perceptual quality can be determined • How to avoid “online” encoding/decoding? TTM4142:: Arne Lie, Evalvid-RA

  8. 2005: What was available • Trace driven simulation need trace files from real sources, e.g. • http://www-tkn.ee.tu-berlin.de/research/trace/ltvt.html • Only the frame SIZES and timing is used, not the content • or synthetic traffic that models real traffic very closely • e.g. GenSyn http://www.item.ntnu.no/~poulh/GenSyn/gensyn.html • Evalvid tools from http://www.tkn.tu-berlin.de/research/evalvid/ • real traces, and media is re-assembled after network simulation for visual inspection and PSNR calculation (Jirka Klaue) • Evalvid interface to ns-2 (Ke Chih-Heng) • http://hpds.ee.ncku.edu.tw/~smallko/ns2/Evalvid_in_NS2.htm • but rate adaptive media will change depending on network state… TTM4142:: Arne Lie, Evalvid-RA

  9. Video coding • Intra-frames (key-frames) • still images self-contained • used at scene changes • Predicted-frames (P-frames) • uses motion-estimation • Bidirectional frames (B-frames) • uses motion-estimation both forward and backward in time • must be relative to an anchor picture (I- or P-frame) GOP TTM4142:: Arne Lie, Evalvid-RA

  10. Hybrid encoding: transform (spatial) + prediction (time) TTM4142:: Arne Lie, Evalvid-RA

  11. Quantization of 8x8 pixel block quantization steps (SQ) after quantization performed TTM4142:: Arne Lie, Evalvid-RA

  12. Motion vector, prediction error • Source: Eckehard Steinbach: Internet Media Streaming TTM4142:: Arne Lie, Evalvid-RA

  13. Scalable media • change either (video / audio) • frame rate / sample rate (temporal) • frame size / sample size (spatial) • compression quantization Q (quality) • =quantiser_scale in MPEG-4 • or a combination • Most players/decoders don’t respond (correctly) to changes in frame size and frame rate •  change of the Q-value (=quantiser_scale) is easiest • the Q-value actually normally change each frame, or even each macro block (video) • but how to avoid doing this “live” in the network simulation? Differences between live adaptation and pre-stored media with adaptation possibilities (scalable video coding) TTM4142:: Arne Lie, Evalvid-RA

  14. Rate controllers varies Q • Adjust output rate according to a bit rate budget on time average and variability constraints • leaky bucket • CBR: constant bit rate • each GOP has the same number of bits (or bit/s) • Q changes from macro block to macro block • Cost: algorithmic delay, variable quality • VBR: variable bit rate • allows for more variability • Q changes less: more stable quality • Quality based (“VBR open loop”, constant Q) • rate totally dependent on content TTM4142:: Arne Lie, Evalvid-RA

  15. VBR open loop @ Q=2 TTM4142:: Arne Lie, Evalvid-RA

  16. Objective quality at Q=2: PSNR • Constant Q gives ~ constant quality TTM4142:: Arne Lie, Evalvid-RA

  17. Rate dependability on Q TTM4142:: Arne Lie, Evalvid-RA

  18. Rate controller objectives • Limits the rate fluctuations & have an average rate constraint, by varying the quantization value Q • at each macro block • at each frame, • or at each GOP • If Congestion Control is applied • the rate controller must have adaptable average rate constraint! • Problem: the rate controller must run at simulation time! TTM4142:: Arne Lie, Evalvid-RA

  19. No rate controller (VBR open loop) Stefan Football • Bit rate too variable to control • Has Long Range Dependence (LRD) News Akiyo Paris TTM4142:: Arne Lie, Evalvid-RA

  20. Rate controller (VBR constrained) TTM4142:: Arne Lie, Evalvid-RA

  21. Adaptive rate controller (VBR constrained) • Red line: no adaptive rate control • Blue line: adaptive rate reduces the bit rate at ~40 second TTM4142:: Arne Lie, Evalvid-RA

  22. Quality of received video • PSNR of video flows examined (with delay constraints) • P-AQM with highest score, and with Statistical Multiplexing Gain • TFRC gains on running over networks with AQM TFRC 1: RED w/ ECN TFRC 2: RED w/ dropping TFRC 3: FIFO 150ms delay constraint TTM4142:: Arne Lie, Evalvid-RA

  23. 32 videos @ 150 ms e2e delay constraint, 32 FTP, Web traffic Original quality 32 video flows (600 kbit/s) Adapt to 400 kbit/s Adapt to 400 kbit/s 600 kbit/s 400 kbit/s 600 kbit/s 400 kbit/s TFRC supported adaptation P-AQM supported adaptation TTM4142:: Arne Lie, Evalvid-RA

  24. How to avoid having “online” encoder to follow the adaptive feedback • CBR changes Q at macro block granularity • too detailed for frame size trace files! • VBR changes Q at frame or GOP granularity • Yes! • “SVBR” (shaped VBR) by Hamdi/Roberts/Rolin ’97 • change Q at GOP scale to constrain video to LB(r,b) constraint • r: average video rate (=leaky bucket rate) • b: bucket size (to allow variability) • very simple, no extra delay • my modification: variable r TTM4142:: Arne Lie, Evalvid-RA

  25. Hamdi’s SVBR leaky bucket controller encoder packetizer to network Q calc. Q next GOP b x r TTM4142:: Arne Lie, Evalvid-RA

  26. How to map r to Q • rate x Q product almost independent on Q • dependent only on content complexity •  at start of any new GOP, assuming complexity change smoothly from GOP to GOP • r (bits/s)  R(k+1) (bits/GOP) using a formula (PhD Thesis) • Q(k+1) = R(k)*Q(k)/R(k+1) (if video complexity does not change) • for stored media, next GOP complexity is known a priori TTM4142:: Arne Lie, Evalvid-RA

  27. Pseudo code from rate adapt algorithm TTM4142:: Arne Lie, Evalvid-RA

  28. Evalvid-RA solutions • Multiple trace files • one per Q-value • Q=[2,3,4,…,31] (ffmpeg) • make SVBR calculate Q(k+1): select GOP(k+1) trace • This requires fixed GOP sizes! • LB(r,b) parameters change at feedback event • but the new Q-value is not used before start of next GOP • Received video file must be assembled • using trace of actual Q(i)-values used, and • multiple *.m4v files TTM4142:: Arne Lie, Evalvid-RA

  29. Long Range Dependence (LRD) • Garrett & Willinger ’94: VBR video traffic is self-similar • the autocorrelation (k) function decays slowly at increasing lag • makes buffer dimensioning & high link utilization very difficult • The cause of LRD: scene complexity changes! • many papers on video characterization (GOP scale, frame scale) • very little related to what kind of rate controller in use! • Hamdi showed in his thesis that • a stream satisfying a LB(r,b) constraint, where r equals the traffic average rate, is not self similar TTM4142:: Arne Lie, Evalvid-RA

  30. rate controllers limits the rate variance… TTM4142:: Arne Lie, Evalvid-RA

  31. (k) of “concatenated” video (GOP) • Positive correlations at lag k poses long bursts of time duration k TTM4142:: Arne Lie, Evalvid-RA

  32. Evalvid-RA: overview TTM4142:: Arne Lie, Evalvid-RA

  33. The tool-set overviewPre-process • ffmpeg -s cif -r 30 -i video.yuv -vcodec mpeg4 -4mv -g 12 –flags sgop -sc_threshold 20000 -qscale 8 -s cif -r 30 -y video_Q12.m4v • mp4.exe -send <IP address> <port #> <MTU> <fps> video_Q12.m4v > st_video_Q12.txt TTM4142:: Arne Lie, Evalvid-RA

  34. The tool-set overview (cont.)Network Simulation • ns-2: evalvid_rateadapt.tcl • modified ns-2 interface to “Evalvid-RA” & adaptive SVBR responding to P-AQM feedback • Tcl init-function makes • video2.dat (frames all Q) • read into memory • used by all nodes sending the same media (different timing) • gop_size.dat (GOP size all Q) • used by et_ra.exe • sd_be_* stores e.g. actual Q used TTM4142:: Arne Lie, Evalvid-RA

  35. The multi-rate trace file (multi Q) • video2.dat: <time s> <bytes Q=2> <type> <MTU> <Q=3> <Q=4>… <Q=31> TTM4142:: Arne Lie, Evalvid-RA

  36. The tool-set overview (cont.)Post-process • et_ra.exe • modified Evalvid original et.exe • Reads packet Tx and Rx trace files • Finds used Q • Reads video2.dat for frame sizes and types • Reads gop_size.dat to assist assembling the resulting MPEG-4 file • ffmpeg to decode to YUV • fixyuv_ra.exe: takes e2e delivery time constraints into account  resulting video file (*.yuv) • psnr.exe: compare decoded YUV to original TTM4142:: Arne Lie, Evalvid-RA

  37. et_ra.exe (Evaluate Trace, rate Adaptive) frame 1: • et_ra.exe: • original et.exe: packet 1 packet 2 packet 3 frame 1 GOP1 GOP1 GOP1 GOP1 frame 2 GOP1 GOP1 GOP1 GOP1 frame 3 GOP1 GOP2 GOP2 frame 4 GOP2 GOP2 frame 5 GOP2 GOP2 GOP3 GOP2 GOP3 GOP3 GOP3 GOP3 GOP3 GOP3 GOP4 GOP4 GOP4 GOP4 GOP4 GOP4 GOP4 GOP5 GOP5 GOP5 GOP5 GOP5 GOP5 GOP5 … TTM4142:: Arne Lie, Evalvid-RA

  38. Limitations of this implementation • GOP time scale rate adaptation • Hamdi confirms that SVBR could be modified to frame scale • Fixed GOP size • live encoders could start a new GOP (i.e. next frame being I-frame) at a feedback event! • relaxation will make distortions? • error concealment (packet loss) • FRAME mode vs. PACKET mode considerations • ffmpeg drops first frame after frame marked with “loss” • No audio yet • limitation in mp4.exe tool TTM4142:: Arne Lie, Evalvid-RA

  39. Research usage of Evalvid-RA • Simulate many flows, coming from many sources, all of them rate adaptive • Have different media sources, not only one • Wireless rate adaptive multimedia • Different congestion control algorithms • Self-limited sources and their actual bandwidth • friendliness (towards TCP, UDP, DCCP, etc.) • Different queuing systems (FIFO, AQMs, QoS/DiffServ e.g.) • Investigate the removal of LRD, or not? • trade latency for loss (short queues) • how to inject new flows • new initiatives for rate adaptation incentives • vary the sources rates • … TTM4142:: Arne Lie, Evalvid-RA

  40. Evalvid-RA lab • Software • Windows/Cygwin (Linux on Win32) • ns-2 w/ Evalvid-RA • ffmpeg & video inspection programs • Script files • The pre-process is already performed • To do: • Modify TCL script / run ns-2 simulation with selected parameters • Run post-process and inspect video quality / statistics TTM4142:: Arne Lie, Evalvid-RA

More Related