An Experimental Evaluation of Rate Adaptation Algorithms in Adaptive Video Streaming over HTTP

1. An Experimental Evaluation of Rate Adaptation Algorithms in Adaptive Video Streaming over HTTP Saamer Akhshabi, Constantine Dovrolis Georgia Institute of Technology Ali C. Begen Cisco Systems February 3, 2011

2. Outline • Overview of adaptive video streaming • Experimental methodology • Rate adaptation under available bw variations • Smooth Streaming player • Netflix player • OSMF player • Live streaming • Competition between multiple players • Conclusions

3. Outline • Overview of adaptive video streaming • Experimental methodology • Rate adaptation under available bw variations • Smooth Streaming player • Netflix player • OSMF player • Live streaming • Competition between multiple players • Conclusions

4. Other Video Streaming Approaches • HTTP Progressive Download

5. Other Video Streaming Approaches • RTSP and RTP

6. Other Video Streaming Approaches • RTMP

7. Adaptive video streaming over HTTP • Several players support it • Several commercial content providers use it • Microsoft Smooth Streaming Player • Netflix Player • Adobe OSMF/Zeri Player • Apple Player

8. HTTP Adaptive Video Streaming

9. HTTP Adaptive Video Streaming: Manifest File and Fragments

10. HTTP Adaptive Video Streaming: HTTP Request and Response

11. Outline • Overview of adaptive video streaming • Experimental methodology • Rate adaptation under available bw variations • Smooth Streaming player • Netflix player • OSMF player • Live streaming • Competition between multiple players • Conclusions

12. Objectives • Examine how commercial adaptive players react to available bw variations • Persistent variations • Short-term variations (spikes) • We control the available bw using Dummynet • In parallel, we collect packet traces at the player • Identify requested bitrates & measure throughput • Also, we estimate playback buffer size (in seconds) • Use fragment timestamps (explained later)

13. Experimental Methodology

14. Outline • Overview of adaptive video streaming • Experimental methodology • Rate adaptation under available bw variations • Smooth Streaming player • Netflix player • OSMF player • Live streaming • Competition between multiple players • Conclusions

15. Smooth Streaming Player

16. Smooth Streaming Player • Sample HTTP Request: • GET /mediadl/iisnet/smoothmedia/Experience/BigBuckBunny720p.ism/QualityLevels(2040000)/Fragments(video=400000000) HTTP/1.1

17. Smooth Streaming PlayerBuffering and Steady State • One fragment per HTTP request • No HTTP pipelining • Two states: • Buffering state • Request fragments as fast as possible • Steady-state • Request new fragment every T seconds

18. Smooth Streaming PlayerBehavior under Unrestricted Available Bandwidth • Average throughput: running average of instantaneous 2-sec TCP throughput measurements. • Fragment throughput: per-fragment throughput measurement

19. Smooth Streaming PlayerBehavior under Unrestricted Available Bandwidth • Two successive, say video, requests sent at times t1 and t2 (t1<t2) with timestamps t‘1 and t‘2 (t‘1 < t‘2) respectively • The playback buffer duration (in seconds) for video at time t2 is estimated as: B(t2) = B(t1) - (t2-t1) + (t‘2 - t‘1)

20. Smooth Streaming PlayerNotice: Important variation in fragment sizes • The video fragment size varies considerably (VBR encoding) • The rate can vary widely around the nominal bitrate (2.75 Mbps) • This can be a significant problem for adaptive video streaming!

21. Smooth Streaming PlayerBehavior under persistent changes in available bandwidth • Rate adaptation occurs after long delays • The player estimates available bw using a running average of the per-fragment TCP throughput measurements

22. Smooth Streaming PlayerPlayback buffer size under persistent changes in the available bandwidth • Playback buffer size decreases when available bandwidth is less than the requested bitrate • Playback buffer size increases when player goes into “buffering state” requesting fragments as fast as possible • Together with switching to bitrate < available bw

23. Smooth Streaming PlayerBehavior under short term available bandwidth variations (negative spikes) • The client reacts to some of the spikes by switching to lower bitrate too late • Stays at that bitrate for long after the spike has passed

24. Outline • Overview of adaptive video streaming • Experimental methodology • Rate adaptation under available bw variations • Smooth Streaming player • Netflix player • OSMF player • Live streaming • Competition between multiple players • Conclusions

25. Netflix Player

26. Netflix Player • Sample HTTP Request: • GET /sa2/946/1876632946.wmv/range/2212059-2252058?token=1283923056\_d6f6112068075f1fb60cc48eab59ea55\&random=1799513140 HTTP/1.1

27. NetflixPlayerBehavior under Unrestriced Available Bandwidth • Player accumulates 5-min playback buffer!

28. Netflix PlayerBehavior under persistent changes in the available bandwidth • Initially, the player requests fragments at all possible bitrates • Occasionally, the player requests higher bitrate than available bw! • Utilize large playback buffer size to optimize video quality

29. Smooth Streaming PlayerBehavior under short term available bandwidth variation with positive spikes • Netflix appears to be more aggressive than Smooth Streaming in requesting higher bitrates

30. Outline • Overview of adaptive video streaming • Experimental methodology • Rate adaptation under available bw variations • Smooth Streaming player • Netflix player • OSMF player • Live streaming • Competition between multiple players • Conclusions

31. Adobe OSMF/Zeri Player

32. Adobe OSMF Player • Sample HTTP Request: • GET /content/inoutedit-mbr/inoutedit\_h264\_3000Seg1-Frag5 HTTP/1.1

33. OSMF PlayerBehavior under persistent changes in the available bandwidth • The client fails to select highest possible bitrate for the given available bw

34. OSMF PlayerBehavior under persistent changes in the available bandwidth • Also, player often oscillates between bitrates • Mostly lowest and highest bitrates

35. Outline • Overview of adaptive video streaming • Experimental methodology • Rate adaptation under available bw variations • Smooth Streaming player • Netflix player • OSMF player • Live streaming • Competition between multiple players • Conclusions

36. Smooth Live Streaming • No major differences with on-demand streaming • Player starts streaming with 8-seconds delay • CableTV networks often use 7-second “profanity delay”

37. Smooth Live Streaming Buffer Fill Level • Playback delay increases over time whenever playback buffer gets empty • Player does not skip fragments

38. Outline • Overview of adaptive video streaming • Experimental methodology • Rate adaptation under available bw variations • Smooth Streaming player • Netflix player • OSMF player • Live streaming • Competition between multiple players • Conclusions

39. Two Smooth Streaming players compete • Players keep oscillating between rates even when available bw is constant • Synchronization can cause simultaneous bitrate drops

40. Two Smooth Streaming players compete • Fairness issue: one stream may get much lower bitrate than the other

41. Outline • Overview of adaptive video streaming • Experimental methodology • Rate adaptation under available bw variations • Smooth Streaming player • Netflix player • OSMF player • Live streaming • Competition between multiple players • Conclusions and future work

42. Summary of the key differences between players • Smooth Streaming player • Playback buffer size of 10s of seconds • Conservative in selecting bitrate (< available bw) • Netflix player • Playback buffer size of few minutes • More aggressive than Smooth player (often > available bw) • Zeri player • Erratic bitrate selection • Under Development (?)

43. Weaknesses in adaptation logic • Smooth Streaming player • Large delay in responding to persistent available bw variations • Erratic rate adaptations under short-term variations • Oscillations and unfairness when multiple players compete • Netflix player • Large delay in responding to persistent available bw variations • Erratic rate adaptations under short-term variations • Requires large playback buffer • Zeri player • Adaptation logic seems broken

44. Future Work • Continue the analysis of commercial players to understand how they work • And identify weaknesses • Expand study of multi-player competition • Design and implement an adaptive steaming adaptation logic that can address all previous issues