CPS 214 Computer Networks and Distributed Systems

1. CPS 214Computer Networks and Distributed Systems • “Live” Video and Audio Streaming • End System Multicast • Analysis of Akamai Workload

2. Presentations • Monday, April 21 • Abhinav, Risi • Bi, Jie • Jason, Michael • Martin, Matt • Amre, Kareem • Wednesday, April 23 • Ben, Kyle • Jayan, Michael • Kshipra, Peng • Xuhan, Yang

3. The Feasibility of Supporting Large-Scale Live Streaming Applications with Dynamic Application End-Points Kay Sripanidkulchai, Aditya Ganjam, Bruce Maggs*, and Hui Zhang Carnegie Mellon University * and Akamai Technologies

4. Motivation • Ubiquitous Internet broadcast • Anyone can broadcast • Anyone can tune in

5. Overlay multicast architectures Router Source Application end-point

6. Infrastructure-based architecture[Akamai] + Well-provisioned Router Source Application end-point Infrastructure server

7. Application end-point architecture[End System Multicast (ESM)] + Instantly deployable + Enables ubiquitous broadcast Router Source Application end-point

8. W Waypoint architecture [ESM] + Waypoints as insurance Router Source Application end-point Waypoint W

9. Sample ESM Broadcasts http://esm.cs.cmu.edu

10. Feasibility of supporting large-scale groups with an application end-point architecture? • Is the overlay stable enough despite dynamic participation? • Is there enough upstream bandwidth? • Are overlay structures efficient?

11. Large-scale groups • Challenging to address these fundamental feasibility questions • Little knowledge of what large-scale live streaming is like

12. Publishers with compelling content need proof that the system works. System has not attracted large-scale groups due to lack of compelling content. Chicken and egg problem

13. The focus of this paper • Generate new insight on the feasibility of application end-point architectures for large scale broadcast • Our methodology to break the cycle • Analysis and simulation • Leverage an extensive set of real-world workloads from Akamai (infrastructure-based architecture)

14. Talk outline • Akamai live streaming workload • With an application end-point architecture • Is the overlay stable enough despite dynamic participation? • Is there enough upstream bandwidth? • Are overlay structures efficient? • Summary

15. Measurements used in this study • Akamai live streaming traces • Trace format for a request [IP, Stream URL, Session start time, Session duration] • Additional measurements collected • Hosts’ upstream bandwidth

16. An Analysis of Live Streaming Workloads on the Internet Kunwadee Sripanidkulchai, Bruce Maggs*, Hui Zhang Carnegie Mellon University and *Akamai Technologies

17. A A A A A A Akamai live streaming infrastructure Source Reflectors Edge servers

18. Extensive traces • 1,000,000 daily requests • 200,000 daily client IP addresses from over 200 countries • 1,000 daily streams • 1,000 edge servers • Everyday, over a 3-month period

19. Largest stream 75,000 x 250 kbps = 18 Gbps!

20. Highlight of findings • Popularity of events [Bimodal Zipf] • Session arrivals [Exponential for short time-scales, time-of-day and time-zone-correlated behavior, LOTS of flash crowds] • Session durations [Heavy-tailed] • Transport protocol usage [TCP rivals UDP] • Client lifetime • Client diversity

21. Request volume (daily) Weekdays Number of requests Weekends Missing logs

22. Audio vs. video Video 7% Most streams are audio. Audio 71% Unknown 22%

23. Stream types • Non-stop (76%) vs. short duration (24%) • All video streams have short duration • Smooth arrivals (50%) vs. flash crowds (50%) • Flash crowds are common

24. Client lifetime • Motivating questions • Should servers maintain “persistent” state about clients (for content customization)? • Should clients maintain server history (for server selection problems)? • Want to know • Are new clients tuning in to an event? • What is the lifetime of a client?

25. Analysis methodology • Windows media format • Player ID field to identify distinct users • Birth rate = Number of new distinct users Total number of distinct users

26. Weekends Xmas Weekdays Daily new client birth rate • New client birth rate is 10-100% across all events. • For these 2 events, birth rate is 10-30%.

27. One-timers: tune in for only 1 day In almost all events, 50% of clients are one-timers!

28. Client lifetime (excluding one-timers) y = 3x y = x For most events, average client lifetime is at least 1/3 of the event duration.

29. Client lifetime • Motivating questions • Should servers maintain “persistent” state about clients (for content customization)? Any state should time-out quickly because most clients are one-timers. • Should clients maintain server history (for server selection problems)? Yes, recurring clients tend to hang around for a while.

30. Where are clients from? Number of IP Addresses Countries Clients are from over 200 countries. Most clients are from the US and Europe.

31. Analysis methodology • Map client IP to location using Akamai’s EdgeScape tool • Definitions • Diversity index = Number of distinct ‘locations’ that a stream reaches • Large streams are streams that have a peak group size of more than 1,000 clients

32. Many small streams reach more than half the world! Almost all large streams reach more than half the world. Time zone diversity

33. Client diversity • Motivating questions • Where should streaming servers be placed in the network? Clients are tuning in from many different locations. • How should clients be mapped to servers? For small streams which happen to have a diverse set of clients, it may be too wasteful for a CDN to map every client to the nearest server.

34. Summary • Publishers are using the Internet to reach a wider audience than traditional radio and TV • Interesting observations • Lots of audio traffic • Lots of flash crowds (content-driven behavior) • Lots of one-timers • Lots of diversity amongst clients, even for small streams

35. Nice pictures…see paper for details Percentage of requests Quicktime Real Windows media

36. Abandon all hope, ye who enter here.

37. Talk outline • Akamai live streaming workload • With an application end-point architecture • Is the overlay stable enough despite dynamic participation? • Is there enough upstream bandwidth? • Are overlay structures efficient? • Summary

38. Not stable More stable Departing hosts have no descendants Stable nodes at the top of the tree X When is a tree stable? Stable nodes X Less stable nodes X Interruptions Time Ancestor leaves

39. 15% stay longer than 30 minutes (heavy-tailed) 45% stay less than 2 minutes! Extreme group dynamics

40. Stability evaluation: simulation • Hosts construct an overlay amongst themselves using a single-tree protocol • Skeleton protocol of the one presented in the ESM Usenix ’04 paper • Findings are applicable to many protocols • Goal: construct a stable tree • Parent selection is key • Group dynamics from Akamai traces (join/leave) • Honor upstream bandwidth constraints • Assign degree based on bandwidth estimation

41. Join Join IP1 IP2 ...

42. Probe and select parent IP2 IP1 IP2 ... IP1

43. Probe and select parent Parent selection algorithms • Oracle: pick a parent who will leave after me • Random • Minimum depth (select one out of 100 random) • Longest-first (select one out of 100 random)

44. Parent leave X Host leaves

45. Parent leave ? Host leaves All descendants are disconnected

46. Find new parent Host leaves All descendants are disconnected All descendants probe to find new parents

47. Mean interval between ancestor change Number of descendants of a departing host Stability metrics Interruptions X Time Ancestor leaves

48. Stability of largest stream Longest-first Random Min depth Oracle: there is stability!

49. Is longest-first giving poor predictions? Oracle, ~100% no descendants Longest-first, 91% Min depth, 82% Random, 72%

50. Stability of 50 large-scale streams Longest-first Percentage of sessions with interval between ancestor change < 5 minutes Random Min depth Oracle There is stability! Of the practical algorithms, min depth performs the best.