PlanetLab: An Open Laboratory for Introducing Disruptive Technology into the Internet

1. PlanetLab: An Open Laboratory for Introducing Disruptive Technology into the Internet David Culler University of California, Berkeley http://www.cs.berkeley.edu/~culler

2. Q: From where will the next 10 major advances in the internet emerge? 2

3. Innovator’s Dilemma • The Internet is an enormous success story • commercially • impact on our daily lives • global reach • Success has an unexpected cost: ossification • difficult to deploy disruptive technologies • correct vulnerabilities • introduce new capabilities 3

4. A: ideas honed through process of “rough consensus and running code” 4

5. Sydney UA Toronto Taiwan Nizhny CUHK Berlin Belas UCL HUJ …PlanetLab Uppsala Lancaster Towards 1,000 widely-distributed nodes • today: 360 machines, 148 sites, 20 countries, • 110 networks • at edge sites and network cross-roads • Universities, Companies, Nat’l labs UBC Copenhagen UW Cambridge WI Chicago UPenn Harvard Utah Intel Seattle Intel MIT Intel OR Intel Berkeley Cornell CMU Bologna ICIR Princeton UCB St. Louis Columbia Duke UCSB Washu KY UCLA GIT Rice UCSD UT ISI Canterbury 5

6. Today’s Internet Best-Effort Packet Delivery Service Limitations • the Internet is “opaque” making it difficult to adapt to network conditions • applications typically split into two pieces: client and server 6

7. Internet services 7

8. Tomorrow’s Internet Collection of Planetary-Scale Services Opportunities • multiple vantage points • anomaly detection, robust routing • Adapt to load, failures, $ • Proximity: close wherever you are • low latency, high bandwidth, predictable, reliable • content distribution, data fusion • multiple, independent domains • survivable storage 8

9. Towards Planetary-Scale Services • CDN and P2P just tip of the iceberg • Research Community developing the architectural building blocks to enable many kinds of distributed services • scalable translation, • dist. storage, • dist. events, • instrumentation, • management 9

10. Key missing element – real hands-on experience • Researchers had no vehicle to try out their next n great ideas in this space • Lot’s of simulations • Lot’s of emulation on large clusters • emulab, millennium, modelnet • Lot’s of folks calling their 17 friends before the next deadline • RON testbed • but not the surprises and frustrations of experience at scale to drive innovation 10

11. Quick Growth • “Underground” meeting March 2002 • Intel seeds effort • First 100 nodes, Operational support • First node up July 2002 • By SOSP (March 2003) 25% of accepted papers refer to PlanetLab • Each following conference has seen dramatic load • OSDI • NDSI 11

12. Some Example Services 12

13. Berkeley: OceanStore `RAID’ distributed over the whole Internet 13

14. Washington - ScriptRoute • Internet measurement tool • open, unauthenticated, community measurement infrastructure vs closed, engineered service UW Google 14

15. CCC BBB A A A A C C C C AA B B B B Princeton: CoDeeN Open Content Distribution Network 15

16. Berkeley – Internet Indirection Infrastructure • Multicast, anycast, mobility, … src dst Put v,ID Get ID 16

17. NetBait Serendipity • Brent Chun built a simple http server on port 80 to explain what planetlab was about and to direct inquiries to planet-lab.org • It also logged requests • Sitting just outside the firewall of ~40 universities... • the worlds largest honey pot • the number of worm probes from compromised machines was shocking • imagine the the epidemiology • see netbait.planet-lab.org 17

18. One example • The monthly code-red cycle in the large? • What happened in March 2003? 18

19. No, not Iraq • A new voracious worm appeared and displaced the older Code Red 19

20. Intel Netbait - March 2003 20

21. Internet Public Health 21

22. Key Concept: Overlay networks 22

23. Overlay network routing 23

24. Evolving the Internet • Add a new layer to the network architecture • overlay networks • Challenge • how to innovate & deploy at scale overlay • purpose-built virtual networks that use the existing Internet for transmission • the Internet was once deployed as an overlay on top of the telephony network Internet 24

25. Sydney UA Toronto Taiwan Nizhny CUHK Berlin Belas UCL HUJ PlanetLab is… Uppsala Lancaster UBC Copenhagen UW Cambridge WI Chicago UPenn Harvard Utah Intel Seattle Intel MIT Intel OR Intel Berkeley Cornell CMU Bologna ICIR Princeton UCB St. Louis Columbia Duke UCSB Washu KY UCLA GIT Rice UCSD UT ISI Canterbury More than a bunch of machines spread around the world 25

26. PlanetLab is… A common software architecture • OS running on each node • Remote installation mechanism • Remote management capability • Central account management service 26

27. design deploy measure PlanetLab is… A network testbed • 450 active research projects • experiment at scale • experiment under real-world conditions • potential for real workloads and users A deployment platform • 15 continuously running services 27

28. PlanetLab is… A microcosm of the next Internet • Fold services back into PlanetLab • evolve core technologies to support overlays and slices • discover common sub-services • Long-term goals • become the way users interact with the Internet • define standards that support multiple “PlanetLabs” 28

29. Software Architecture • Distributed virtualization • slice a network of virtual machines • isolation • isolate services from each other • protect the Internet from PlanetLab • Unbundled Management • OS defines only local (per-node) behavior • global (network-wide) behavior implemented by services • multiple competing services running in parallel • shared, unprivileged interfaces 29

30. Slices 30

31. Per-Node View Node Mgr Local Admin VM1 VM2 VMn … Virtual Machine Monitor (VMM) 31

32. Virtualization • Hypervisors (e.g., VMWare) • don’t scale well • don’t need multi-OS functionality • Paravirtualization (e.g., Xen, Denali) • not yet mature • Virtualize at system call interface (e.g., Jail, Vservers) • reasonable compromise • doesn’t provide the isolation that hypervisors do • Unix processes • isolation is problematic • Java Virtual Machine • too high-level 32

33. PlanetLab VMM • Linux: significant mind-share • Vservers: virtualizes at the system call interface • each vserver runs in its own security context • private UID/GID name space • limited superuser capabilities (e.g., no CAP_NET_RAW) • uses chroot for file system isolation • scales to 1000 vservers per node (29MB each) • Plkmod: enforces isolation • processor and link schedulers, disk quotas • network virtualization • safe raw sockets (UDP, TCP, ICMP, GRE) • rate limits exceptional packets, unique addrs/ports 33

34. Infrastructure Services • Brokerage Services • buy, sell, trade, pool resources • least mature / greatest potential (market-based) • Environment Services • create / maintain programming environments • challenge: space efficiency • Monitoring Services • resource discovery • network / service health • adaptive applications • Routing Underlay • discover network topology 34

35. PLC: Brokerage Service create_slice(name, credentials) delete_slice(name, credentials) assign_owners(name, ssh_keys[ ], credentials) set_state(name, boot_script, credentials) set_resources(name, share, credentials) instantiate_slice(name, nodes[ ], credentials) rcap = acquire(rspec) bind(name, rcap) 35

36. Stork: Environment Service • Defines a boot_script to be run when VM inits • invokes Stork • Maintains a repository of packages • those registered by a client service + dependencies • Shared package area on each node • retrieves one copy of each required package • installation scripts run in context of client VM • Optionally… • manages user accounts for client slices • establishes tunnels to neighboring nodes 36

37. Monitoring Service – Sophia, Pier,… Distributed query processor Information Plane observe analyze react A A A S S S … … 37

38. Pluto: Routing Underlay Expense Service Overlay Networks Library of Routing Services Primitives Topology Probing Kernel Raw Topology Information Scope 38

39. Representative Sample of the Internet? 39

40. Global System Architecture Slices, management, distribution, Network measurement Scriptroute, PlanetProbe, I3, etc. Application-level multicast ESM, Scribe, TACT, etc. Distributed Hash Tables Chord, Tapestry, Pastry, Bamboo, etc. Wide-area distributed storage Oceanstore, SFS, CFS, Palimpsest, IBP Resource allocation Sharp, Slices, XenoCorp, Automated contracts Distributed query processing PIER, IrisLog, Sophia, etc. Content Dist. Networks CoDeeN, ESM, UltraPeer emulation, Gnutella mapping Management and Monitoring Ganglia, InfoSpect, Scout Monitor, BGP Sensors, etc. Overlay Networks RON, ROM++, ESM, XBone, ABone, etc. Virtualization and Isolation Xen, Denali, VServers, SILK, Mgmt VMs, etc. Router Design implications NetBind, Scout, NewArch, Icarus, etc. Testbed Federation NetBed, RON, XenoServers Etc., etc., etc. A Rich Research Agenda 40

41. Confluence of Technologies • Cluster-based scalable distribution, remote execution, management, monitoring tools • UCB Millennium, OSCAR, ..., Utah Emulab, ... • CDNS and P2Ps • Gnutella, Kazaa, ... • Proxies routine • Virtual machines & Sandboxing • VMWare, Janos, Denali,... web-host slices (EnSim) • Overlay networks becoming ubiquitous • xBone, RON, Detour... Akamai, Digital Island, .... • Service Composition Frameworks • yahoo, ninja, .net, websphere, Eliza • Established internet ‘crossroads’ – colos • Web Services / Utility Computing • Authentication infrastructure (grid) • Packet processing (layer 7 switches, NATs, firewalls) • Internet instrumentation 41

42. Current Institutions (partial) Princeton University Purdue University Rensselaer Polytechnic Inst. Rice University Rutgers University Stanford University Technische Universitat Berlin The Hebrew Univ of Jerusalem University College London University of Arizona University of Basel University of Bologna University of British Columbia UC Berkeley UCLA UC San Diego UC Santa Barbara University of Cambridge University of Canterbury University of Chicago University of Illinois University of Kansas University of Kentucky University of Maryland University of Massachusetts University of Michigan University of North Carolina University of Pennsylvania University of Rochester USC / ISI University of Technology Sydney University of Tennessee University of Texas University of Toronto University of Utah University of Virginia University of Washington University of Wisconsin Uppsala University, Sweden Washington University in St Louis Wayne State University Academia Sinica, Taiwan Boston University Caltech Carnegie Mellon University Chinese Univ of Hong Kong Columbia University Cornell University Datalogisk Institut Copenhagen Duke University Georgia Tech Harvard University HP Labs Intel Research Johns Hopkins Lancaster University Lawrence Berkeley Laboratory MIT Michigan State University National Tsing Hua Univ. New York University Northwestern University 42

43. What Planet-Lab is about? • Create the open infrastructure for invention of the next generation of wide-area (“planetary scale”) services • post-cluster, post-yahoo, post-CDN, post-P2P, ... • Potentially, the foundation on which the next Internet can emerge • think beyond TCP/UDP/IP + DNS + BGP + OSPF... as to what the net provides • building-blocks upon which services and applications will be based • “the next internet will be created as an overlay in the current one” (NRC) • A different kind of network testbed • not a collection of pipes and giga-pops • not a distributed supercomputer • geographically distributed network services • alternative network architectures and protocols • Focus and Mobilize the Network / Systems Research Community to define the emerging internet 43

44. Join the fun ... www.planet-lab.org • It is just beginning • towards a representative sample of the internet (1,000s) • PlanetLab consortium recently formed • Princeton, UCB, UWash • Intel, HP, Google, … • Hands-on experience with wide-area services at scale is mothering tremendous innovation • nothing “just works” in the wide-area at scale • Joining forces we all learn more Thanks 44