Sneakernet : Clouds with Mobility

1. Sneakernet: Clouds with Mobility Kenneth Church (Johns Hopkins) James Hamilton (Amazon)

2. Clouds with Mobility • Standard view of Clouds • Big Datacenters with no mobility, • But mobility is a big opportunity • Moore’s Law: • Everything is getting better • But at different rates • Mobility Gap (18x per decade) • Moveable media (flash/disk) >> Wires • Sneakernet Alternatives to WANs • Media on the move in: • Shipping Containers • Car Trunks • Laptops • Cell Phones

3. WANs v. SneakerNet:Throughput, Cost, Latency, Convenience • For tiny payloads (MBs)  • Wires (WANs) • For modest payloads (GBs/TBs)  • Post Office (Jim Gray  http://aws.amazon.com/importexport) • For serious payloads (PBs)  • Man with a Shipping Container (3 PBs/day) • Like a man with a van • Benchmarks: • 5.3 containers ≈ 2008 AT&T Backbone (16 PBs/day) • ½ container is big enough for the Internet Archive

4. ½ container (3 PBs) is big enough for Internet Archive(1 man can drive the Internet a day’s drive in a day)

5. The Mobility Gap:Moveable Media >> Wires 1024x/57x = 18x per decade

6. The Mobility Gap  SneakerNet • In the limit (time, payload size) • Eventually (when the gap becomes wide enough) • For large enough payloads • SneakerNet >> Wires • Throughput, Cost, Latency, Convenience • More obvious: Throughput, Cost • Less obvious: Latency, Convenience • Intuitively, wires (speed of light) are fast (and trucks are sloooow) • But for large payloads (TBs), • Trucks are faster (and more convenient) • A truck can move the Internet (3 PBs) a day’s drive in a day: Truck ≈ ¼Tbps >> WAN ≈ 10 Gbps • Especially for a one-shot ad hoc transfer • It is quicker and easier to hire a man with a shipping container • Than to provision new network capacity • Provisioning new network capacity (private lines) • Not quick (or convenient) • Usually requires long-term commitments • Subject to availability (there isn’t that much capacity; AT&T transfers just 16 PBs/day in 2008)

7. The Mobility Gap SneakerNet(More Consequences) • Eventually, when disk becomes infinitely more plentiful than networking, • Do whatever it takes to alleviate network bottlenecks • Even if doing so consumes vast quantities of disk • Cache everything everywhere forever

8. Gray’s Legacy: Amazon Web Serviceshttp://aws.amazon.com/importexportThroughput, Cost, Latency, Convenience

9. Jim Gray’s Motivation for SneakerNet • Gray’s Question: • What is the best way to move a terabyte from place to place? • The Next Generation Internet (NGI) promised • Gps desktop-to-desktop by 2000 • So, if you have NGI, • then 1TB transfer  8k seconds (a few hours) • Unfortunately, most of us are still waiting for NGI • We still use 1-100 Mbps • So, it is takes us days or months to move 1TB • Using Last Generation Internet (LGI) • UPDATE: We’re still waiting for NGI… • Worse: The Mobility Gap  We’ll always be waiting

10. Update to Gray’s QuestionAfter 10 years (1000x more disk capacity): 1TB  1PB • Gray’s Question: What is the best way to move a terabyte from place to place? • Updated Version: What is the best way to move a petabyte from place to place?

11. Petabytes on the Move • Standard clouds: • Datacenters tied to physical locations • Generalized clouds: • Datacenters + Data on the Move • Shuttling between Datacenters • Shipping containers • Shuttling between work and home (as we commute to work) • Laptops • Car trunks • Cell phones (flash in pockets)

12. The Cloud of the Future:4M PBs, Mostly on the Move

13. Challenges • Last Inch: • Easy enough to drive a shipping container from here to there, • but loading & unloading? • Can we design a shipping container so • it can be plugged into a cloud • as easily as plugging a USB disk into a laptop? • Ditto for car trunks and cell phones • Naming Conventions (and File Futures) • How do we refer to blobs that • used to be here • or will be there? • Anything you can do with a file, • I would like to do with a file future • Normally, can’t do much with a file until you have it on your machine • Exceptions: scheduled cron jobs • Suggestion: drag & drop file futures

14. The Last Inch: On-Ramps & Off-Ramps for the Information Super-Highway • 1 Packet per day scenarios: • e.g., daily commute  point-to-point data connection between work and home • Phones (GBs/day) • Small payloads (MBs)  Wireless (radio stack, blue tooth, WiFi) • Larger payloads  Wired USB (power: can’t afford to run WiFi all day) • Laptops (1 TB/day) • Disks in Car Trunks (10 TBs/day) • Power is less of a concern (than with phones): • Plenty of power to run WiFi all day • Small payloads (½ TB/day): • Wireless (8 hours of 144 Mbps WiFi in both garages) • Larger payloads: Wires (or carry disks)

15. Another 1 Packet per Day Scenario • Ship Shipping Containers (3 PBs/day) • Man with Shipping Container • (like a man with a van) • Scenario: Replace expensive batteries & generators with geo-redundancy • Shuttle containers back and forth between two datacenters that are a day’s drive apart • Driver connects a bunch of jumper cables at the end of his shift (and the data off-loads overnight) • 1 Tbps cables can transfer a PB in hours (3PBs per night) • Patch panel with 100 slots for 10 Gbps • Chilled water

16. Wires v. Sneakernet for 1 Packet per Day Scenarios Throughput, Cost, Latency, ConvenienceIn the limit, Mobility Gap will eventually favor Sneakernet for large payloads

17. Naming Conventions & File Futures • How do we refer to blobs on the move? • Separate signaling from payloads • Scenario: Email with large (GBs  PBs) attachments • Payload (Attachment): • Don’t fail for large attachments • Rather, fall back to sneakernet (if necessary) • Signaling URLs with serial numbers • Tracking service: where’s my package • Permalink: a link that I can give to friends and family • A query to a search service: • Are there any other copies of this blob that are easier to get to from where I am right now? • Other blobs like this one? • Who else is interested in this blob? • Updates? • Signaling URLs  File futures: • Anything you can do with a file, • I would like to do with a file future

18. From Point-to-Point Scenarios  Haggling • Buildoutsneakernet slowly • Start with Point-to-Point Scenarios • Transactions: • e.g., post office, man with a shipping container • Subscriptions: • e.g., commute  data connection between work & home • Killer Apps • Transactions: • Ad hoc copies, email with large attachments • Subscriptions: • Backup, mirrors, geo-redundancy, remote sync, hub & spoke network, (big) podcasts, CDN (Content Distribution Networks)

19. Haggling: http://ica1www.epfl.ch/haggle/ • Buildoutsneakernet slowly • Start with Point-to-Point Scenarios • When take-rates are sufficiently high  Haggling (and more) • Switching/Hitchhiking: • Cars exchange packets when parked near one another • Warehouses (moving companies): • Aggregate loads to fill packets • CDN (Content Distribution Networks) • Caching, Geo-distribution, Syncing, Compression • Peer-to-Peer • Hub & Spokes (Fed-Ex)

20. Hub and Spokes (Fed-Ex) • Most routes make connections in major hub cities • Hub & Spoke Network Buildout Plan: • Hubs: More network capacity at work • Spokes: As consumers commute to work, • They sync their homes to work • Where there is more network capacity • Airline Analogy • Work  Hub Airport • Daily Commute  Commuter Airline

21. Conclusions • Advantages & Disadvantages • Throughput, Cost, Latency & Convenience • Large payloads (PBs)  Sneakernet • Small payloads (MBs)  Wires • Mobility Gap (Disk/Flash are getting better faster than Wires) • In the limit, for large enough payloads, Mobility Gap  Sneakernet • Hybrids: Email with large attachments • Large attachment  Sneakernet • Email + Signaling URLs  Wires • 1 Packet per Day Scenarios • Daily commute  Data Connection between Work & Home • Hub & Spoke Network Buildout: Work  Hub Airport; Daily Commute  Hub Airline • Backup Power (Batteries & Generators)  Geo-Redundancy • Man with shipping container shuttles between datacenters • Research Challenges: Last Inch & Naming Conventions (File Futures)