1 / 16

RAMCloud Scalable High-Performance Storage Entirely in DRAM

RAMCloud Scalable High-Performance Storage Entirely in DRAM.

cormac
Download Presentation

RAMCloud Scalable High-Performance Storage Entirely in DRAM

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. RAMCloudScalable High-Performance Storage Entirely in DRAM John Ousterhout, ParagAgrawal, David Erickson, Christos Kozyrakis, Jacob Leverich, David Mazières, SubhasishMitra, Aravind Narayanan, Guru Parulkar, Mendel Rosenblum, Stephen M. Rumble, Eric Stratmann, and Ryan Stutsman Stanford University Presented by Sangjin Han (many slides are stolen from https://ramcloud.stanford.edu)

  2. Features (or Goals) • Low latency: 5-10 µs (not milliseconds) • High throughput: 1M operations/s • Key-value storage with 1000s of servers • No replicas • Entirely in DRAM • Disks only for backup • Fast recovery (1-2 secs)

  3. Why DRAM?

  4. Data Model create(tableId, blob)=>objectId, version read(tableId, objectId)=> blob, version write(tableId, objectId, blob)=> version cwrite(tableId, objectId, blob, version)=> version delete(tableId, objectId) Tables Object Identifier (64b) (Only overwrite ifversion matches) Version (64b) Blob (≤1MB) RAMCloud Overview & Status

  5. Overall Architecture 1000 – 100,000 Application Servers … Appl. Appl. Appl. Appl. Library Library Library Library DatacenterNetwork Coordinator Master Master Master Master Backup Backup Backup Backup 32-64GB/server … 1000 – 10,000 Storage Servers

  6. Per-Node Architecture • One server per data • 2-3 backups for buffered logging

  7. Coordinator • Centralized server for data placement • E.g, • Clients obtain a map from the coordinator • And cache

  8. Fast Recovery (First Try) RecoveryMaster • Reconstruct data from backup logs • Bottleneck: disk B/W • Solution: more disks Backups Crashed Master

  9. Fast Recovery (Second Try) • Randomly distribute log shards • Bottleneck: recovery master • Solution: no single recovery master CrashedMaster RecoveryMaster ~1000Backups

  10. Fast Recovery (Third Try) • Temporarily distributed recovery masters • Happy? • 35 GB recovery in 1.6s using 60 nodes (SOSP 2011) CrashedMaster RecoveryMasters Backups

  11. Typical Datacenter Latency

  12. RAMCloud Latency

  13. Publication History • Unveiling: SOSP 2009 WIP • White paper: Operating Systems Review 2009 • What you read • (Call for) low latency: HotOS 2011 • Fast recovery: SOSP 2011 • Ongoing work: index, transaction, transport, … • Visit http://ramcloud.stanford.edu

  14. Thoughts & Discussion • Garbage collection? • Infiniband vs. Ethernet • HW support for transport layer? • Killer app? • Non volatile memory? • Synchronous vs. asynchronous query • Moving data vs. Moving code • How many papers out of this project? :P

  15. Thanks

  16. SORRY • My English still s^^ks • Would you repeat the question • SLOWLY and • CLEARLY ???

More Related