1 / 23

Pond – The Ocean Store Prototype

Pond – The Ocean Store Prototype. Pond – The Ocean Store Prototype. Presented By Jon Hess cs294-4 Fall 2003. Pond – The Ocean Store Prototype. Overview Goals Features Design Implementation Experimental Results. Pond – The Ocean Store Prototype.

davidashley
Download Presentation

Pond – The Ocean Store Prototype

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Pond – The Ocean Store Prototype Pond – The Ocean Store Prototype Presented By Jon Hess cs294-4 Fall 2003

  2. Pond – The Ocean Store Prototype Overview • Goals • Features • Design • Implementation • Experimental Results

  3. Pond – The Ocean Store Prototype Goals – A Distributed File System Offering • Incremental Scalability • More servers translates to more available data • Secure Sharing • Access Control • Long term durability • With high probability data should not be able to leave the system

  4. Pond – The Ocean Store Prototype Key Features • Location Independent Routing • Tapestry • Byzantine Update Agreement • For management of the inner ring • Push based cache correction • Overlay locality aware multi-cast network • Continuous archiving • Erasure codes

  5. Pond – The Ocean Store Prototype Design • Two tier network • Upper tier composed of well connected powerful servers • Serialize changes to data • Lower tier composed of user workstations • Cache data • Archive data • Read / Write data

  6. Pond – The Ocean Store Prototype The Data Object • Can be thought of as corresponding to a File • Is composed of immutable versions • Each version Is broken Into B-tree of blocks • Is referenced by an AGUID • Versions by VGUID • Blocks by BGUID • Can be conditionally operated on

  7. Version – VGUID MD BGUID IB IB Pond – The Ocean Store Prototype Data Object - AGUID

  8. Version – VGUID MD BGUID IB IB Pond – The Ocean Store Prototype Data Object - AGUID Previous Version Newest Version Version - VGUID MD BGUID IB

  9. Pond – The Ocean Store Prototype • Retrieving Data • AGUID: secure hash of name and public key • Contact primary replica to find VGUID • From the VGUID retrieve BGUID’s • Copy the block data to the local system • Join the dissemination tree • Act as a cached copy

  10. Pond – The Ocean Store Prototype • Controlling Data • Primary Replica • Publishes AGUID to VGUID mappings • Digitally signs • Enforces access control • Serializes writes • Pushes cache updates • Archives data

  11. Pond – The Ocean Store Prototype • Writing data • Send a request to the primary replica • Replica verifies credentials • Checks predicates • Creates new VGUID and then associates data • Pushes update down disseminationtree

  12. Pond – The Ocean Store Prototype Archive Servers Erasure Primary Replica Writer Caching Readers

  13. Pond – The Ocean Store Prototype • Archiving Data With Erasure Codes • Divides data into N chunks • Encodes chunks to M erasure blocks • M > N • Any N of the M blocks is sufficient for reconstruction • Located by erasure block number and BGUID. • How does one know the BGUID? • The AGUID is unavailable?

  14. Pond – The Ocean Store Prototype • Primary Replica – The Inner Ring • Byzantine internal decisions • Decisions published with by public key • Each node has a fraction of the private key • Enough fractions to prove a Byzantine agreement was reached are required to sign a decision

  15. Pond – The Ocean Store Prototype • Inner Ring – Changing Nodes • Byzantine decision • Decides to elect • Decides Who to elect • Chooses the key set • Old keys are deleted • By Byzantine assumption, conspiring nodes do not have enough keys to publish

  16. Pond – The Ocean Store Prototype • The Responsible Party • Publishes node statistics • Used to nominate nodes to inner ring • Has no say over the actions of the inner rings • There could be many of them • Being compromised would not destroy the network

  17. Pond – The Ocean Store Prototype • Implementation of the Pond Prototype • Pros • 50,000 lines of Java • Event based between modules • Some modules are pluggable • Highly portable • Cons • Garbage collector ‘Stops The World’

  18. Pond – The Ocean Store Prototype Storage Overhead • B-Tree dominates cost of small files • Convergence at 32KB • Erasure Codes add 4.8x storage penalty

  19. Pond – The Ocean Store Prototype Write Latency Components • For small updates • Computing the signature dominates • For large updates • Computing the erasure fragments dominate Tests are local to minimize network’s effect

  20. Pond – The Ocean Store Prototype Write Throughput • Increasing data size amortizes signature time • Approaches 8MB/s as block size grows • With archiving enabled • Performance peaks at 2.6MB/s

  21. Pond – The Ocean Store Prototype Propagation Efficiency • As Replicas Increase • Network economy becomes more efficient • Less high RTT links are used • Tests are with 10, 20, and 50 replicas • This is 2%, 4% and 10%! of the network • Are these number likely to occur in practice?

  22. Pond – The Ocean Store Prototype Andrew Benchmark • WAN • Read Performance • Up to 4.6x better • Write Performance • Up to 7.3x worse • LAN • Read Performance • From 2x to 3x worse • Write Performance • From 8x to 80x worse Are these tradeoffs acceptable?

  23. Pond – The Ocean Store Prototype Questions?

More Related