1 / 19

Global-Scale Persistent Storage: An OceanStore Overview

Explore the architecture of OceanStore designed for global-scale persistent storage, featuring connectivity to various devices, data safety, and cooperative utility model. The design highlights, system architecture, access control, data routing strategies, and more are discussed in detail.

ddarling
Download Presentation

Global-Scale Persistent Storage: An OceanStore Overview

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Principal Resource OceanStore: An Architecture for Global-Scale Persistent Storage: John Kubiatowicz, DavicBindel, Yan Chen, Steven Czerwinski, Patrick Eaton, Dennis Geels, Ramakrishna Gummadi, Sean Rhea, Hakim Weatherspoon, Westley Weimer, Chris Wells, Ben Zhao. Proceedings of ACM ASPLOS 2000. Additional Materials http://oceanstore.cs.berkeley.edu/ www.cs.fsu.edu/~awang/courses/cop5611_s2006/oceanstore.ppt www.cs.fsu.edu/~awang/courses/cop5611_s2004/lecture_21_distributed_fs3.ppt Presenter Stanley Ziewacz April 10, 2008

  2. Global-Scale Persistent Storage • Global Scale • 1010 users • 10,000 files per user • Population Clocks • U.S. 303,807,847 • World 6,660,028,725 • 16:12 GMT (EST+5) Apr 08, 2008 • Mole • 6.023 X 1023 atoms in exactly 12 grams of carbon-12

  3. Persistent Data Storage • Connectivity to all types of computing devices • Desktop, laptop, palmtop, cellphone, etc. • Information Safety • Avoid prying eyes and survive malicious hands • Durable—1000 years? • Redundancy with continuous repair and redistribution for long-term storage • Uniform and highly-available access to data • Data divorced from location • Servers close to clients

  4. Cooperative Utility Model • OceanStore Service Providers • Client pays one monthly bill to one company • Clients can use resources on other OSPs • OSPs buy and sell capacity among themselves • Millions of servers

  5. Design Highlights • Infrastructure is only trusted in the aggregate • Servers may crash without warning • Servers may leak data • Data can be cached anywhere, anytime • Promiscuous caching • Nomadic Data

  6. Applications • Groupware • Personal information managers • Digital libraries • Scientific data repositories

  7. System Architecture • Naming • Data location and routing • Update model and conflict resolution • Deep archival storage • Introspection

  8. Naming Globally Unique Identifiers GUIDs • Object GUID • Secure hash of owner’s key and human-readable name • Server GUID • Secure hash of server’s public key • Archival fragment GUID • Secure hash over the data it stores

  9. Access Control • Restricting Readers • If data is not completely public • Encrypt • Distribute key to users with read permission • To revoke read permission • Request that replicas be deleted or re-encrypted • Restricted at clients • Restricting Writers • All writes are signed • Owner can provide access control lists for objects • Restricted at servers

  10. Data Location and Routing • Support location-independent routing • Message routes to discover a destination • Then message routes directly to destination • Two routing strategies • Bloom filters, fast probabilistic algorithm • Plaxton-style routing

  11. Bloom Filter • A Bloom filter • Represents a set S = {S1, … Sn} • Is depicted by a m bit array, filter[m] • Uses r independent hash functions • h1…hr • for i = 1…n • for j = 1…r • filter[hj[Si]] = 1 www.cs.fsu.edu/~awang/courses/cop5611_s2004/lecture_21_distributed_fs3.ppt

  12. Bloom Filter Example • filter[] = {1, 1, 0, 1, 0, 1} • Does x belong to the set? • filter[h1(x)] = filter[0] = 1 • filter[h2(x)] = filter[3] = 1 • filter[h3(x)] = filter[5] = 1 • Does z belong to the set? • filter[h1(z)] = filter[2] = 0  no • filter[h2(z)] = filter[3] = 1 • filter[h3(z)] = filter[5] = 1 www.cs.fsu.edu/~awang/courses/cop5611_s2004/lecture_21_distributed_fs3.ppt

  13. Attenuated Bloom Filters www.cs.fsu.edu/~awang/courses/cop5611_s2006/oceanstore.ppt

  14. Variation on Plaxton Routing • Each object GUID has root node • Root ID matches GUID’s hash in the most bits • But replicas can be placed anywhere • Publishing process for replicas • Do Plaxton hops from replica location to root • Place a pointer to replica locale at each hop

  15. Update Model • Client generates updates • Primary tier of replicas commit • Evaluate update’s predicates in time order • Perform action with earliest predicate

  16. Updating Ciphertext • Only 4 predicates available • Compare-version • Compare-size • Compare-block • Search • Actions available • Replace-block • Insert-block • Delete-block • Append

  17. Serializing Updates • Primary tier of replicas • Byzantine agreement protocol • Final commit order • Multicast committed updates • Tentative Updates • Sent to several random replicas • Tentative commits spread by epidemic

  18. Deep Archival Storage • Objects exist in both active and archival form • Archival form is permanent read-only • Form treated as series of fragments of data • Fragments spread over the network structure • Use any n fragments to reconstruct data • In principle every version is archived

  19. Introspection • Cluster recognition • Each client machine has an event handler triggered by each data access • Replica Management • Event handlers monitor client requests and system load • Detect periodic migration of clusters from site to site • OceanStore can monitor my work/travel routine

More Related