1 / 32

Anwitaman Datta School of Computer and Communication Systems

Dealing with dynamics in Very Large Decentralized Systems. Anwitaman Datta School of Computer and Communication Systems Ecole Polytechnique Fédérale de Lausanne (EPFL) Switzerland. Funded by:. NCCR-MICS: www.mics.ch/.

lilika
Download Presentation

Anwitaman Datta School of Computer and Communication Systems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Dealing with dynamics inVery Large Decentralized Systems Anwitaman Datta School of Computer and Communication Systems Ecole Polytechnique Fédérale de Lausanne (EPFL) Switzerland Funded by: NCCR-MICS: www.mics.ch/ Swiss National Centres of Competence in Research Mobile Information & Communication Systems Evergrow: www.evergrow.org/ 5thMay2006 Munich EC FP6, IST priority “Complex System Research” Contract no. 001935 (FET-IP) Ever-growing global scale-free networks, their provisioning, repair and unique functions.

  2. The peer-to-peer (P2P) paradigm P2P is more than justPirate-to-Piratefile-sharing applications!

  3. The P2P paradigm: Application perspective Sharing resources in large-scale networks knowledge content <rdf:Description about='' xmlns:xap='http://ns.abode.com/xap/1.0/'> <xap:CreateDate>2001-12-19T18:49:03Z</xap:CreateDate> <xap:ModifyDate>2001-12-19T20:09:28Z</xap:ModifyDate> <xap:Creator> Brahma </xap:Creator> </rdf:Description> … processing bandwidth storage

  4. The P2P paradigm: Systems perspective • Decentralized, Self-organizing • Lack of global control or knowledge • Autonomous participants – dynamic system and workload • External sources of unpredictability (many failures) • Centralization may lead to bottle-necks, become point of failure, administrative burden or censorship, or may simply not scale! … and still we shall like to build systems where we canprovide(albeit probabilistic)guarantees…to achieve certain desired properties

  5. Basic P2P applications Efficient and reliable resource discovery • internet-scale index structures (structured overlays) Persistent and available storage These functionalities can be used as building blocks substrates for other P2P applications and systems

  6. Critical challenges for P2P systems • Membership dynamics (Churn) • Members join, leave, (may) re-join • Physical address changes over sessions • Load-balancing • Storage load at peers • Access load at peers • Traffic at peers • etc. Today’s talk Another major part of my thesis (not presented today)

  7. Outline: Today’s talk • I. Structured overlays under churn • Structured overlays • A family of adaptive overlay repair strategy • Results II. Storage systems under churn • P2P storage systems • Exploration of maintenance-cost/performance trade-offs • Results III. Conclusions (may be)

  8. Structured overlays How to locate resources in a large-scale dynamic distributed system reliably?

  9. Structured Overlay Networks A generic substrate for many networked systems P2P Applications using structured overlays • Resource discovery • Communication • Storage • Content distribution networks • Information retrieval and semantic overlays • Peer meta-information management • …

  10. Structured Overlay Networks A generic substrate for many networked systems Efficient and reliable resource discovery • Internet-scale index structures (structured overlays) End user has/wants some specific Resource (name) P2P Applications:Resource  Unique Key(using a suitable function) 1 0 Key space

  11. Structured Overlay Networks A generic substrate for many networked systems • Partition a (key-)space among peers • So that peers are responsible for specific partitions (and keys) 1 0 Key space

  12. Structured Overlay Networks A generic substrate for many networked systems • Partition a (key-)space among peers • So that peers are responsible for specific partitions (and keys) • Embed a graph (using routing table) among these peers (respective partitions) which defines the topology So that there is connectivity of these partitions 1 0 Key space

  13. Structured Overlay Networks 1 0 Key space A generic substrate for many networked systems • Partition a (key-)space among peers So that peers are responsible for specific partitions (and keys) • Embed a graph (using routing table) among these peers (respective partitions) which defines the topology So that there is connectivity of these partitions • A routing strategy (often greedy) • So that any partition can be reached efficiently and reliably

  14. Structured Overlay Networks queryfor ‘100’ 0* 1* 00* 01* 10* 11* C D A E B F 1* : C, D 01* : B 1* : E 01* : B 1* : C, D 00* : F 0* : A, B 11* : E 0* : A, F 11* : E 0* : B, F 10* : D Stores data with key prefix 00 Stores data with key prefix 00 Stores data with key prefix 01 Stores data with key prefix 10 Stores data with key prefix 10 Stores data with key prefix 11 Examples of Overlay Networks • Various topologies: Tree, Ring, Hybrid, Hypercube, XOR, … • Numerous systems: P-Grid, Chord, Pastry, CAN, Kademlia, … P-Grid data structures

  15. Problem statement Churn and the overlay • Identity issues of peer (physical address changes over sessions) • How do I find the peer I used to know previously? • For routing in structured overlay • For using the content stored at returning peers • Exploit trust relationships among peers for other applications • Need a directory service for managing ID-to-IP mapping

  16. Solution approach TKDE 2004 Churn and the overlay P-Grid Self-referential directory routing based on logical address (and cached IP) routing based on logical address lookup IP address in case of failure (if local cache does not work) lookup IP address directory (logical ID <-> IP address) • Our idea: A self-referential directory to maintain peers’ ID-to-IP mappings to achieve logical independenceof the overlay network from the underlying physical network.

  17. Solution approach 0 : 1,1410 : 11,13 2 12,13,14 12 12,13,14 0 : 5,710 :6,13 6 1 : 1,311 : 2,12101: 8,13 8,9 Peering into the self-referential directory This toy example uses 4-bit representation of ID as the corresponding key Information about peer4is stored corresponding to key 0100 at peers 5,14 LEGEND ID Presently online 0 1 ID Presently offnline Up-to-date cache 1 :2 ,12 00 01 10 11 Stale cache at 5,14 4, 5 000 001 010 011 100 101 9 1 : 8,201 : 3, 10000: 1,7 2,3 14 1 : 2,1200 : 9,4011: 3,10 4,5 1 : 11,1200 : 1,9010: 5,14 3 6,7 0 : 4,711 : 2,12101: 8,13 11 8,9 0 : 5,911 :2,12100: 6,11 13 10,11 1 1 : 12, 1301 : 5, 10001: 9,4 1 1 : 12, 1301 : 5,14001: 9,4 1 : 6,1301 :10,14000: 1,7 1 : 8, 1300 : 7,9011: 3,10 1 : 6,800 : 1,7010: 5,14 0 : 4,911 : 2,12100: 6,11 7 1 4 2,3 5 4,5 10 6,7 8 10,11

  18. Solution approach Overlay maintenance using self-referential directory • Encountering unusable routes trigger queries • Recursive queries heal the network • Self-healing routing • A family of more efficient and adaptive overlay maintenance schemes (than proactive approaches) • e.g., Extreme cases: Correction on Use, Correction on Failure

  19. Some key results Self-healing routing • Efficient (lazy) overlay maintenance mechanisms • Introduced the dynamic equilibrium study of overlays under churn • an usual tool from Cybernetics • [TKDE 2004, WDAS 2004] A collateral (using the self-referential directory): • A decentralized PKI based on probabilistic quorum among overlay replicas • As an alternative to PGP like web-of-trust based models • [CEC 2003]

  20. Some key results CoU (eager) vs. CoF (lazy) Probability of a peer being online

  21. Some key results Contour map for cost vs. churn tolerance CoU Relative freq. of a peer changing address w.r.to queries in the overlay Probability of a peer being online

  22. Structured overlays under churn Recapitulation • Efficient (lazy) overlay maintenance mechanisms • Introduced a general purpose self-referential directory • Used self-healing routing using the directory as an overlay maintenance mechanism • Introduced and performed a dynamic equilibrium study of overlays under churn

  23. Collaborative storage So far we looked intohow to locateresources in a large-scale dynamic distributed system reliably. How to achieve available and persistent collaborative storage in such a dynamic environment efficiently?

  24. Collaborative storage Availability and persistence in a volatile environment • Once someone stores something, it should be in there! • Needs redundancy; i.e., content replication or erasure codes • Redundancy needs to be maintained, but efficiently • e.g., use ability to locate returning peers so that maintenance can be lazy

  25. Problems with existing approaches Availability and persistence in a volatile environment Redundancy maintenance strategy landscape • Proactive maintenance - Has prohibitive cost

  26. Problems with existing approaches Availability and persistence in a volatile environment Redundancy maintenance strategy landscape • Lazy mechanism used: Deterministic procrastination - In a system called TotalRecall - Threshold based repairs (to exploit returning peers) - Repair only when unavailable blocks exceed a threshold - Lower maintenance cost (over time) - Knee jerk - spiky bandwidth usage - Very vulnerable to correlated failures

  27. Solution approach Availability and persistence in a volatile environment Randomized subset sampling • Probe redundant blocks randomly - Until a threshold of available blocks are detected - Compensate redundancy for detected unavailable blocks - The repair process is continuous - The size of the subset probed depends on dynamics - For same maintenance cost: - Tolerates larger amount of dynamicity - Much better resilience against correlated failures

  28. Solution evaluation Availability and persistence in a volatile environment Evaluation methodology • Dynamic equilibrium analysis - validated with simulations Inherent to the environment (churn) i of N redundant blocks Outflow because of churn Inflow because of churn Outflow because of repairs Inflow because of repairs Depends on maintenance scheme

  29. Some key results Health of the storage system Results for: 8 out of 32 erasure code Maintenance scheme parameters chosen such that cost of repairs is same. Strategy-A: Deterministic procrastinationStrategy-B: Randomized subset sampling (our idea)Si = prob. that i (out of N) fragments are available

  30. Some key results Resilience of the storage system Results for: 8 out of 32 erasure code Maintenance scheme parameters chosen such that cost of repairs is same. Strategy-A: Deterministic procrastinationStrategy-B: Randomized subset sampling (our idea) fcorr: fraction of network affected in a correlated failure (apart normal churn)

  31. Storage systems under churn Recapitulation • Lazy randomized maintenance mechanism • Adaptive to the rate of dynamics • Explores the cost-resilience trade-offs • Re-used the dynamic equilibrium analysis methodology

  32. Dynamics in P2P systems Conclusions • Structured overlays under churn • Self-referential directory • Dynamic equilibrium analysis (from Cybernetics) • Storage systems under churn • Adaptive exploration of cost-performance tradeoffs • Using rateless Digital fountain codes (ongoing) • Other topics • Multifaceted load-balancing: [VLDB 2005], range queries [P2P 2005] • Topological aspects: de Bruijn [P2P 2004], small-world [NetDB 2005] • Updates [ICDCS 2003]

More Related