1 / 21

Scalable and Dynamic Quorum Systems

Scalable and Dynamic Quorum Systems. Moni Naor & Udi Wieder The Weizmann Institute of Science. Quorum Systems. A quorum system is an intersecting family of sets over some universe. Formal Definition: U – Universe Hello World Hello World

ada
Download Presentation

Scalable and Dynamic Quorum Systems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Scalable and Dynamic Quorum Systems Moni Naor & Udi Wieder The Weizmann Institute of Science

  2. Quorum Systems • A quorum system is an intersecting family of sets over some universe. • Formal Definition: • U – Universe • Hello World • Hello World • is called a quorum system. A,B are called quorum sets. • Examples • Majority – all sets that contain more than half of the elements. • Dictatorship – all sets that contain a specific element. • Many more…

  3. Quorum Systems - Applications • The universe is associated with a set of processors. • Mutual exclusion: In order to enter a critical section, a user must get permission from a quorum set. • Intersection property guaranties mutual exclusion. [GB85]. • Data replication: Divide the quorum sets into reading sets and writing sets. • Intersection property guaranties effective search. • More….

  4. Example – The PATHS Quorum System • Suggested by [NW94]. • Element  Pair of dual (crossing) edges. • Quorum Set Left – Right path + Top – Bottom path. • Every top-bottom path in the dual grid intersects every left-right path in the grid. • Data replication: • Top Bottom – Writing. • Right Left – Reading.

  5. Measures of Quality - Load • Load • Each quorum set is chosen by the user with some probability. • Imposes a probability of accessing a processor. • The load measures the access probability of the busiest processor. • Example: In the Majority system, if a random set of size n/2 + 1 is chosen, the load is asymptotically half (and this is best possible). • Theorem [NW94]: The load of a quorum system is at least  The load is at least .(For every system and every access strategy). • The PATHS system has load of . Let c be the size of the smallest quorum set.

  6. Measures of Quality - Availability • Availability. • Assume each processor fails with some fixed probability p. What is the probability that there still exists a quorum set? • Theorem[PW95] : If then the singleton has the ‘best’ availability. • The PATHS system has a live quorum set with probability , ( for ).

  7. Measures of Quality – Probe Complexity • A random portion of the processors are down. • How many processors should be queried before a live quorum is found? • Adaptive algorithms – more efficient [HP01 , KXG00, PW96]. • Non adaptive algorithms (predefined set of probed elements) – Easy to implement in parallel. • Given a network topology, what is the communication complexity of accessing these processors [Bazzi96]. • Processors from the same quorum should be ‘close’ to one another.

  8. Our Contributions • Probe Complexity: • Lower bound for non-adaptive algorithms. • Tight upper bound for PATHS. • Tight adaptive upper bound for PATHS. • Introduce Dynamic PATHS (DPATHS). • Dynamic P2P style version in which processors may join and leave. • Scales well. • Maintains the good properties of PATHS.

  9. Theorem: If then If load is then The PATHS system has a non-adaptive probe complexity of Probe Complexity vs Load • Probe complexity depends on the load. • Example: Only a small number of processors participate in quorum sets Small probe complexity and high load. • Notation: • 1/c – the load of the quorum system. • X – set of probed elements. Non-adaptive algorithm must choose X in advance. • p – failure probability.

  10. Survived dual edge. Probe Complexity – Upper Bound • A failed edge corresponds to a surviving dual edge that crosses it. • An edge fails with probability • Dual edge fails with probability • A top – bottom path must circumvent red components of dual graph. • By Menshikov’s Theorem - All red components are of diameter O(log n), therefore a strip of width O(log n) would contain a path w.h.p. Conclusion: The non-adaptive probe complexity of PATHS is

  11. Non-adaptive probe complexity is strictly higher than adaptive complexity. Survived dual edge. Adaptive Probe Complexity • All red components have small diameter. • All red component have small volume. • Smart DFS finds a path of length . • Optimal load implies quorum sets are of size . • Conclusion: PATHS has optimal adaptive probe comp.

  12. Paths Quorum System - Summary • Load - . Optimal • Availability – Very high when . Optimal • Probe Complexity – • Non-adaptive - . Optimal (for load). • Adaptive - . Optimal (for load). • Load after failures - . Optimal. • Network implementation - Supports probing algorithms (think of edges as links).

  13. Dynamic Quorum Systems. • Motivation: Peer-to-Peer applications. • Related Work – dynamic probabilistic quorums[AM03] • Processors may join and leave the quorum system. • Objectives: • Low cost of Join/Leave • Maintain the intersection property (integrity). • New quorum sets • Adjust old quorum sets that are no longer valid. • Scalability • Load should reduce when processors join. • Availability should increase when processors join. • Probe complexity should not grow by much.

  14. Dynamic Quorums Main Idea: Assign each processor to a point in a continuous space. • The underlying space is the unit square. • Divide the space into cells using a Voronoi diagram. • CAN[01] suggests dividing the space into rectangles. • A cell consists of all the points that are closest to the processor. • Average degree of a cell is always 6. • Adding and removing a point is a local computation.

  15. Voronoi Diagram – The Delaunay Graph

  16. Voronoi Diagram – Join\Leave

  17. Random Voronoi Diagram • If each processor chooses its location uniformly and independently then: • The average degree is 6 (always), the maximum degree is with high probability. • The average area of a cell is . The maximum area of a cell is with high probability. • The average projection on the axis lines is of length .The maximum projection is with high probability.

  18. The Dynamic Paths Quorum System

  19. Integrity • Old Quorum sets may not be valid anymore. • Solution – application dependent. New processor must join the quorum set.

  20. Dynamic Paths – Measurs of Quality • Load – Depends on the size of the cell’s projection on the axis lines. Might reach . • Availability – Probability a live quorum set exists when converges to 1 (when n ). The exact rate of convergence is unknown. • Probe Complexity – Probably good. A Menshikov style theorem for Voronoi Diagrams is conjectured but not proven. If all cells are of size then there exists such that for failure probability , the qualities of PATHS follow to DPATHS.

  21. Open Questions • Load balancing schemes. Devise simple and reliable protocols for maintaining all cells in equal size. Good for many applications (such as building expanders [NW03]). • Sample log n locations, and enter the largest cell encountered. • Deterministic solutions? Simpler solutions? • Menshikov style theorems for random Voronoi Diagrams - pushing the failure probability all the way to half. (Hard). • Can resilience for worst case faults be achieved without damaging other parameters?

More Related