1 / 30

INS/Twine : A Scalable Peer-to-Peer Architecture for Intentional Resource Discovery

INS/Twine : A Scalable Peer-to-Peer Architecture for Intentional Resource Discovery. Magdalena Balazinska , Hari Balakrishnan, and David Karger MIT – Laboratory for Computer Science http://nms.lcs.mit.edu/. Problem Description. Abundant ubiquitous computation and communication

debbie
Download Presentation

INS/Twine : A Scalable Peer-to-Peer Architecture for Intentional Resource Discovery

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. INS/Twine : A ScalablePeer-to-Peer Architecture for Intentional Resource Discovery Magdalena Balazinska, Hari Balakrishnan, and David Karger MIT – Laboratory for Computer Science http://nms.lcs.mit.edu/

  2. Problem Description • Abundant ubiquitous computation and communication • Increasingly large computing environments • Dynamic environments • Many possible “cool” applications • Locate resources using intentional descriptions

  3. INR INR INR INR INR INR INS Overview Client describes attributes of required resources Self-configuring resolver network Resources advertise their capabilities INR: Intentional Name Resolver

  4. Resource Discovery Goals • Allow client applications to locate services and devices • Handle sophisticated resource descriptions • Handle dynamism in the operating environment • Scale to large numbers of resources

  5. Sensor Proxy Sensor Proxy Sensor Proxy Existing Solutions for Scalability Partitioning Cameras Bldg 3 Resolver Resolver ? Floors 1-3 Floors 4-6 Bldg 1 Bldg 2 Resolver Resolver Resolver Resolver

  6. Sensor Proxy Sensor Proxy Sensor Proxy Existing Solutions for Scalability Hierarchies Resolver Resolver Resolver Resolver Resolver Resolver Resolver

  7. INS/Twine Contributions • Collaborating peer resolvers: no content or location constraints on queries • Scalability and load distribution via hash-based partitioning of resource descriptions among resolvers • Semi-structured resource descriptions with arbitrary attribute-set • Network dynamism • Designed for an environment where all resources are equally important to users

  8. Sensor Proxy INS/Twine Approach Overview resource = camera resource = motion sensor Resolver Resolver subject = traffic Resolver Resolver Resolver Resolver Resolver subject = traffic resource = motion sensor subject = traffic resource = camera subject = traffic

  9. INS/Twine Approach Overview • A resource advertises its descriptions and network location to any resolver • The description is converted into a canonical form: attribute-value tree (AVTree) • Using the content of the advertised description, the resolver determines which resolvers should know about the resource • The resolver forwards the description to these resolvers for storage • Similarly for queries

  10. Resolver … Res adv. Strand Mapper a1 v1 a2 h : 0110 1001 0000 v2 Strand Key Router 0110 1001 0000 0110 1001 0000 Best(01101001000) K nodes are chosen Key Distributed Hash Table Architecture of One Resolver h = hash(a1v1-a2v2)

  11. resource subject root camera traffic subject resource resource resource camera traffic camera camera manufacturer model manufacturer model ACompany AModel resource resource camera camera model manufacturer AModel ACompany Splitting Descriptions into Strands Resource description: AVTrees Six strands • Each strand is then hashed into a 128 bit value which determines the nodes that will store the resource information. • All queries, even short stranded queries require asking only one resolver!

  12. Distributed Hash Table: Chord N5 • Nodes and keys have 160-bit ID’s • Chord maps ID’s to “successor” • Successor: Node with next highest ID N10 N110 N20 N99 Circular ID Space N32 Stores key-values for keys 21..32 N80 N60 Keys 33..60

  13. Basic Lookup N120 N10 “Where is key 80?” N105 Successor pointer N32 “N90 has K80” N90 K80 N60

  14. “Finger table” allows log(N)-time lookups ½ ¼ K = log(n) immediate Successors for robustness Stabilization methods for concurrency 1/8 1/16 1/32 1/64 1/128 finger[i] points to successor (n + 2i) log(n) fingers in all N80

  15. Sensor Proxy Back to Example resource = camera resource = motion sensor Resolver Resolver subject = traffic Resolver Resolver Resolver Resolver Resolver subject = traffic resource = motion sensor subject = traffic resource = camera subject = traffic

  16. Properties of INS/Twine • For a resource description with a attributes, t at the top-level : • Number of strands is : s = 2a – t • For R resources, S strands, K replication level, and N resolvers : • Storage requirement at each resolver : (RSK)/N • Advertisement: • SK resolvers contacted (+ O(log N) for key routing) • Query: • K resolvers contacted (+ O(log N) for key routing) • 100% success rate for less than K failures • Failure rate decreases exponentially with K

  17. State Management • Resources join, move, leave and fail • Resolvers join and fail • How to maintain consistency while achieving fault tolerance? • Hard state • Soft state • Hybrid solution implemented in INS/Twine

  18. INR INR INR INR INR INR INR INR Resource State Management D D D d INR: Intentional Name Resolver

  19. INR INR INR INR INR INR INR INR Resource State Management Remove Remove Remove INR: Intentional Name Resolver

  20. INR INR INR INR INR INR INR INR Resource State Management D D D d INR: Intentional Name Resolver

  21. INR INR INR INR INR INR INR INR Resource State Management Expire Expire Expire INR: Intentional Name Resolver

  22. Evaluation: Data Distribution Data distribution rather even. Each resolvers holds a small fraction of resource descriptions

  23. Evaluation: Query Resolution Even distribution of queries among resolvers

  24. Conclusion • Intentional resource discovery • Scalable peer-to-peer network of resolvers • Hash-based mapping of resource descriptions to resolvers • Dynamic and even distribution of resource information and queries • Handles dynamism of resources and resolvers http://nms.lcs.mit.edu/projects/twine/

  25. Appendix

  26. INS Overview INR: Intentional Name Resolver

  27. Describing Resources • INS name-specifier • XML • AVTrees

  28. Problems using concatenation • If numerous resources share the same prefix, some nodes may receive significantly more load than others • Fully solving short stranded queries requires the colaboration of a linearly growing number of resolvers (with respect to network size) • 1) and 2) are contradictory requirements!

  29. Distributed Hash Table: Chord A distributed hash-table is used to map keys onto resolvers efficiently: From: Chord: A Peer-to-Peer Lookup Service for Internet Applications Ion Stoica, Robert Morris, David Karger, Frans Kaashoek, Hari Balakrishnan Proc. ACM SIGCOMM Conf., San Diego, CA, September 2001.

  30. Problems using prefixes • More insertions for each resource. Small factor since we expect descriptions to be rather short • Very popular prefixes may overload certain nodes : many advertisements and queries => the prefix should then become unusable • Nodes stop storing resources for that prefix • Nodes answer queries for the prefix specifying that they provide a partial answer due to the vague nature of the query

More Related