1 / 10

Scalable Adaptive Data Dissemination Under Heterogeneous Environment

Scalable Adaptive Data Dissemination Under Heterogeneous Environment. Yan Chen, John Kubiatowicz and Ben Zhao UC Berkeley. Dissemination Tree of OceanStore. Using Application-Level Multicast for Data Dissemination. Scalability Fault-tolerance Efficiency Adaptability. Our Solutions.

kynton
Download Presentation

Scalable Adaptive Data Dissemination Under Heterogeneous Environment

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Scalable Adaptive Data Dissemination Under Heterogeneous Environment Yan Chen, John Kubiatowicz and Ben Zhao UC Berkeley

  2. Dissemination Tree of OceanStore

  3. Using Application-Level Multicast for Data Dissemination • Scalability • Fault-tolerance • Efficiency • Adaptability

  4. Our Solutions • Use Tapestry, Distributed Overlay Routing & Location Infrastructure • Randomized data structure with search locality • Insensitive to faults, and self-repairable • Ease-of-maintenance • http://www.cs.berkeley.edu/~ravenben/tapestry.pdf

  5. Our Solutions (Cont’d) • Dedicated Infrastructure • Complemented with intelligent replica placement • Application-Level Semantics & Optimization • Dynamic transmission (selective dissemination) • Dynamic notification of updates

  6. Dissemination Tree Construction • Client Contact Statistically Closest Server Which Has the Data Through Tapestry • Autonomous decision - Path & load piggybacked • If client unsatisfied with QoS, server dynamically replicate data close to client • Model the Replica Placement as “Minimal Set Covering” Problem • Each server covers certain subset of clients (w.r.t. certain QoS, latency, bandwidth, etc.) • Approximate the solution with greedy algorithm • Distributed load balancing

  7. RealCast Tree Management Protocols • Bi-directional Messaging • Heartbeatmessage stream from root to clients • Refresh message from children to parent • Scalability • Each member only maintains states for direct children and parent • “Join” request can be handled by any member • Continuous Self-tuning and Auto-repair • Periodically check for better parent • Topology-aware through Wide-area Network Measurement and Monitoring Services (WNMMS)

  8. push invalidate push update App-Level Semantics and Optimization • Selective Dissemination • Dynamic update notification • Quantitative analytic model for dynamically choosing between poll (pull everytime), push invalidate and push update based on access/update pattern and clients’ preferences to certain metrics (e.g. average latency)

  9. Preliminary Simulation • Topology generated with GT-ITM (120 nodes) • Synthetic hot-cold pattern workload (100 objects) • Base-line is the push invalidate from Cao/Liu paper • Number of messages reduced by 10 - 20% • Average response latency reduced by 30 - 40%

  10. Conclusions • Dissemination Tree: Large-scale Data Dissemination with App-level Multicast • Key Techniques • Use distributed location services, Tapestry • “Minimal Set Covering with Load Balancing” for replica/service placement • App-level semantics and optimization • Preliminary Results • Feasibility of the infrastructure • Flexibility and effectiveness of app-level optimization

More Related