Network-Aware Operator Placement for Stream-Processing Systems CS253 project presentation

1. Network-Aware Operator Placement for Stream-Processing SystemsCS253 project presentation Min Chen, Danhua Guo {michen, dguo}@cs.ucr.edu 12/4/2006

2. Roadmap • Motivation • Background Knowledge • Project Architecture • Evaluation • Conclusion {michen,dguo}@cs.ucr.edu

3. Motivation • Stream-based overlay network (SBON) • Operator placement (RELAXATION) • Designed for large-scale DSPS • Network efficiency • Low delay • Cost space {michen,dguo}@cs.ucr.edu

4. Motivation • Cost Space • Idea: Problem -> Virtual space -> physical node • Latency (& Load) dimensions {michen,dguo}@cs.ucr.edu

5. Background Knowledge • Network Usage: A Blended Metric • Bandwidth-delay product • Individual applications perspective • Small delay • Network perspective • Scaling bandwidth by latency {michen,dguo}@cs.ucr.edu

6. Architecture • Overview {michen,dguo}@cs.ucr.edu

7. Operator Placement Algorithm • RELAXATION • Adapt to changing conditions • Decentralized • Considers impacts of shared placement decisions {michen,dguo}@cs.ucr.edu

8. Operator Placement Algorithm • Algorithm • Virtual Operator Placement {michen,dguo}@cs.ucr.edu

9. Operator Placement Algorithm • Algorithm • Physical Operator Mapping • Find k nodes whose coordinates are near the target cost space coordinate. • Sort the list by distance to the target coordinate. • Walk the sorted list, returning the first node already running the operator. • Failing that, return the nearest node that meets the application’s resource criteria {michen,dguo}@cs.ucr.edu

10. Operator Placement Result • Δ=0.1 and Ft = 1 {michen,dguo}@cs.ucr.edu

11. Evaluation • Based on PlanetLab trace files • Efficiency compared with other algorithm • Optimal: exhaustive search • Producer: place the operator at producer • Consumer: place the operator at consumer • Random: place the operator at a random node {michen,dguo}@cs.ucr.edu

12. Evaluation • Set up • Bandwidth: 2KB/s • # of Nodes: 226 nodes from PlanetLab • # of Queries: 1000 • Query set up: 4 producers, 1 comsumer, 1 unpinned operator per query {michen,dguo}@cs.ucr.edu

13. Evaluation • Result {michen,dguo}@cs.ucr.edu

14. Evaluation • Delay penalty after placing 1000 queries {michen,dguo}@cs.ucr.edu

15. Conclusion • Our experiment shows that SBON is efficient in terms of Network Usage (Bandwidth, Latency) {michen,dguo}@cs.ucr.edu

16. Reference • Peter Pietzuch et. al. “Network-Aware Operator Placement for Stream-Processing Systems” • Peter Pietzuch, Jonathan Ledlie, and Margo Seltzer, “Supporting Network Coordinates on PlanetLab”, In Proceedings of WORLDS 2005, San Francisco, CA, December 2005 {michen,dguo}@cs.ucr.edu