An Adaptive Multi-Objective Scheduling Selection Framework For Continuous Query Processing

1. An Adaptive Multi-Objective Scheduling Selection Framework For Continuous Query Processing A Presentation @ IDEAS, Montreal, Canada, July 27, 2005 Timothy M. Sutherland Bradford Pielech Yali Zhu Luping Ding and Elke Rundensteiner Worcester Polytechnic Institute Worcester, MA, USA

2. Continuous Query (CQ) Processing Register Continuous Queries May have different QoS Requirements. Streaming Data Streaming Result Stream Query Engine May have time-varying rates and data distribution. Available resources for executing each operator may vary over time. Run-time adaptations are required for stream query engine.

3. Runtime Adaptation Techniques • Operator Scheduling • Query Optimization • Distribution • Load Shedding • Others

4. Operator Scheduling for CQ • Operator Scheduling • Determines the order to execute operators • Allocates resources to query operators • Existing Scheduling Algorithms • Round Robin (RR) • First In First Out (FIFO) • Most Tuples In Queue (MTIQ) • Chain [BBM03] • Others 4 scheduler 3 1, 2, 3, 4 1 2 stream B stream A

5. Properties of Existing Scheduling Algorithms • Uni-Objective • Designed for a single performance objective • Increase throughput • Reduce memory • Reduce tuple delay • Fixed Objective • Cannot change objective during query run • May be insufficient for CQ processing

6. Performance Requirements in CQ • May be multi-objective • Example • Run time-critical queries on memory-limited machine • Two performance goals: less result delay and less memory • May vary over time • Example • System resource availability may change during query run • Under light workload: faster throughput • Under heavy workload: less memory • Existing scheduling algorithms • Not designed for multiple changing objectives • As a result, each has its strengths and weaknesses

7. 0 0.09 0.09 0 0 0 0.09 0.18 3 3 3 3 3 3 3 0 0.09 0 0 0 0 0.09 0 0 3 1.25 3.25 1 2 4 2 2 2 2 2 2 2 0 0 0 0 0 0.9 0.9 0 1 1 1 1 1 1 1 2 1 2 1 0 3 1 2 Scheduling Example: FIFO FIFO Start at leaf and process the newest tuple until completion. End User σ = 1 T = 0.75 3 Time: σ = .1 T = 0.25 2 • FIFO’s queue size grows quickly. • FIFO has fast first outputs σ = 0.9 T = 1 1 Stream

8. 0 0 0 0 0 0 0 0 0 0 0 3 3 3 3 3 3 3 3 3 3 0.2 0 0.1 0 0.1 0.2 0.3 0.4 0.3 0 0 4.5 4.75 2 1 5.75 3.25 0 6 2.25 3.5 2 2 2 2 2 2 2 2 2 2 1.5 0.6 0.5 0.7 1.6 1.7 0 0 0.8 1.8 0.9 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 2 1 1 1 Scheduling Example: MTIQ MTIQ Schedule the operator with the largest input queue. End User σ = 1 T = 0.75 3 Time: σ = .1 T = 0.25 2 • MTIQ’s queue size grows at a slower rate • Tuples remain queued for longer time σ = 0.9 T = 1 1 Stream

9. Performance Comparison

10. Summary of Problem • What does CQ need: • Runtime Adaptation • Multiple prioritized optimization goals • Dynamically changing optimization goals • Our solution -- AMoS • Adaptive Multi-Objective Scheduling Selection Framework

11. Outline • Introduction and Motivation • The AMoS Framework • Experimental Evaluation • Conclusion

12. General Idea of AMoS Framework • Meta-scheduler • User specify multiple performance objectives • Rank scheduling algorithms based on performances • Dynamically select the current best algorithm • Design logic • Simple, low overhead and effective Performance Requirements Algorithm Evaluator Statistics Scheduling Algorithm Library Selecting Strategy Algorithm Selector Decision Request Scheduler Decision

13. Specifying Performance Objectives • Metric: Any statistic calculated by the system. • Quantifier: Maximize or Minimize • Weight: The relative weight / importance of this metric • The sum of all weights is exactly 1.

14. Adapting Scheduler • Step 1: Scoring Statistics Periodically • Step 2: Scoring Scheduling Algorithms • Step 3: Selecting Scheduling Algorithm

15. Step 1: Scoring Statistics Performance Objectives Stats Score Matrix • Update stats scores of the current scheduling algorithm Acurrent • Zi_new-- score of stat i for Acurrent • μiH, maxiH, miniH -- mean, max and min history value of stat i. • μiC -- most recent collected stat i. • decay -- decay factor in (0, 0.5). Exponentially decay out-of-date data. • Zi_new ∈(-1, 1).

16. Step 2: Scoring Scheduling Algorithms Stats Score Matrix Performance Objectives • Update score of scheduling algorithm A. • ScoreA-- score for A. • zi – score of stat i for A. • qi – -1 for minimize, 1 for maximize • wi – Weight in table of objectives. • Add 1 to shift from [-1, 1] to [0, 2]. • ScoreA ∈(0, 2).

17. Issues In Scheduler Selection Process • The framework need to learn each algorithm • Solution: All algorithms are initially run for once • Algorithm did poorly earlier may be good now • Solution: Periodically explore other algorithms • Reason for adopting the Roulette Wheel Strategy

18. Step 3: Selecting Next Algorithm Roulette Wheel [MIT99] • Chooses next algorithm with a probability equivalent to its score • Favors the better scoring algorithms, but will still pick others. • Well performed ones have better chances • Others also have chances to be explored • Lightweight so overhead is very low • Proven to be effective in experimental study

19. Overall Flow of the Adapting Process Input: performance objectives & candidate scheduling algorithms Initially run all algorithms once Periodically Scoring statistics change requested? N Repeat until query is done Y Rank candidate algorithms Select next algorithm to run

20. Summary of the AMoS Framework • Light-weight • Use runtime statistics collected by the system • Ranking formula are simple yet effective • Self learning • No apriori information needed • Learn behaviors of scheduling algorithms on the fly • Easily extendable • Add more scheduling algorithms • Add more performance objectives • Add more selecting strategies

21. Outline • Introduction and Motivation • The AMoS Framework • Experimental Evaluation • Conclusion

22. Experimental Setup • Evaluated in CAPE system [cape04] • A prototype continuous query system • Query plans • Consists of join and select operators • Input streams • Simulated with Poisson arrival pattern • Performance objectives • Different number of objectives • Different weight of objectives

23. One Performance Objective

24. Two Performance Objectives 50% focus on output rate, 50% focus on tuple delay

25. Two Performance Objectives (cont.) 70% focus on tuple delay, 30% focus on output rate 30% focus on tuple delay, 70% focus on output rate

26. Three Performance Objectives Equal focus (33%) on output rate, memory and tuple delay

27. Conclusions • Identified the lack of support for multi-objective adaptation • Existing approaches only focus on single objective • Cannot change objective during query run • Proposed a novel scheduling framework: • Allows applications to control performance objectives • Alters scheduling algorithm based on run-time performances • Independent of scheduling algorithms or performance objectives • AMoS strategy shows very promising experimental results. • Developed and evaluated in the CAPE system • W/ single objective, performs as well as the best algorithm • W/ multiple objectives, overall better than any algorithm

28. Thank You! For more information, please visit: davis.wpi.edu/~dsrg