Scheduling and Workload Balancing for Clouds Czesław Smutnicki

1. Scheduling and Workload Balancing for Clouds Czesław Smutnicki Wrocław University of Science and Technology, Wrocław, Poland czeslaw.smutnicki@pwr.edu.pl www.pwr.edu.pl ICA CON, 2-3 June 2016

2. Wrocław University of Science and Technology, 100+ years of tradition

3. Wrocław, Poland

4. Wrocław University of Science and Technology. Technopolis (ICT), 2014. Cloud for education

5. Wroclaw University of Science and Technology. Virtual digital library, 2014. Cloud for education and repository

6. Presentation Schedule • Clouds and CIS • Resources • Optimization • Off-line models • On-line models • Criteria evaluation • Calculations • Proposals • Conclusion

7. Computational Cloud (CC) • infrastructure (hardware, operational platform, software, services, broadband, communication links, etc.), • accessible transparently, remotely and virtually through the network, • eliminates the need for maintaining single-handedly expensive computing resources, • sharing the strong power physical resources ensured by a CC provider among large number of users, • offers reduced cost of service, • ensures rational management of computing goods in case of solving very hard problems derived from science and practice CC offers: for scientists - a cheaper alternative to clusters, grids, and supercomputers to solve hard problems computationally expensive; for education – convenient tool providing e-platform; for mass users - the powerful tool to solve quickly applicative problems for the commercial and business usage depending on the client profile.

8. Complex/Distributed Information Systems (CIS) Efficient distribution of various types of ICT resources is crucial for the efficiency of management of CC systems, as well as for their business justification. CC allows on direct application of Complex Information System (CIS) in a broad spectrum of this meaning. We understand here CIS as special class of computer system composed of huge number of workstations (clients), distributed servers of contents or services and the net linking all these players. Generally, it works in the mode question-and-answer, although some deputed task directed to a server can be split and sub-ordered to next servers.

9. CC and CIS resources and their distribution One can consider CIS as CC resources dispersed among nodes in the net, called sometimes SaaS cloud computing model with centralized (balanced) management or web-based service. It is clear that events in this system have discrete character, with high dynamics of changes, no-predictive (random) set of coming events and huge size of dimension. From modeling point of view CIS can be perceived as the non-stationary mixed open/closed queening system with queues of limited length, each of which has set its own service policy. Such complex system cannot be analyzed analytically, because of insufficient power of theoretical methods. Then, many researchers consider simulation as thesuitable tool for describing and analyzing behavior of CIS.

10. Resource allocation and management Managing the system resources lead us to several problems considered commonly in the literature as discrete resource project constrained scheduling and workload balancing optimization. Depending on the character of introduced model, destination of the cloud (public, private, educational, etc.), a lot of formal models and solution algorithms were proposed. It is based on, among others, discrete optimization technology, stochastic processes, on-line algorithms and multi-criteria analysis. A large amount of approaches were proposed and analyzed in recent years.

11. CC and CIS. Constraints. Scenarios. Beside mentioned aspects, a lot of attacks and malfunctions have been observed in the network, influencing on availability/unavailability of resources of some kind, thus on system dependability. In order to save the viability of service quality after the malfunction, reconfiguration of the system architecture has been recommended. Usually several scenarios of the changes (called sometimes in the literature choreography) are possible and can be considered. Each scenario has been evaluated from several points of view. Our aim is to select the best target scenario. This leads to a problem of discrete optimization (because of finished set of possible scenarios) and multi-criteria (because of the number of evaluation criteria taken into account) with unusual technology of goal function evaluation (simulation). Such view rarely exist.

12. Solution Approach Starting from a critical survey of models and methods used in scheduling and work-loading in CC systems we direct toward an approach using the best approaches recommended in modern optimization for particularly hard discrete single- or multi-criteria problems, with some proposals for using these tools to solve the problem of optimal balancing in CC and reconfiguring of CIS. Note that either balancing in CC as well as flow of task in CIS can be considered as an discrete optimization problem. It is commonly known, that combinatorial optimization (CO) problems derived from practice, due to features of the domain, require especially increased calculation power in order to find a solution satisfactory for the user. Application in real-time systems formulates quite sharp requirements.

13. CC and load balancing

14. CC, load balancing, high availability

15. CC, load balancing, high availability, high capacity

16. CC, load balancing, increased high availability

17. Cloud management. Identical resources. cloud + workload balancer(s)  Communicationlinks + network balancer Communicationlinks + network balancer  

18. Cloud management. Uniform resources. cloud + workload balancer  Communication + network balancer Communication + network balancer  

19. Cloud management. Unrelated resources. cloud + workload balancer  Communication + network balancer  Communication + network balancer 

20. CIS management. Inhomogeneous resources. Dispersed cloud + dispersed workload balancer    Communication + network balancer       Users

21. Optimization tools for balancing. Where we are?

22. Off-line balancing. Deterministic model I. Parallel identical mprocessors, task preemptions, makespan Packing m bins of the calculated size, splitting them if necessary. RR

23. Off-line balancing. Deterministic model II. Parallel identical mprocessors; no task preemption; makespan Schedule: LPT rule; greedy Experimentally: 1.0 ... 1.1

24. Off-line balancing. Deterministic model III. Parallel uniformm processors, task preemptions, makespan • Find assignment of task to processors by using processor share • Find the schedule

25. Off-line balancing. Deterministic model IV. Parallel unrelated m processors, task preemptions, makespan; task j, processor i • Find assignment of tasks to processors • Find the schedule LP program

26. Off-line balancing. Deterministic model V. Parallel uniform m processors, task preemptions; makespan unit processing times Transportation problem: Transportation network: n sources and m*n targets; arc (j,(i,k)) has cost which corresponds to the completion time of task j processed on processor i as k-th in order; variable y

27. Off-line balancing. Deterministic model VI. Parallel uniform mprocessors; no task preemptions; makespan Schedule: modified LPT rule

28. Off-line balancing. Deterministic model VII. Parallel uniform m processors, task preemptions; flowtime unit processing times Transportation problem: Transportation network: n sources and m*n targets; arc (j,(i,k)) has cost which corresponds to the completion time of task j processed on processor i as k-th in order; variable y

29. On-line models. Basic notions.

30. On-line models. Permanent tasks. Summary

31. On-line balancing. Model I. Parallel identical mprocessors; no task preemption; makespan (Greedy) Assign new task to the least loaded processor. Competitive ratio 2-1/m; 3/2 and 5/3 are the best for m=2 and m=3 respectively Improved to 1.945, 1.923 for small m>=4 (LB is 1.852) Randomized: 4/3 for m=2 and (LB=1.582) for m>2

32. On-line balancing. Model II. Parallel uniform mprocessors; no task preemption; makespan

33. On-line balancing. Model III. Limited assignment Parallel identical m processors; no task preemption; makespan

34. On-line balancing. Model IV Parallel unrelated m processors; no task preemption; makespan

35. On-line models. Various task models. Summary

36. MULTICRITERIA OPTIMIZATION. USER PREFERENCES • Known a priori • Known a posteriori • No user preferences • Iteratively extracted • Pareto frontier

37. Pareto front. Hyper-Volume Indicator

38. After the analysis we propose the following taxonomy: • x is deterministic, function K(x) is given by a formula (clear, the most frequent case), • x is deterministic, function K(x) is given by a deterministic polynomial-time algorithm (e.g. longest path in the graph defined by x), • x is deterministic, function K(x) is given by a deterministic exponential-time algorithm (e.g. TSP for given set of cities x), • x is deterministic, function K(x) is given by a deterministic algorithm provided in form of pseudocode or program code, • x is random variable, function K(x) represents certain measure on x (e.g. moments, probability), • x is random variable, function K(x) is given by an algorithm (e.g. RR in simple queening systems), • x is fuzzy variable, function K(x) represents certain defuzzified measure on x • x is any variable, function K(x) is given as the result of running program code (especially the result of a simulation), • x is any variable, function K(x) is given as the result of sensor measurement.

39. Proposals • Set the balancer as the algorithm with some measured values and some control parameters • Measure necessary values, make estimation • Adjust control parameters by solving the optimization problem • Store solution in repository • Evaluate solution by simulation • Extend repository by simulating „if else” cases • Take best solution from the repository

40. CO Troubles • Inability of finding feasible solution • High calculation cost • Slow convergence • Premature convergence • Search stagnation • Imprecise data

41. OPTIMIZATION. TIME/COST OF CALCULATIONS CURSE OF DIMENSIONALITY Pleasewait. Calculationswilllast 3 289 years NP-HARDNESS  LAB INSTANCE 5..20 VARIABLES ! ! ? NONLINEAR FUNCTION OF 1980 VARIABLES !!! INSTANCE FROM PRACTICE

42. Approximate Approaches • constructive/improvement • priority rules • random search • greedy randomized adaptive • simulated annealing • simulated jumping • estimation of distribution • tabu search • adaptive memory search • variable neighborhood search • evolutionary, genetic search • differential evolution • biochemistry methods • immunological methods • ant colony optimization • particle swarm optimization • neural networks • threshold accepting • path search • beam search • scatter search • harmony search • path relinking • adaptive search • constraint satisfaction • descending, hill climbing • multi-agent • memetic search • bee search • intelligent water drops * * * * * METHODS RESISTANT TO LOCAL EXTREMES

43. Balancing as a configuration Referring to the taxonomy provided one can say that the form of optimization task for the case of CIS collapse depends on the style of CIS description.

44. Balancing with data estimation Taking into account fundamental features of the CIS architecture and activity, the simulation seems to be the most adequate method of criteria calculation. Effects of such approach are manifold. Solution corresponds to configuration of services/balancing in CIS, i.e. their distribution among nodes. The service located at the node using contrary policy of queue or resources is treated as different solution. Requirements coming to CIS from workstations are treated as the noise from statistical point of view. Single simulation provides a measurement of some parameter(s) representing criteria treated as observed realization of the random variable. Statistically important sample is necessary

45. CONCLUSIONS • Universal method ? • Data collection/estimation • Adaptation • Repository of workload methods • Repository of scenarios/user profile • Approximate methods • Open repository • Task splitting • Unreliable search • Cost reduction • Green calculations • Multi-criteria analysis • Educational, scientific and business app

46. Thank you for your attention Czesław Smutnicki