PERFORMANCE MODEL DRIVEN QOS GUARANTEES AND OPTIMIZATION IN CLOUDS

1. Jim (Zhanwen) Li Carleton University John Chinneck Carleton University Gabriel Iszlai IBM Marin Litoiu York University PERFORMANCE MODEL DRIVEN QOS GUARANTEES AND OPTIMIZATION IN CLOUDS ICSE Workshop on Software Engineering Challenges of Cloud Computing Presented by: Yun Liaw

2. Outline • Introduction • Cloud Architecture • Engineering for QoS Optimization • Case Study • Conclusions and Comments

3. Introduction • Cloud management is responsible for all resources used by all applications deployed in the cloud • The opportunity for global resource optimization is a major driver for Cloud implementation

4. Introduction • Each application and resource have their own price, thus the profit of the application provider (AP) and cloud provider (AP) can thus be calculated • But QoS is another goal that needs to be achieved of cloud management – constraint of optimization • For simplicity, this paper only consider the response time as the QoS parameter • This paper assumes that: “When the total AP profits are maximized, the CP can arrange that its profit is also maximized”

5. Cloud Architecture • The three-level hierarchy Cloud Architecture • Application developer: tunes the code over time, discovering and improving inefficient operations • AP admin: tunes the runtime configuration to make the best use of the existing resources • CP admin: physical resource allocation

6. Service Architecture • Assume that deployable unit is the concurrent process, termed a task • A service model comprises UserClass, Service, ServerTask, Resource and Host • UserClass raises request from outside • Services request each other inside and outside of the service, forming a web of inter-service traffic • Service are implemented by Applications which run as system tasks or thread pools (ServerTasks)

7. Service Architecture A simplified metamodel for a service architecture

8. Service Architecture Represented by LQM Host processor LQM: Layered Queuing Model

9. The Performance Model • The role of performance model • To predict the effect of simultaneous changes to many decision variables • To assist in making optimal decisions over many variables • Layered Queuing Model [*] • A kind of extended queuing network • Can be solved to predict:throughputs, mean queuing delays, mean service delays at entries, and resource utilization • Benefit of using LQM here: corresponds to architecture, represents the layered resource behavior [*] G. Franks, et al., “Layered Bottlenecks and their mitigation,” Proc. Int. Conf. on Quantitative Evaluation of Systems, 2007 [*] G. Franks, et al., “Enhanced Modeling and Solution of Layered Queuing Networks,” IEEE Trans. On Softw. Eng., 2008 [*] J. Rolia, et al., “The Method of Layer,” IEEE Trans. Softw. Eng., 1995

10. Workload and QoS Requirement • Workload describes the intensity of the streams of user requests for service, in terms of • throughput fcfor user class c, • or the number Nc of users that are interacting and their think time Zc • Zc represents the user’s mean delay between receiving a response, and issuing its next request • Assume that each service has its own service class c, with Nc users and think time Zc sec • The QoS requirement of each class can be defined by: • Throughput fc, min • Response time: Rc, max • By Little’s Law, fc ≧ fc,min = Nc / (Rc, max + Zc) Arrival rate (throughput) Queue length Little’s Law: L = λ×W Waiting time

11. The Optimization Loop

12. Engineering for QoS and Optimization • The design requirements of Cloud: • The application software should be efficient and adaptable to different run-time situation • The cloud must provide infrastructure for deploying and monitoring application elements and user QoS • The cloud feedback loop must be able to track the performance model and to optimize the management decisions

13. Developer Responsibility • Mostly the cloud hides resource management from the developer, but for QoS, efficient execution is the developer’s responsibility • Another subtle goal for the developer is to provide flexibility in the concurrency architecture (the allocation of functions to tasks) • Concurrency creates flexibility, but introduces overheads • The developer also needs to provide the structure of the performance model • E.g., the service’s component interact diagrams

14. The estimation tool for updating the model param. periodically The model-based optimization architecture

15. Optimization technique • Network Flow Model (NFM) • Shows the flows of execution demands (in CPU-sec per sec) at the processors • And how they combine to produce flows at the tasks, services and user-responses • Nodes in NFM: • Nodes for processors have a flow equal to the processor utilization • Nodes for tasks have flows from all processors on which the task is deployed • Nodes for user classes have flows from the tasks which implements the service

16. [*] Z. Li, et al., “Fast Scalable Optimization to Configure Service Systems having Cost and Quality of Service Constraints,” in Proc. International Conference on Autonomic Computing, 2009 NFM Example The maximum demand rate available at host h – host h’s capacity [X,Y,Z] X: the lower flow bound Y: the upper flow bound Z: the cost per flow unit The demand rates Demand rate of a request issued by user Fc = γsc/dsc

17. Optimization technique

18. Objective Function • Each Service class c offered to users has a price per response of Pc • Each host h has a price of CPU execution of Ch per CPU-sec, including unused CPU-sec allocated in order to reduce contention delays • Each task t has a reservation αht, in CPU-sec per sec, on some host h • The Application App has a profit function: ψApp: sets of user classes τApp: sets of tasks

19. Objective Function • Remark that this paper assumed that if the AP’s profit is maximized, the CP’s profit is also maximized • Constraints of the objective function: • A maximum user response time Rc, max for each class c, or a minimum class throughput fc, min • A minimum profit PROFITapp, min for each application

20. Case Study • A Cloud with VMs and from 1 to 50 application instances • Applications are the same but with different parameters • Two deployment scenarios: • “Incremental” deployment scenario: Place each application on a sufficient number of processors to meet its profit and QoS constraints, disjoint from already allocated processors • “full” deployment scenario: The deployment is optimized the overall profit based on the previously mentioned objective function • Pros: The full deployment can maximize the profit • Cons: When the new instance is created, the existing instance also needs to be re-deployed

21. Case Study - result

22. Conclusion & Comments • Conclusion • An approach of resource allocation optimization is summarized • This approach is effective and scalable, and the prototype has been implemented • Further development: take into account for memory allocation, communication delays, VM overhead costs, and licensing costs of software replicas • Comments: • The assumption related to the CP profit does not verified • No much details described • References of this paper (performance models) might be valuable for my ongoing work