Middleware Challenges for the Emerging Application Environments

1. Middleware Challenges for the Emerging Application Environments Giovanni Pacifici(giovanni@us.ibm.com)

2. Real-Time Enterprise N2 World • Pervasive connections • Everyone is a web app developer Complexity • Multimodal applications • Time critical events • Business agility • Software stack complexity becoming unmanageable • Exploding number of apps • Exploding Interdependence • Heterogeneous environments Emerging Application Environments

3. Scalable Responsive Simplified Enable Infrastructure Scalability, Simplification and Responsiveness Peer to Peer Plug and Play Policy Driven Event Driven Autonomic Abstraction, Encapsulation, Virtualized

4. Configuration Manager Client Node 1 ST AM Node 2 Client ST FA Node 3 L4 Switch FA AM Client Node 4 FA AM Current Scalability Limitations of Middleware and Multi-Tier Architectures L7 Switch • Centralized management model • Specialized nodes • Significant administrative overhead to grow or shrink a deployment • Not scalable communication infrastructure • Centralized and not scalable performance management controllers

5. ST AM Client ST AM Client L4 Switch ST Client FA FA AM FA AM Middleware Self-Organization Approach to Scalability Node 1 Node 2 Node 3 Node 4 • no specialization of nodes • decentralized model • self-configurable infrastructure • scalable architecture and management Node 5

6. Key Functionalities Request routing and load balancing Self-organizing Self-healing Configuration data dissemination Single-console system monitoring Middleware Scalability: Challenges peer

7. Scalable Responsive Simplified Enable Infrastructure Scalability, Simplification and Responsiveness Peer to Peer Plug and Play Policy Driven Event Driven Autonomic Abstraction, Encapsulation, Virtualized

8. Application Architecture Complexity

9. App App App Abstracted APIs Abstracted APIs Virtualization (Mapping and Substitution) Resource APIs Resource APIs Physical Resources Physical Resources How Do We Deal with This Mess? • Virtualization: “A technique for hiding the physical characteristics of computing resources from the way in which other systems, applications, or end users interact with those resources” • Dominant benefit of virtualizationgoing forward • Abstraction of physical interfaces • Isolation

10. Virtual Application Server Virtual Application Server Virtual Environment Virtual Application Server Virtual Servers Virtual Storage Virtual Networks Virtualization Decouples Virtual and Physical Environments Physical Environment Fixed sizes, limited ports/slots, incompatible versions, rigid configurations, workloads bound to boxes, … NetworkHardware Blades Storage Serversand Storage SMP Servers Reducing Complexity Through Virtualization • Virtual resources and their configurations are decoupled and insulated from physical environment • Durability: limits the impact of physical changes • Pre-built virtual resources serve as the units of product distribution and provisioning • Greater flexibility for allocating computing resources when needed and where needed • Deploy to Resource pools • Goal-based management • Virtualization will be extended in scope from single server to aggregations of servers, storage, and network components. • From making partitioning technology a large system look like many – partitioning technology • Into make many small systems look like one from a management perspective

11. Virtualization is a Disruptive Technology • Will transform data center management • Virtualization will extend beyond single systems to multi-system pools consisting of servers, network and storage, thus creating a new platform for integrated management and optimization of data center resources. • Will transform desktop management • The Enterprise desktop will become a virtual machine image, standardized by the IT staff, secured by Enterprise policies, and streamable to hosted servers or client machines. • Will transform software lifecycle management • Virtual appliances will become the unit of software distribution, licensing, maintenance, archival and service/support. • Will drive new hardware, software and services technologies • Hardware support for virtualization, new programming models, new licensing models, new service & support models.

12. Virtual Machine Resource Definition Resource Definition Image Image Middleware Middleware Operating System Operating System Virtual Software Appliances Appliances Virtualization • Virtual Appliances: pre-wired, pre-configured, production-ready software stack packaged inside virtual machine images designed to run under a VM hypervisor • Contains customization logic • May contain management agents • Associated meta-data manifest describing capabilities and requirements • Marrying Appliances with Virtualization • Appliances: ease of use, purposed • Virtualization: fast replication, isolation, consolidation • Change the way enterprise software is packaged and distributed, allowing for the development of self-contained application stacks that are easy to deploy and more reliable than traditional methods • Change the way enterprise software is managed by including management intelligence into an appliance thereby making it easy to manage from the outside • Emerging Technologies and Research Areas • Best of breed self-managing virtual appliances focusing on multi-image ones (end-to-end solution) • Develop tools to create, configure, provision and life-cycle manage virtual appliances • Develop techniques to manage virtual appliances at runtime to ensure high performance, availability, and electrical power conservation VirtualAppliance • Integrated software stacks for easier production usage by partners and customers • Preinstall and configured • Common management enablement • Common patterns • Management functionality

13. VSA VSA VSA VSA VSA VSA VSA VSA Repository VSR Registry DeploymentManager VSR VSR VSR Configuration and Lifecycle Management Virtual Software Appliance (VSA) Vendor Environment Internet Application Structure Developer Logical Topology VSA Factory VSA Engineer Register VSA Repository Datacenter Deployer Customer Environment Virtual Software Resources (VSR) Deployed, configured and running instance of VSA

14. Application Middleware OS VM Hypervisor Physical Node Deployment and Activation of Virtual Appliances VSA Master Image VSA Clone Deployment Configuration Parameters VSA OS Configuration VSA OS Configuration VSA Stack Configuration VSA Stack Configuration Application Middleware OS DeploymentManager VSA Model CapabilityRequirements VSA Model CapabilityRequirements CMDB VSA Topologyvalidation and resolution logic

15. Model Driven Deployment: Adding Application to Middleware VSAs Drag and Drop VSAs

16. Model Driven Deployment: Adding Application to Middleware VSAs Drag and Drop Logical App Structure

17. Model Driven Deployment: Adding Application to Middleware VSAs Create LAS → VRST Hosting Links • Configuration of both containers auto updated based on requirements from LAS

18. Model Driven Deployment: Adding Application to Middleware VSAs Save Topology

19. Model Driven Deployment: Adding Application to Middleware VSAs Drag and Drop Physical Resources (VM Hosts)

20. Model Driven Deployment: Adding Application to Middleware VSAs Create VSRT → VM Hosting Links

21. Model Driven Deployment: Adding Application to Middleware VSAs Deploy Topology to Physical Resources

22. Life Cycle Management of Virtual Appliances • Two approaches: • update by replacement – a new version of appliance is created by vendor and shipped to customer • internal update – each VM is individually updated with patches of its software stack • Update by replacement: • state management problem – new appliance does not include the state acquired by the old appliance • customization parameters • business application installs • runtime data (caches, cookies, sessions, etc.) • downtime problem – old appliance must be brought down before new appliance may be started • Internal update • difficult to generalize as different software stacks may require different match mechanisms • Hybrid approaches possible

23. 100% 50% 0% 55%* Utilized Servers Node 1 ST AM FA Classification – Flow Control - Routing Placement Executions Node 2 ST AM Stock Trading Node 3 ST FA Account Mngmt Financial Advice Node 4 On Demand Router ST FA Runtime Management: from JVMs to Virtual Appliance Management • Runtime management in existing middleware infrastructures • goal oriented resource management for web application environments • Supports multi-tiered applications where each request uses multiple resources distributed • Supports multiple applications deployed and replicated on different but overlapping subsets of machines • Expand to manage Virtual Software Appliances and heterogeneous workloads • Manage both request/response workloads and long-running workloads like batch jobs on same pool • Leverage virtualization technology to enable “anywhere” placement of any workloads • Leverage new control knobs: migration, suspension, resource control • Make many small systems look like one from a management perspective Virtual Resource Pool

24. Anywhere Placement of Workloads Job Submission and Monitoring Lucene Povray Blast Povray DB Inst B DB Inst B DB Inst A J2EE App J2EE App DB Inst A DB Inst A DB Inst B J2EE App  Blade 1  Blade 2  Blade 3  Blade 4 Virtual Server Resource Repository App Job Scheduling and Placement Controller Web Request Flow Controller WAS OS Suspended VSRs VSR

25. Merging Job Scheduling and Placement • Decides when, where, and how may instances of each container should run • Application characterized by memory and CPU requirements • Resource Requirements derived from performance goals • Average response time goal for web applications • Completion time goal for long running jobs • Server machine characterized by memory and CPU capacity • Application placement algorithms • Bases on multi-dimensional bin packing techniques • Constraints • Memory used by applications and their containers does not exceed a threshold on any server • CPU usage of applications and their containers does not exceed a threshold on any server • Constraints on the number of servers where an application should run, on the number of instances of an application that may be started on a node, etc. • Collocation restrictions and allocation restrictions • Objectives • Fairness – equalize application utility whenever possible • Minimize the number of placement changes

26. Use Virtualization for Power Management Power vs. Performance prototypical example • Key observations: • The majority of the power used by a blade is static (i.e., used before workload is started) • Can be as much as ~80% • An (over)simple calculation: • 2 blades, each 40% busy: ~170 watts • 1 blade, 80% busy: ~95 watts • Savings = ~44% • Power usage grows with workload intensity: linearly or as a convex function (when frequency scaling is implemented) • Key energy-saving strategies: • Workload consolidation and machine shut-down • Workload reduction (via flow control) • Workload distribution CPU-intensive workload(s) w/ allocation limited by hypervisor 4 VMs running (idle) Power usage (watts) 1 VM running (idle) Hypervisor only powered off CPU usage (MHz)

27. Summary • Middleware infrastructures must address key challenges • Scalability • no specialization • Self-configuration • Complexity • Achieve simplification using pre-configured software stacks inside virtual containers • Model-driven tools to simplify deployment, provisioning and change management • Flexibility • Enable “seamless” and “anywhere” placement of heterogeneous workloads through isolation, migration, suspension and resume techniques • Make many small systems look like one from a management perspective • Future Challenges • Manage software updates • Integrate security concerns • Physical and virtual configuration and connectivity • Software stack proliferation

28. Thank You giovanni@us.ibm.com