1 / 138

Mastering Cloud Computing

Mastering Cloud Computing. Rajkumar Buyya, Christian Vecchiola, Thamarai Selvi. Chapters. • • • • • • • • • • • • • • • •. Part I: Foundations Chapter 1 — Introduction Chapter 2 — Principles of Parallel and Distributed Computing Chapter 3 — Virtualization

spenser
Download Presentation

Mastering Cloud Computing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Mastering Cloud Computing Rajkumar Buyya, Christian Vecchiola, Thamarai Selvi

  2. Chapters • • • • • • • • • • • • • • • • Part I: Foundations Chapter 1 — Introduction Chapter 2 — Principles of Parallel and Distributed Computing Chapter 3 — Virtualization Chapter 4 — Cloud Computing Architecture Part II: Cloud Application Programming and the Aneka Platform Chapter 5 — Aneka: Cloud Application Platform Chapter 6 — Concurrent Computing: Thread Programming Chapter 7 — High-Throughput Computing: Task Programming Chapter 8 — Data Intensive Computing: Map-Reduce Programming Part III: Industrial Platforms and New Developments Chapter 9 — Cloud Platforms in Industry Chapter 10 — Cloud Applications Chapter 11 — Advanced Topics in Cloud Computing

  3. Chapter 1 - Introduction

  4. I have a lot of infrastructure that I want to rent … I need to grow my infrastructure, but I do not know for how long… I have a surplus of infrastructure that I want to make use of I cannot invest in infrastructure, I just started my business…. I have infrastructure and middleware and I can host applications I want to focus on application logic and not maintenance and scalability issues I have infrastructure and provide application services I want to access and edit my documents and photos from everywhere..

  5. Subscription-Oriented Cloud Services: X{compute, apps, data, ..} as a Service (..aaS) Manjrasoft Public Clouds Applications Development and Runtime Platform Compute Cloud Manager Storage Private Cloud Clients Other Govt. Cloud Services Cloud Services 10

  6. Public Clouds Applications Development and Runtime Platform Compute Storage All users, on any device Cloud Manager Private Resources Private Cloud Private Cloud (Government) Organization Personnel Government Agencies

  7. Cloud Deployment Models Private/Enterprise Clouds Hybrid/Inter Clouds Public/Internet Clouds * 3rd party, multi-tenant Cloud infrastructure & services: * Mixed usage of private and public Clouds: Leasing public cloud services when private cloud capacity is insufficient * A public Cloud model within a company’s own Data Center / infrastructure for internal and/or partners use. * available on subscription basis to all.

  8. No capital investments Quality of Service Pay as you go Billing IT outsourcing Security Cloud Computing?

  9. Software as a Service Web 2.0 Interfaces End user applications Scientific applications Office automation, Photo editing, CRM, and Social Networking Examples: Google Documents, Facebook, Flickr, Salesforce Platform as a Service Runtime Environment for Applications Development and Data Processing Platforms Examples: Windows Azure, Hadoop, Google AppEngine, Aneka Infrastructure as a Service Virtualized Servers Storage and Networking Examples: Amazon EC2, S3, Rightscale, vCloud

  10. 2010: Microsoft Azure 1970: DARPA’s TCP/IP 1999: Grid Computing 1984: IEEE 802.3 Ethernet & LAN 1997: IEEE 802.11 (Wi-Fi) 2008: Google AppEngine 1966: Flynn’s Taxonomy SISD, SIMD, MISD, MIMD 1989: TCP/IP IETF RFC 1122 2007: Manjrasoft Aneka 1969: ARPANET 2005: Amazon AWS (EC2, S3) 1984: DEC’s VMScluster 1951: UNIVAC I, First Mainframe 1975: Xerox PARC Invented Ethernet 2004: Web 2.0 Clouds 1990: Lee-Calliau WWW, HTTP, HTML 1960: Cray’s First Supercomputer Grids Clusters Mainframes 1950 1960 1970 1980 1990 2000 2010

  11. Chapter 2 – Parallel and Distributed Computing

  12. ParallelApplications Parallel Programming Environments Sequential Applications Compilers, PVM, MPI, …. Cluster Middleware: Single System Image and Availability Infrastructure PC / Workstation PC / Workstation PC / Workstation PC / Workstation Common SW & Applications Common SW & Applications Common SW & Applications Common SW & Applications Operating System Operating System Operating System Operating System Network Interface HW Network Interface HW Network Interface HW Network Interface HW High speed network connection

  13. Applications User interface for presentation Distributed System Stack Middleware Support for heterogeneous resource sharing, communication, and programming environments for application development Operative System Execution platform including network connectivity services Hardware Computer and network hardware

  14. MPI Programming, JMS, MSMQ, MQS Message Based Communication Paradigms / Architectural Models

  15. Architectures Sequential Era Compilers Applications Problem Solving Environments Architectures Compilers Parallel Era Applications Problem Solving Environments 2030 2020 1990 2000 2010 1970 1980 1960 1940 1950

  16. Instruction Stream Data Input Data Output Processor

  17. Single Instruction Stream Data Input 1 Data Output 1 Processor 1 Data Output 2 Data Input 2 Processor 2 Data Output N Data Input N Processor N

  18. Instruction Stream 1 Instruction Stream N Instruction Stream 2 Single Data Output Stream Single Data Input Stream Processor 1 Processor 2 Processor N

  19. Instruction Stream 1 Instruction Stream N Instruction Stream 2 Data Output 1 Data Input 1 Processor 1 Data Input 2 Data Output 2 Processor 2 Data Output 3 Data Input N Processor N

  20. Cache L1 Cache L1 Core 2 IPC Channel IPC Channel Processor 1 Processor 2 Processor N Processor 2 Processor 1 Processor 2 Memory Bus Memory Bus Memory Bus Memory Bus Local Local Local Global System Memory Memory Memory Memory

  21. Messages IPC Messages IPC Large Level (Processes, Tasks) Task 1 Task 2 Task N Shared Memory Shared Memory function fJ() {…} function f2() {…} function f1() {…} Medium Level (Threads, Functions) Function 1 Function 2 Function J Fine Level (Processor, Instructions) a[1] = … b[1] = … a[k] = … b[k] = … a[0] = … b[0] = … Statements Statements Statements Very Fine Level (Cores, Pipeline, Instructions) x + load

  22. Applications Frameworks for distributed programming Middleware IPC primitives for control and data. Operating System Networking and Parallel Hardware Hardware

  23. Social Networks, Scientific Computing, Enterprise Applications Applications (SaaS) Frameworks for Cloud Application Development Middleware (PaaS) Virtual hardware, networking, OS images, and storage. Hardware and OS (IaaS)

  24. Client-server request Two Tier (Classic Model) client server response Three Tier client server Server/client N Tier server client Server/client Server/client server

  25. P2P peer peer peer peer peer peer peer

  26. RPC Node B Node A Procedure Registry Main Procedure Procedure C:Node B Procedure A Procedure C RPC Service RPC Library Procedure B Program C (RPC Server) Program A (RPC Client) Return Value Marshaling Parameters Marshaling and Procedure Name Return Value Unmarshaling Parameters Unmarshaling and Procedure Name Network

  27. DOF interaction Node B Node A Application B 21 10 16 Instance Remote Instance 15 5: Object Activation 1: Ask for Reference Object Skeleton Object Proxy 11 17 6 20 4 9 14 Remote Reference Module Remote Reference Module Application A 8 7 12 13 3 19 2 18 Network

  28. Web Service WSDL(s) Web Server UDDI Registry WS Client WSDL Web Server Application Web Service WSDL WS Client Application Application

  29. WS technology Stack WSFL Web Service Flow Static  UDDI Service Discovery Quality of Service Direct  UDDI Service Publication Management Service Description Security WSDL XML-based Messaging SOAP Network HTTP, FTP,e-mail, MQ, IIOP, ….

  30. SOAP Messages POST /InStock HTTP/1.1 Host: www.stocks.com Content-Type: application/soap+xml; charset=utf-8 Content-Length: <Size> <?xml version=“1.0”> Envelope <soap:Envelope xmlns:soap=“http//www.w3.org/2001/12/soap-envelope” soap:encondingStyle=“http//www.w3.org/2001/12/soap-encoding”> Header: Metadata & Assertions <soap:Header></soap:Header> <soap:Body xmlns:m=http://www.stocks.org/stock> <m:GetStockPrice> <m:StockName>IBM<m:StockName> </m:GetStockPrice> </soap:Body> Body: Method Call </soap:Envelope>

  31. SOAP Messages POST /InStock HTTP/1.1 Host: www.stocks.com Content-Type: application/soap+xml; charset=utf-8 Content-Length: <Size> <?xml version=“1.0”> Envelope <soap:Envelope xmlns:soap=“http//www.w3.org/2001/12/soap-envelope” soap:encondingStyle=“http//www.w3.org/2001/12/soap-encoding”> Header: Metadata & Assertions <soap:Header></soap:Header> <soap:Body xmlns:m=http://www.stocks.org/stock> <m:GetStockPriceResponse> <m:Price>34.5<m:Price> </m:GetStockPriceResponse> </soap:Body> Body: Execution Result </soap:Envelope>

  32. Chapter 3 – Virtualisation

  33. Virtual Image Applications Applications Guest Virtual Hardware Virtual Storage Virtual Networking Virtualization Layer Software Emulation Host Physical Storage Physical Hardware Physical Networking

  34. Applications Application - level Virtualization Programming Languages Execution Stack Programming Language level Virtualization Operative Systems OS- level Virtualization Hardware Hardware - level Virtualization

  35. Virtual Resources Sharing Isolation Aggregation Emulation Virtualization Physical Resources

  36. How it is done? Technique Virtualization Model Emulation Application Execution Environment Programming Language High-Level VM Process Level Storage Operating System Multiprogramming Virtualization Network Hardware-assisted Virtualization Full Virtualization Hardware System Level Paravirtualization …. Partial Virtualization

  37. Applications Applications API calls API Libraries Libraries User ISA ABI System calls User ISA Operative System Operative System ISA ISA Hardware Hardware

  38. Least privileged mode (user mode) Ring 3 Privileged modes Ring 2 Ring 1 Ring 0 Most privileged mode (supervisor mode)

  39. Guest In memory representation Virtual Image Storage VMM Host emulation Virtual Machine binary translation instruction mapping interpretation …… Host

  40. VM VM VM VM ISA Virtual Machine Manager VM VM VM VM ISA ABI Virtual Machine Manager Operative System ISA ISA Hardware Hardware

  41. Virtual Machine Instance ISA Instructions (ISA) Interpreter Routines Routines Interpreter Dispatcher Allocator Virtual Machine Manager

  42. Privileged Instructions Sensitive Instructions User Instructions

  43. VM VM VM VM VM VM Virtual Machine Manager Server A (running) Server B (running) Before Migration VM VM VM VM VM VM Virtual Machine Manager Server A (running) Server B (inactive) After Migration

  44. User Applications (unmodified ABI) Management Domain (Domain 0) • VM Management • HTTP interface • Access to the Xen Hypervisor Ring 3 User Domains (Domain U) • Guest OS • Modified codebase • Hypercallsinto Xen VMM Ring 2 Ring 1 Ring 0 Privileged instructions Xen Hypervisor (VMM) • Memory management • CPU state registers • Devices I/O Hardware trap Hardware (x86)

  45. User Applications (unmodified ABI) Ring 3 Ring 2 Ring 1 Guest Operating System • Unmodified codebase • VMM unaware Ring 0 Hypervisor • Binary translation • Instruction caching Hardware trap (sensitive instructions) Dynamic / cached translation (sensitive instructions) Hardware (x86)

  46. User Applications I/O Guest Operating System User Applications VMware Workstation Virtual Machine Instance Host Operating System VMware Hypervisor (VMM) • Direct access to hardware •I/O, memory, networking for guests • Save/Restore CPU state for host OS VMware Driver Hardware (x86)

  47. serverd (daemon) VMware VMware Web Server VMware VM Instance VM Instance VM Instance Host Operating System VMware Hypervisor (VMM) • Direct access to hardware •I/O, memory, networking for guests • Save/Restore CPU state for host OS VMware Driver Hardware (x86)

  48. SNMP vxpa CIM broker VM VM VM syslog hostd DCUI VMX VMX VMX Third-party CIM plug-ins VMM VMM VMM User world API Virtual Ethernet adapter and switch Distributed VM file system Network stack Storage stack Resource scheduling Device drivers VMkernel Hardware

  49. Application Virtualization Zimbra Platform Virtualization vFabric vCloud vCenter vCenter Infrastructure Virtualization vSphere vSphere vSphere vSphere ESXi ESXi ESXi ESXi ESX ESX ESX ESX Server Server Server Server Data Center Data Center Cloud

  50. VMWPs User Applications (Ring 3) User Applications (Ring 3) User Applications (Ring 3) WMI VMMS Hypervisor-aware Kernel (Ring 0) Hypervisor-aware Wndows Kernel (Ring 0) Hypervisor-aware Linux Kernel (Ring 0) Hypervisor-unaware Kernel (Ring 0) VSPs VID VSCs / ICs VSCs / ICs WinHv WinHv LinuxHv I/O Stack Drivers I/O Stack Drivers I/O Stack Drivers Latest Pictures. VMBus VMBus VMBus Unenlightened Child Partition Root / Parent Partition Enlightened Child Partition Enlightened Child Partition Hypervisor (Ring -1) Address Management Partition Management Hypercalls MSRs APIC Scheduler Processor Memory Hardware (x86)

More Related