1 / 29

CS6223: Distributed Systems

CS6223: Distributed Systems. Cloud Computing. Origin of the Term “Cloud Computing”. “Comes from the early days of the Internet where we drew the network as a cloud… we didn’t care where the messages went… the cloud hid it from us” – Kevin Marks, Google

lea
Download Presentation

CS6223: Distributed Systems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CS6223: Distributed Systems Cloud Computing

  2. Origin of the Term “Cloud Computing” “Comes from the early days of the Internet where we drew the network as a cloud… we didn’t care where the messages went… the cloud hid it from us” – Kevin Marks, Google First cloud around networking (TCP/IP abstraction) Second cloud around documents (WWW data abstraction) The emerging cloud abstracts infrastructure complexities of servers, applications, data, and heterogeneous platforms

  3. A Working Definition of Cloud Computing Cloud computing is a model for enabling convenient, on-demandnetwork access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management or configuration effort. The cloud model has the following features: 5 essential characteristics, 3 service models, and 4 deployment models.

  4. Five Essential Cloud Characteristics • On-demand self-service • Broadband network access • Resource pooling • Location transparency • Rapid elasticity • Measured service

  5. Three Cloud Service Models • Software as a Service (SaaS) • Use cloud’s applications (S/W) over a network • Platform as a Service (PaaS) • Deploy customer-developed applications in a cloud • Infrastructure as a Service (IaaS) • Rent processing, storage, network, and other fundamental computing resources

  6. Service Model Architectures

  7. Four Cloud Deployment Models • Private cloud • enterprise owned • Community cloud • shared infrastructure for specific community • Public cloud • Sold to the public, mega-scale infrastructure • Hybrid cloud • composition of two or more clouds

  8. Possible Trends of Cloud Computing Small enterprises use public SaaS and public clouds and minimize growth of data centers Large enterprise data centers may evolve to act as private clouds. They may also use hybrid clouds to leverage both private and public clouds Public clouds may adopt standards in order to run workloads from hybrid cloud infrastructures 8

  9. Core objectives of Cloud Computing Core objectives and principles of cloud computing: Security Scalability Availability Performance Cost-effective On-demand acquire / release resources Pay for what you use Leverage others’ core competencies Turn fixed cost into variable cost 9

  10. Analyzing Cloud Security • Some key issues: • trust, multi-tenancy, encryption, compliance • Clouds are massively complex systems that consist of standard functional units replicated thousands of times • Cloud security is a tractable problem • There are both advantages and challenges

  11. Security Advantages of Cloud Computing More resources for security, e.g., dedicated security team, good security infrastructure, fault tolerance and reliability, … Hypervisor protection against network attacks Data held by unbiased party On-demand security controls Real-time detection of system tampering Low-cost disaster recovery and data storage solutions Rapid re-constitution of services …… 11

  12. Challenges of Cloud Security • Data dispersal and international privacy laws • Possible massive outages • Multi-tenancy (with conflict of business interests) • Attraction to hackers (high value target) • Difficulties in keep data access logging • Data ownership issues • Security of virtual OSs in the cloud • Encryption needs for cloud computing • ………..

  13. Foundational Elements of Cloud Computing Primary Technologies Virtualization Service Oriented Architectures Distributed Computing Broadband Networks Browser as a platform Free and Open Source Software ……. • Other Technologies • Web 2.0 • Web application frameworks • Service Level Agreements • Autonomic Systems • ……

  14. Virtualization Cloud computing separates applications from the underlying infrastructure Host operating system provides an abstraction layer for running virtual guest OSs The key is the “hypervisor” or “virtual machine manager” Enable guest OSs to run in isolation of other OSs Run multiple types of OSs Enable portability (migration) of virtual servers between physical servers Increase utilization of physical servers

  15. Utility Computing • “Computing may someday be organized as a public utility” - John McCarthy, MIT Centennial in 1961 • Huge computational and storage capabilities available from utilities • Metered billing (pay for what you use) • Simple to use interface to access the capability (e.g., plugging into an outlet)

  16. Web 2.0 Web 2.0 is the trend of using the full potential of the web Viewing the Internet as a computing platform Running interactive applications through a web browser Interconnecting mobile devices Enhanced effectiveness with greater human participation “Web 2.0 is the business revolution in the computer industry caused by the move to the Internet as a platform, and an attempt to understand the rules for success on that new platform.” – by Tim O'Reilly

  17. Web Services • Self-describing and stateless modules that perform discrete units of work and are available over the network • Web service providers offer APIs that enable developers to exploit functionality over the Internet, rather than delivering full-blown applications. • Standards based interfaces • e.g., SOAP, WSDL, WS-Security, WS-Transaction • Many loosely coupled interacting modules form a single logical system (e.g., legos)

  18. Major Cloud Platforms Amazon’s Elastic Compute (EC2) and Simple Storage (S3) Google’s Google Apps Microsoft’sAzure Saleforce.com’sCRM clouds (e.g.,Sales Cloud, Service Cloud and Force.com) IBM’s Smart Business portfolio 18

  19. Examples of Google Cloud Users The government of City of Washington D.C. moved most of the government functions to Google Cloud: Migrating data of 38,000 employees to Google Apps Replace office software by Gmail Google Docs (word processing and spreadsheets) Google video for business Google sites (intranet sites and wikis) 500,000+ organizations use Google Apps GE moved 400,000 desktops from Microsoft Office to Google Apps and then migrated them to Zoho for privacy concerns 19

  20. Facebook’s Use of Open Source and Commodity Hardware (8/08) By Jonathan Heiliger, Facebook's vice president of technical operations: 80 million users + 250,000 new users per day 50,000 transactions per second, 10,000+ servers Built on open source software Web and App tier: Apache, PHP, AJAX Middleware tier: Memcached (Open source caching) Data tier: MySQL (Open source DB) Thousands of DB instances store data in distributed fashion (avoids collisions of many users accessing the same DB) Use redundancy of simple hardware devices for reliability

  21. Amazon Cloud • Amazon cloud components • Elastic Compute Cloud (EC2) • Simple Storage Service (S3) • SimpleDB • New Features • Availability zones • Place applications in multiple locations for failovers • Elastic IP addresses • Static IP addresses that can be dynamically remapped to point to different instances (no change of DNS)

  22. CityU CSLab Cloud Configuration (a case study) • This is one rack (a large DataCenter has rows of these racks). This rack is IBM Blade Center. • A rack has many rack-units (server boards inserted into the rack). Each rack-unit has 2 XEON CPUs, memory (16 – 96GB), and disk (~100G). The local disk on each rack-unit is small, only for O/S (no data or FS). • Each rack has 3 storage arrays (30TB in total) connected by SAN (Storage Area Network) to all rack-units, i.e., they all share the same disk storage. • The disk storages of racks are further interconnected by optical fibers (high speed).

  23. Clusters and VM Configuration • All server boards (rack units) are grouped into clusters. In Cslab, 14 boards are configured into 4 clusters (a cluster acts like a big server), each with around 4 rack-units. • Usually the same type of svr-boards (with the same CPU platform) are grouped into the same cluster. This configuration is easy for VM automatic migration between svr-boards in the same cluster. • VMs are configured on clusters. There can be hundreds VMs configured on a cluster. Sys-Admin uses an interface (supported by VMWare) to create a VM by specifying CPU, memory, storage requirements, and VM platform (i.e., UNIX, Linux, Solaris, MS-Window, etc).

  24. VM Configuration / Mornintoring • The Sys-Admin can use the interface to monitor the clusters (and svr-boards inside each cluster), VMs on each cluster (and svr-boards), and actual resource usage of VMs. • A VM can be for a server, a web-server, database server, etc, or a server for all students to do a course work for a subject. A VM can also be configured for each staff office’s desktop (no PC is needed in staff’s office). • The Max number of VMs on a cluster is mainly limited by memory and storage size (they cannot be shared). • A VM has its own IP address / domain name, all the same as a physical machine. Users can run all kind of applications on the VM. • A VM can work for relatively a long time, from a day to a few years. It can survive the underneath H/W (CPU or storage) replacement.

  25. VM Auto-Migration • When one svr-board needs to stop for maintenance, Sys-Admin can mark the server for maintenance through the admin interface. Then, all VMs on this server can automatically migrate to other servers without stopping the applications running on the server. • Applications on the VM may suffer some delay (or packet loss) if they involve network access, because VM migration involves the change of MAC address (from one board to another) and the change of packet routing. • This migration can also happens for load balancing purpose. The system software can monitor the load on all servers, and auto-migrate VMs from heavy loaded servers to light ones.

  26. VM Migration within & across Clusters • Another mode for VM management is “energy saving”. If this mode is turned on, the system will try to auto-migrate the VMs to some of the server-boards, and turn off the power of the rest of idle server-boards for energy saving. • Since all VMs on the same cluster share the same disk storage (some clusters, particularly connected by SAN, also share the same disk), it is easy to migrate VMs between server-boards and this migration can be done automatically. • Each VM is represented as a “folder” in the disk storage system and the files on this VM are all under this “folder”. • The migration of VMs across clusters may need Sys-Admin to use the interface to do it manually.

  27. Data Center Networks (DCN) • This is a floor map of CityU’s data center. It has rows of racks. Each little box of a row is a rack. • Data center network is a LAN that inter-connect server farms inside a data center and further connect the data center to the Internet.

  28. CISCO’s Architecture of DCN • A DCN is in a tree structure, where the top are CRs (core routers) that connect the data center to the Internet. • CRs are connected to Layer-2 domains by ARs and ASs. • ARs (access router) and ASs (aggregation switch) are duplicated for reliability (a pair for each Layer2 domain). • Switches under each pair of ARs (or ASs) form a layer-2 domain. • A Layer-2 domain typically has several thousands of servers. • The bottom of the tree are rows of ToRs (Top of Rack), each ToR typically has 20 – 40 servers.

  29. Thoughts on Cloud Computing…… “A way to increase capacity or add capabilities on the fly without investing in new infrastructure, training new personnel, or licensing new software.” “The idea of loosely coupled services running on an agile, scalable infrastructure should eventually make every enterprise a node in the cloud.” “We are at the beginning of the age of planetary computing. Billions of people will be wirelessly interconnected, and the only way to achieve that kind of massive scale usage is by massive scale, brutally efficient cloud-based infrastructure.” …… 30

More Related