雲端計算 Cloud Computing

1. 雲端計算Cloud Computing Text Book:Cloud Computing for Machine Learning and Cognitive Applications, by Kai Hwang, The MIT Press, 2017 (開發圖書代理)

2. Info • Time：Tue. 13:10~15:00 (Room 208), 15:10~16:00 (Lab 501) • Evaluation： • LabExperiments 20% • Operation Tests (Twice) 40% • Midterm (Written Test) 20% • Final (Written Test) 20% • Website：http://ares.ee.nchu.edu.tw/Course.files/cc107/index.html

3. Outline • Principles of Cloud Computing Systems • Virtual Machines, Docker Containers, and Server Clusters • Cloud Architectures and Service Platform Design • Cloud Programming with Hadoop and Spark • Machine Learning Algorithms and Model Fitting • Intelligent Machines and Deep Learning Networks

4. Chapter 1Principles of Cloud Computing Systems

5. Elastic Cloud Systems for Scalable Computing • Traditional computer systems have emphasized high-performance computing (HPC) applications • In terms of raw speed in batch processing • The new demand for network-based computing requires high-throughput computing (HTC) systems • Built with parallel and distributed computing technologies • Triggered the upgrading of many data centers into Internet clouds • Can serve millions of users simultaneously

6. Elastic Cloud Systems for Scalable Computing(cont.) • To advance big data computing in web, cloud, and Internet of things (IoT) services • Building HTC clouds using low-cost servers, distributed storage systems, and high-bandwidth networks

7. Enabling Technologies for Cloud Computing • The driving forces behind cloud computing • The ubiquity of broadband and wireless networking, falling storage costs, and progressive improvements in Internet computing software • Cloud users are able to demand more capacity at peak hours, reduce costs, experiment with new services, and remove unneeded capacity • Service providers can increase the systemutilization via multiplexing, virtualization, and dynamic resource provisioning • The concept of cloud computing • Evolved from cluster, grid, and utility computing

8. Enabling Technologies for Cloud Computing(cont.) • Cluster and grid computing leverage the use of many computers in parallel • Utility and software as a service (SaaS) provide the computing resources • Cloud computing leverages dynamic resources to deliver a large number of services to end users • Frees up users to focus on user applications development • By outsourcing the job execution to cloud providers

9. Enabling Technologies • Clouds are enabled by the progress in developing new hardware, software, and networking technologies • These technologies play instrumental roles in making cloud computing a reality • Most of these technologies are mature enough today to meet the increasing demand • The rapid progress in multicore CPUs, memory chips, and disk arrays • Possible to build faster data centers with huge storage spaces • Resource virtualization • Enables rapid cloud deployment with HTC and disaster recovery capabilities

10. Enabling Technologies(cont.)

11. Enabling Technologies(cont.) • The progress of providing SaaS, Web 2.0 standards, and Internetperformance • All contributed to the emergence of cloud services • Clouds are designed to serve numerous tenants over massive volumes of data • The availability of large-scale, distributed storage systems is the underlying foundation of today’s data centers • Cloud computing is greatly benefitted by • The progress made in license management and automatic billing techniques

12. Enabling Technologies(cont.) • Private clouds within an organization • Easier to secure and are more trustworthy than public clouds • Could be opened or converted to public clouds • Once the private clouds become mature and more secure • The boundary between public and private clouds might blur in the future • Most future clouds will likely be hybrid in nature

13. Convergence of Technologies • Cloud computing is enabled by the convergence of four technologies • Hardware virtualization and multicore chips • Possible to have dynamic configurations in clouds • Utility and grid computing technologies • Lay the necessary foundation for cloud computing • Recent advances in service oriented architecture (SoA), Web 2.0, and mashups of platforms • Push the cloud another step forward • Autonomic computing and automated data center operations have enabled cloud computing • Cloud computing explores multicore and parallel computing technologies

14. Convergence of Technologies(cont.)

15. Convergence of Technologies(cont.) • To realize the data-intensive systems • One needs to integrate hardware, Internet, and data centers • Today’s Internet technology emphasizes SoA and Web 2.0 services • Utility and grid computing lay the distributed computing foundation • Needed for cloud computing • The widespread use of data centers with virtualization techniques • Applied to automate the resources provisioning process in clouds

16. Convergence of Technologies(cont.) • Utility computing is based on a business model • Customers receive computing resources from cloud or IoT service providers • Presents some technological challenges in almost all aspects of computer science and engineering • e.g., Users may demand new network-efficient processors, scalable memory and storage schemes, distributed operating systems (OS), middleware for machine virtualization, new programming models, effective resource management, and application program development • Necessary to facilitate mobile cloud computing in various IoT application domains

17. Evolution of ScalableDistributed/ParallelComputing • The general computing trend is toward increased leveraging on shared web resources over the Internet • The evolution from two tracks of system development • HPC systems and HTC systems • On the HPC side, supercomputers are gradually replaced by clusters of cooperative computers • Out of the desire to share computing resources • Supercomputers - Massively parallel processors, MPP • The cluster is a collection of homogeneous compute nodes physically connected in close range to one another

18. Evolution of ScalableDistributed/ParallelComputing (cont.) • On the HTC side, peer-to-peer (P2P) networks are formed for distributed file sharing and content delivery applications • Emphasis from the HPC paradigm to the HTC paradigm • P2P, cloud computing, and web service platforms focus more on HTC than HPC applications • The HTC paradigm pays more attention to high-flux multicomputing • Internet searches and web services are requested by millions of users simultaneously • The new performance goal • Shifted from speed to measuring the high throughput or the number of tasks completed per unit of time

19. Evolution of ScalableDistributed/ParallelComputing (cont.)

20. Evolution of ScalableDistributed/ParallelComputing (cont.) • Cost, energy efficiency, security, and reliability in clouds are also of vital importance • In this era of big data, a data deluge problem are faced • Data comes from IoT sensors, lab experiments, simulations, society archives, and the web in all scales and formats • Preservation, movement, and access of massive data sets require generic tools • Supporting high-performance, scalable file systems, databases, algorithms, workflow, and visualization • A new data-centric paradigm of scientific discovery is based on data-intensive technologies

21. Evolution of ScalableDistributed/ParallelComputing (cont.) • New tools for data capture, data creation, and data analysis are needed • The cloud and IoT technologies are driven by the surge of interest in the data deluge situation • The Internet and World Wide Web are used by billions of people every day • Large data centers or clouds must be designed to provide not only big storage • But also distributed computing power to satisfy the requests from a large number of users simultaneously • The emergence of public or hybrid clouds requires upgrading many data centers • Using larger server clusters, distributed file systems, and high-bandwidth networks

22. Evolution of ScalableDistributed/ParallelComputing (cont.) • With massive smartphone and tablet usage requesting services • The cloud engines, distributed storage, and mobile networks must interact closely with the Internet • To deliver mashup services in web-scale mobile computing over the social and media networks • Advances in virtualization make it possible • To use Internet clouds to process a huge number of user service requests • The differences among clusters, P2P systems, and clouds may become blurred • Some view the clouds as computing clusters with modest changes in virtualization

23. Evolution of ScalableDistributed/ParallelComputing (cont.) • Others anticipate the effective processing of huge data sets • Generated by web services, social networks, and IoT • In this sense, many users consider cloud platforms a form of utility computing or service computing • The basic architecture and design considerations of data centers • A cloud architecture is built with commodity hardware and network devices • Almost all cloud platforms choose the popular x86 processors • Low-cost terabyte disks and gigabit Ethernet are used to build data centers

24. Evolution of ScalableDistributed/ParallelComputing (cont.) • Data center design focuses more on the performance/price ratio than speed performance alone • Storage and energy efficiency are more important than sheer speed performance

25. Data Center Growth and Cost Breakdown • A large data center may be built with thousands of servers • Smaller ones are built with only hundreds of servers • The cost to build and maintain data center servers has increased over the years • The cost of utilities exceeds the cost of hardware after just three years • The cost to run a data center is dominated by about 60% in management and maintenance costs • The server purchase cost did not increase much with time

26. Low-Cost Design Philosophy • High-end switches or routers may be too cost-prohibitive in building data centers • Using high-bandwidth networks may not fit the economics of cloud computing • Commodity switches and networks are preferred in data centers • Using commodity x86 servers is more desirable over the use of expensive mainframes • The software layer handles the network traffic balancing, fault tolerance, and expandability • Nearly all the cloud computing data centers use the Ethernet as the fundamental network technology • There exists a cycle of interactions among four technical areas

27. Low-Cost Design Philosophy (cont.) • The cloud technology is driven by the surge of interest on data deluge • Cloud computing impacts the eScience greatly • eScience is computationally intensive science carried out in highly distributed network environments • Or science that uses immense data sets that require grid computing • It explores the multicore and parallel computing technologies, and in turn enable a data deluge • To realize the vision of data-intensive systems and building of generic tools • Needs to address workflows, databases, algorithms, and virtualization • Cloud computing is a transformative approach

28. Low-Cost Design Philosophy (cont.) • Promises much more than the data center model • Fundamentally changing how to interact with information • The cloud provides services on demand • e.g., Infrastructure, platform, or software • At the platform level, MapReduce offers a new programming model • Transparently handles data parallelism with natural fault tolerance capability

29. Virtualized Resources in Cloud Systems • Working with large data sets will typically mean sending the computations (programs) to the data • Rather than copying the data to the workstations • This reflects the trend in IT • Moves computing and data from desktops to large data centers • On-demand provision of software, hardware, and data as a service are available • Data explosion promoted the idea of cloud computing • A cloud is a pool of virtualized computer resources

30. Virtualized Resources in Cloud Systems (cont.) • A cloud can host a variety of different workloads • Including batch-style backend jobs and interactive, user-facing applications • A cloud allows workloads to be deployed and scaled out quickly • Through the rapid provisioning of virtual machines (VMs) or physical machines (PMs) • The cloud supports redundant, self-recovering, highly scalable, programming models • Allow workloads to recover from many unavoidable hardware or software failures • The cloud system should be able to monitor resource use in real time • To enable rebalancing of allocations when needed

31. Internet Clouds • Cloud computing applies a virtualized platform with elastic resources on demand • By dynamically provisioning hardware, software, and data sets • The idea is to move desktop computing to a service-oriented platform • Using server clusters and huge databases at data centers • Cloud computing leverages its low cost and simplicity to benefit users and providers • Machine virtualization has enabled such cost-effectiveness

32. Internet Clouds (cont.) • Cloud computing is intended to satisfy many user applications simultaneously • The cloud ecosystem must be designed to be secure, trustworthy, and dependable • Some computer users think of a cloud as a centralized resource pool • Others consider a cloud as a server cluster which practices distributed computing over utilized servers • Cloud computing is a large-scale distributed computing paradigm • Driven by economics of scale • A pool of abstracted virtualized, dynamically-scalable, managed computing power, storage, platforms, and services

33. Internet Clouds (cont.)

34. Internet Clouds (cont.) • Delivered on demand to external customers over the Internet • Six common characteristics of the Internet clouds • The cloud platform offers a scalable computing paradigm built around the data centers • Cloud resources are dynamically provisioned by data centers upon user demand • The cloud system provides compute, storage, and flexible platforms for upgraded web services • Cloud computing relies heavily on the virtualization of all kinds of resources • Cloud computing defines a new paradigm

35. Internet Clouds (cont.) • For collective computing, data consumption, and delivery of information services over the Internet • Clouds stress the cost of ownership reduction in mega data centers • A distributed computing system • Tends to be owned and operated by an autonomous administrative domain for on-premises computing needs • e.g., A research laboratory or company • On-premises software is installed and runs on computers on the premises, i.e., in the building, of the person or organization using the software • These traditional systems have encountered several performance bottlenecks

36. Internet Clouds (cont.) • Constant system maintenance, poor utilization, and increasing costs associated with hardware/software upgrades • Cloud computing resolves or provides relief from these difficulties • As an on-demand computing paradigm

37. Cloud Computing versus On-Premise Computing • Conventional computing systems involve • Buying the hardware equipment • Acquiring the necessary system software • Installing the system & testing the configuration • Executing the application codes and management of resources, etc • In the case of clouds • All hardware and software resources are leased from the provider • Without much capital investment on the part of users • Only the execution phase requires service charge • One can easily save 80-95% of the cost to execute small jobs by using the cloud

38. Cloud Computing versus On-Premise Computing (cont.) • Very appealing to small businesses • Eliminates the need to invest in permanent and expensive computers or servers • Traditional computing applications are primarily executed on local hosts • Being on the premise • e.g., Desktops, desk-side workstations, notebooks, tablets, etc • Differs from cloud computing primarily in resource control and infrastructure management • Compare three basic cloud service models with the on-premise computing paradigm

39. Cloud Computing versus On-Premise Computing (cont.) • Consider five types of hardware and software resources • Storage, servers, VMs, networking, and application software

40. Cloud Computing versus On-Premise Computing (cont.) • Basic cloud service models • Infrastructure as a service (IaaS) or infrastructure cloud • Platform as a service (PaaS) or platform cloud • Software as a service (SaaS) or application cloud • For on-premise computing at local hosts • All resources must be acquired by the users except networking • Shared between users and the provider • This creates a heavy burden and operating expense on the part of the users • In cloud computing

41. Cloud Computing versus On-Premise Computing (cont.) • Users entrust their program execution to a remote cloud through the Internet • Thereby eliminating such an expense • Cloud computing differs from network computing or outsourced computing • Users leave all infrastructure management and program execution to the cloud platform • Acts as a compute/storage rental company • Users lease the computing power from the cloud providers • A cloud platform provides many VMs that execute dedicated services to users

42. Cloud Computing versus On-Premise Computing (cont.) • Both separately and simultaneously • Clouds can benefit individuals, families, communities, and organizations, concurrently • In IaaS clouds like AWS EC2 • The user only needs to worry about application software deployment • The VMs are jointly deployed by the user and provider • The vendors are responsible for providing the remaining hardware and networks • In PaaS clouds like Google App Engine • Both application codes and VMs are jointly deployed by the user and vendor

43. Cloud Computing versus On-Premise Computing (cont.) • The remaining resources are provided by the vendors • In the SaaS model like the Salesforce cloud • Everything is provided by the vendor including the application software • Cloud computing reduces a user’s infrastructure management burden • From two resources to none • As one moves from IaaS to PaaS and SaaS services • The advantages of separating the application from resources investment and management

44. Cloud Design Objectives • Six design objectives for cloud computing • Shifting computing from desktops to data centers • The shift of computer processing, storage, and software delivery away from the desktop and local servers to data centers over the Internet • Service provisioning and cloud economics • Providers supply cloud services by signing SLAs with consumers and end users • SLA - Service-Level Agreement • Services must be economically feasible with efficiency in computing, storage, power consumption, etc • Pricing models are based on a pay-as-you-go policy • Scalability in performance

45. Cloud Design Objectives (cont.) • The cloud platforms and software and infrastructure services must be able to increase in performance as the number of users increase • Data privacy protection • The concern regarding users’ data and record privacy must be addressed • To make clouds a successful and trusted service • High-quality of cloud services • The QoS of cloud computing must be standardized to make clouds interoperable among multiple providers • New standards and interfaces • Solving the data lock-in problem associated with data centers or cloud providers • Universally accepted application programming interfaces (APIs) and access protocols are needed

46. Cloud Design Objectives (cont.) • To provide high portability and flexibility of virtualized applications • Many executable application codes are much smaller than the web-scale data sets they process • Cloud computing avoids large data movement during execution • Result in less traffic on the Internet and better network utilization • The core of a cloud is the server cluster or VM cluster • The cluster nodes are used as compute nodes • A few control nodes are used to manage and monitor of the cloud activities

47. Cloud Design Objectives (cont.) • The scheduling of user jobs on a cloud • Requires assigning the work to virtual clusters created for users • The gateway nodes • Provide the access points of the service from the outside world • Also can be used for security control of the entire cloud platform • In physical clusters, users expect a static demand of resources • Clouds are designed to face fluctuating workloads • Thus dynamically demand varying amounts of resources

48. Cloud Design Objectives (cont.) • Private clouds will satisfy this demand more efficiently • Data centers and supercomputers • Some similarities and fundamental distinctions • In the case of data centers • The scaling is a fundamental requirement • The server clusters are built with thousands to even a million servers (nodes) • e.g., Microsoft has a data center in the Chicago area that has one hundred thousand 8-core servers that are housed in 50 containers • In supercomputers • A separate data farm is used • A data center uses disks on server nodes plus memory cache and databases

49. Cloud Design Objectives (cont.) • Supercomputers and data centers also differ in networking requirements • Supercomputers use custom-designed high bandwidth networks like fat trees or 3D torus networks • Networks in data centers are primarily IP-based commodity networks like 10 Gbps Ethernet, which is optimized for Internet access

50. Cloud Architectures Compared with DistributedSystems • The generic architecture of contemporary clouds • A cloud taxonomy based on service models, applications, ownership, etc • Clouds can be treated as centralized systems • Most clouds are converted from big data centers and warehouse computer systems • Can be also considered distributed systems • Many clouds are used in co-locations or mashup configurations • Co-locations - Computing services provided to many organizations by a third-party • Cloud applications are often distributed to distance clouds from where the big data is located