350 likes | 461 Views
The CloudStack development story and future vision. Sheng Liang, CTO Cloud Platforms, Citrix Systems August 29, 2012. AWS is setting the standard… as measured by capacity…. Total Number of Objects Stored in Amazon S3.
E N D
The CloudStack development story and future vision Sheng Liang, CTO Cloud Platforms, Citrix Systems August 29, 2012
AWS is setting the standard…as measured by capacity… Total Number of Objects Stored in Amazon S3 • Every day through 2011, AWS added the same amount of server processing capacity, on average, that it took to run the Amazon online retailing operation in 2000, when it was a $2.76bn company. • Prickett-Morgan. “AWS Cloud Double Fluffs in 2011.” The Register, 6 Jan 2012. Peak Requests: 500,000+ per second Source: UBS
…data center footprint and geographic distribution… • …the company said that with the opening of its AWS data center in São Paulo, Brazil in mid-December, the company has doubled its AWS data-center footprint. • Prickett-Morgan. “AWS Cloud Double Fluffs in 2011.” The Register, 6 Jan 2012. AWS Regions Amazon Edge Locations (CloudFront & Route 53)
…and, most importantly, revenue… Amazon Web Services Revenue Model $M • It has been estimated that AWS could be a $1 billion business for the online retailer come next year…could hit $2.5B in 2014. • Hickey, Andrew. “Amazon Q3 Cloud Revenue Skyrockets” CRN. 26 Oct 2011. Source: UBS
How did Amazon build its Cloud? Amazon eCommerce Platform AWS API (EC2, S3, …) Amazon Proprietary Orchestration Software Open Source Xen Hypervisor Networking Servers Storage
How can we build a cloud using CloudStack? User Portal Amazon eCommerce Platform AWS API (EC2, S3, …) Cloud API “Cloud OS or Data Center OS” Amazon Proprietary Orchestration Software Apache CloudStack XenServer Open Source Xen Hypervisor VMware KVM OVM Hyper-V Bare-Metal Networking Servers Storage
The Virtual Datacenter OS allows businesses to efficiently pool all types of hardware resources - servers, storage and network – into an aggregated on-premise cloud VMware press release Sept 15, 2008
Eucalyptus is the only cloud architecture to support the same application programming interfaces (APIs) as public clouds, and today Eucalyptus is fully compatible with the Amazon AWS public cloud infrastructure. Eucalyptus Systems Press Release April 2009
Nimbula 3Tera Joyent Zimory Eucalyptus VMware OnApp OpenNebula Enomaly Cassatt Yunteq Abiquo
4.XRefactor 2.0 Refactor AWS Compatibility 3.0 Quality Improvements Prototype 1.0 GA 2.2 Refactor 2009 2010 2012 2008 2011 April 2012: Apache CloudStack July 2011: Citrix Acquires Cloud.com Nov 2009: CloudStack 1.0 GA May 2010: Cloud.com Launch & CloudStack 2.0 GA Sept 2008: VMOps Founded
4.XRefactor 2.0 Refactor AWS Compatibility 3.0 Quality Improvements Prototype 1.0 GA 2.2 Refactor Initial target: hosting companies like Rackspace and Savvis 3 engineers built a fully functional prototype in 5 months Use the demo to sell to early customers (ReliaCloud, CloudCentral, 1800hosting.com, Go Daddy, etc.) 2009 2010 2012 2008 2011
4.XRefactor 2.0 Refactor AWS Compatibility 3.0 Quality Improvements Prototype 1.0 GA 2.2 Refactor Took 6 more months to make 1.0 software production ready Deployed on 5 production customers 2009 2010 2012 2008 2011
4.XRefactor 2.0 Refactor AWS Compatibility 3.0 Quality Improvements Prototype 1.0 GA 2.2 Refactor Product first, architecture second From web hosting to enterprise workload Multi-hypervisor, SAN, and VLAN support Learn needs of enterprise workload from: Tata Communications, Korea Telecom, Macquarie Telecom Competition: vCloud Express 2009 2010 2012 2008 2011
4.XRefactor 2.0 Refactor AWS Compatibility 3.0 Quality Improvements Prototype 1.0 GA 2.2 Refactor Private cloud demand picked up Zynga wanted private cloud Support Amazon-style flat networking and security groups Competition: Eucalyptus 2009 2010 2012 2008 2011
4.XRefactor 2.0 Refactor AWS Compatibility 3.0 Quality Improvements Prototype 1.0 GA 2.2 Refactor Second major refactoring of CloudStack code Network-as-a-service combing both Amazon and traditional style networking More flexible orchestration engine 2009 2010 2012 2008 2011
4.XRefactor 2.0 Refactor AWS Compatibility 3.0 Quality Improvements Prototype 1.0 GA 2.2 Refactor Citrix acquisition Rapid growth of CloudStack user base Quality is more important than new features 2009 2010 2012 2008 2011
4.XRefactor 2.0 Refactor AWS Compatibility 3.0 Quality Improvements Prototype 1.0 GA 2.2 Refactor Third major refactoring of CloudStack code Apache contribution drive rapid growth of CloudStack developer base Apache license compliance Services framework Hadoop integration 2009 2010 2012 2008 2011
Optimize Hadoop on cloud infrastructure • Use HDFS as object store +
Server failure comes from: • 70% - hard disk • 6% - RAID controller • 5% - memory • 18% - other factors • Application can still fail for other reasons: • Network failure • Software bugs • Human admin error 8% Annual Failure Rate of servers KashiVenkateshVishwanath and NachiappanNagappan, Characterizing Cloud Computing Hardware Reliability, SoCC’10
Internet Core Routers … Access Routers Aggregation Switches Load Balancers … Top of Rack Switches Servers
40% • Bugs in failover mechanism • Incorrect configuration • Protocol issues such as TCP back-off, timeouts, and spanning tree reconfiguration Effectiveness of network redundancy in reducing failures PhillipaGill, NavenduJain & NachiappanNagappan, Understanding Network Failures in Data Centers: Measurement, Analysis and Implications, SIGCOMM 2011
Traditional-Style Amazon-Style Cloud workloads Reliable hardware, backup entire cloud, and restore for users when failure happens Tell users to expect failure. Users to build apps that can withstand infrastructure failure Link Aggregation VM Backup/Snapshots Storage Multi-pathing Ephemeral Resources VM HA, Fault Tolerance Chaos Monkey VM Live Migration Multi-site Redundancy
Designing a zone for a traditional workload vSphere or XenServer Enterprise Traditional-Style Availability Zone Hypervisor vCenter/XenCenter SAN Storage Enterprise Networking (e.g., VLAN) L2 VLANs Networking Hypervisor Cluster Hypervisor Cluster Hypervisor Cluster Load Balancing VPN Network Services Enterprise Storage (e.g., SAN) Multi-tier VLANs OVF Multi-tier Apps
Designing a zone for an Amazon-style workload Amazon-Style Availability Zone XenServer Advanced Software Defined Networks (e.g., Security Groups, EIP, ELB,...) Hypervisor Local EBS Server Racks Server Racks Server Racks Server Racks Storage Object store Server Racks Server Racks Server Racks Server Racks L3 SDN based L2 Networking Elastic IP Security Groups Server Racks Server Racks Server Racks Server Racks ELB GSLB Network Services 3rd Party Tools (e.g., RightScale, enStratus) Elastic Block Storage CloudFormation Multi-tier Apps
CloudStack can Support Both Styles Apache CloudStack Traditional Style Availability Zone Traditional Style Availability Zone AWS-style Availability Zone AWS-style Availability Zone AWS-style Availability Zone
146 Companies 238 Developers Global User Groups Service Providers 100’s of Production Clouds 32,000 Community Members Enterprises Universities
Apache CloudStack community projects • Smart Storage • Hadoop + S3 API for object store • NetApp (FlexPod, object store) • Basho RIAK CS • Caringo object store • Cloudian S3 • PaaS • CloudFoundry implementation through IronFoundry and Stackato teams • Engine Yard • Cumulogic • GigaSpaces • SDN • Nicira • Midokura • Big Switch Networks • Stratosphere • Backup/DR • Sungard • Networking • Cisco (VXLAN, Nexus) • Brocade (ADX)
“The Apache Way” • Collaborative software development • Commercial-friendly standard license • Consistently high quality software • Respectful, honest, technical-based interaction • Faithful implementation of standards • Security as a mandatory feature
Innovative Cloud Applications and Services … Networking Servers Storage Innovative Cloud Infrastructure
More information: http://cloudstack.org http://cloudstack.jp