1 / 54

Assured Cloud Computing Developments and Directions for Testbeds

Assured Cloud Computing Developments and Directions for Testbeds. Derek Dagit. Cloud Computing: NIST .

lore
Download Presentation

Assured Cloud Computing Developments and Directions for Testbeds

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Assured Cloud Computing Developments and Directions for Testbeds Derek Dagit

  2. Cloud Computing: NIST “Cloud computing is a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction. This cloud model is composed of five essential characteristics, three service models, and four deployment models.” • Characteristics: • On-demand Self-service • Broad Network Access • Resource Pooling • Rapid Elasticity • Measured Service • Service Models: • SaaS: Software as a Service • PaaS: Platform as a Service • IaaS: Infrastructure as a Service • Deployment Models: • Private Cloud • Community Cloud • Public Cloud • Hybrid Cloud

  3. Terminology: SaaS / PaaS / IaaS SaaS : The applications are provided. Use a thin client or API to interact with programs running on the provided systems. • MapQuest • Twitter PaaS : Systems are provided. Install your own applications on the provided system. • Google App Engine • VMware Cloud Foundry IaaS : Physical/Virtual machines are provided. Provision memory, CPU cores, storage, and possibly networking, and then install your own OS. • Amazon Elastic Compute Cloud • Microsoft Azure

  4. Heterogeneity and Propriety We have a mix of different APIs—most proprietary—making it difficult or infeasible to deploy and to evaluate security. What if we had a standard API that was open and freely available? What is “the Linux of Cloud Computing Platforms?”

  5. Cloud Computing : OpenStack “The OpenStack project has been created with the audacious goal of being the ubiquitous software choice for building cloud infrastructures.” —Ken Pepple, Deploying OpenStack, O’Reilly “Cloud computing is a computing model, where resources such as computing power, storage, network and software are abstracted and provided as services on the Internet in a remotely accessible fashion. Billing models for these services are generally similar to the ones adopted for public utilities. On-demand availability, ease of provisioning, dynamic and virtually infinite scalability are some of the key attributes of cloud computing.” — docs.openstack.org

  6. “OpenStack is a collection of open source software projects that enterprises/service providers can use to setup and run their cloud compute and storage infrastructure.” • — docs.openstack.org • The OpenStack Consortium has grown rapidly in the past year: • NASA • Rackspace • Citrix • Dell • AMD • OpenStack services are available via Amazon’s S3 and EC2 APIs. Applications written for Amazon Web Services will work with OpenStack. • Intel • Cisco • HP • Over 140 others

  7. OpenStack’s Core Components • Compute (“Nova”) Orchestrates large networks of Virtual Machines. Responsible for VM instance lifecycle, network management, and user access control. • Object Storage (“Swift”) Provides scalable, redundant, long-term storage for things like VM images, data archives, and multimedia. • Image Service (“Glance”) Manages VM disk images. Can be a stand-alone service. Supports private/public permissions, and can handle a variety of disk image formats.

  8. OpenStack Nova Nova was contributed by NASA from the Nebula platform. Nova allows users to create, destroy, and manage virtual machines using user-supplied images. Corresponds to Amazon’s EC2. Users can use OpenStack API or Amazon’s EC2 API. Uses Python and Web Server Gateway Interface (WSGI).

  9. OpenStack Nova: Architecture

  10. OpenStack Nova: nova-api A daemon that is the workhorse of Nova. • Handles API requests. • Manages most orchestration. • Enforces some policies. If it can, it will handle the request on its own with help from the database. Otherwise, it will delegate to the other nova daemons using the message queue as well as the database.

  11. OpenStack Nova: nova-compute Worker that does the actual work of starting and stopping virtual machine instances. Takes its orders from the message queue, and executes the appropriate VM API calls to accomplish the task. Commonly uses “libvirt” (RedHat), but can use Xen, vSphere (VMware), or Windows Management Interface.

  12. OpenStack Nova: nova-network Worker that does the actual work of configuring the network. Network is specified as one of three types: • Flat • FlatDHCP • VLAN

  13. OpenStack Nova: nova-scheduler Worker that simply takes a VM instance request from the queue and chooses a host. • Simple:Least Load • Chance:Random available host (default) • Zone:Random within availability zone Large deployments will want to implement more comprehensive and sophisticated algorithms.

  14. OpenStack Nova: nova-volume Worker that manages volumes for persistent storage. Compatible with a several vehicles, including AoE, iSCSI, and Sheepdog.

  15. OpenStack Nova: Message Queue The queue is used for message passing among daemons. Currently implemented by RabbitMQ (VMware, OSS, Erlang). Could swap out RabbitMQ with any system that supports Advanced Message Queuing Protocol (AMQP). Queue Types: • Fanout • Host • Topics

  16. OpenStack Nova: Database Stores the state of running cloud infrastructure. Supports the usual suspects: sqlite3, MySQL, PostgreSQL, etc.

  17. OpenStack Swift Swift was contributed by Rackspace, and powers their Cloud Files product. Swift is not a distributed file system, but a distributed object store. Access files via the API(OpenStack/S3), not via NFS or the like.

  18. OpenStack Swift: Architecture

  19. OpenStack Swift: Presentation • swift-proxy • Handles incoming requests and delegates to the appropriate process. • Uses OpenStack API out-of-the-box or Amazon S3 with middleware. • Capable of authentication and authorization.

  20. OpenStack Swift: Resource • swift-accountoperates an sqlite3 database for accounts • swift-containeroperates an sqlite3 database for object “containers” • swift-objectmaps objects themselves onto the node’s storage All three manage replication and self-consistency.

  21. OpenStack Glance Virtual Machine Disk Image Service Can be used stand-alone, but integrates well as service between Nova and Swift. Supports a range of virtual disk image formats and container formats.(VMDK, ISO, AMI, etc.)

  22. OpenStack Glance: Architecture

  23. OpenStack Glance: glance-api Similar to nova-api, delegates requests as appropriate. Communicates with glance-registry and with the Image Store (Swift, Amazon S3, a file system, read-only HTTP, etc.) Basic REST API, JSON

  24. OpenStack Glance: glance-registry Interacts with glance-api and manages simple database holding image metadata. Ships with a reference implementation using sqlite3.(If scaling, replace this reference implementation.)

  25. Identity & Dashboard Components OpenStack will integrate two additional components in the next release—planned for Q2 of 2012—named “Essex.” • Identity (“Keystone”)Provides integrated authentication among OpenStack components, and can leverage external authentication systems. • Dashboard (“Horizon”)Provides a browser-based “control panel” application for administrators and users.

  26. Cloud Computing Testbed The CCT is designed for both applications and systems research in cloud computing. Testbed resources support research both internal to UIUC and external: Networks and Network Instrumentation Operating Systems Databases Storage Virtual Machines Distributed Systems Data-Mining Web Search Multimedia

  27. CCT Inception The CCT was created from resources provided jointly by the National Science Foundation (NSF), Yahoo!, Intel, Hewlett-Packard, and the University of Illinois at Urbana-Champaign. The CCT is housed within and supported primarily by staff and faculty from the Department of Computer Science at the University of Illinois. The CCT is one of the sites participating in the world's first networked cloud testbed, called Open Cirrus.

  28. CCT: Compute Nodes • Compute Nodes (x128): • HP DL160 • 2 quad-core CPUs • 16 GB RAM • 2 TB storage

  29. CCT: Head Nodes • Head Nodes (x2): • HP DL380 • 2 quad-core CPUs • 8 GB RAM • 292 GB storage

  30. CCT: Storage Nodes • Storage Nodes (x4): • HP DL380 • 2 quad-core CPUs • 16 GB RAM • 72 TB storage

  31. CCT: Networking • Network: • Gigabit to external network • Gigabit among compute nodes, head nodes. • 10 Gigabit among storage nodes • Flat topology • Racks have 4 HP ProCurve 2650 switches, connecting to 2 ProCurve 5412zl switches for the testbed. • Compute nodes have 2 NICs, connected to each 5412zl switch

  32. CCT Configuration • Partitioned into application and research clusters • 64 Nodes running CentOS 5.7 providing Hadoop • Remaining 64 nodes allocated for systems research, normally CentOS 5.7, but provisioned to suit. • System imaging handled by Cobbler, and configuration is done through cfengine. • Network topology is flat, and is managed by UIUC Campus IT. • Users have home and project space allocated on storage nodes and mounted via NFS.

  33. CCT OpenStack Plans Integrate with existing IT configuration tools to deploy OpenStack on the CCT. • Create cfengine profiles for OpenStack atop CentOS. • Storage: • Consolidate NFS /home and /project shares to one storage node • Use remaining storage nodes to provide attached volumes • Configure 2nd NICs on each compute node to be volume-only • Authentication: configure Keystone for Active Directory • User management: Use either CLI tools or Dashboard • Enable Glance for user-provided images.

  34. CCT OpenStack Plans: Extending • Configure multiple zones to enable a variety of hypervisors of user’s choice • Systems and Applications research will involve customizations and instrumentation of different VMMs • Federated authentication • Enable use of the CCT to those outside UIUC with existing credentials • Facilitate “cloudbursting” to/from other cloud infrastructures • Sophisticated Network Configurations • Support network and security research involving experiments in isolation • Experimentation with SDN architectures for data centers

  35. Federated Authentication OpenStack has the ability to plug into existing, external authentication frameworks. Federated authentication works by handing the job of authentication to the system controlled by the user’s organization. Shibboleth is one such infrastructure, and UIUC is a member institution.

  36. Shibboleth “The Shibboleth System is a standards based, open source software package for web single sign-on across or within organizational boundaries. It allows sites to make informed authorization decisions for individual access of protected online resources in a privacy-preserving manner.” — shibboleth.internet2.edu • A project of the Internet2 Middleware Initiative • Uses Security Assertion Markup Language (SAML) • XML-based open standard for authentication between security domains. • Enables federated single-sign-on

  37. Challenges to Network Innovation Innovation in the network is lagging. Why? • Proprietary Network Hardware • Incompatibilities between vendors • Network switch hardware and software in a black box • Too dangerous to experiment on production networks • Too expensive to test deployments • Hardware vendors implement features only for revenue

  38. Software Defined Networks It is helpful to abstract servers using virtual machines. We can apply a similar abstraction to networks themselves. We would like to be able to programmatically configure and define networks.

  39. Benefits of SDN Abstraction • Unity in conceptualizing pieces of the network. • One comprehensive view with which to monitor and control the network • Separate job of configuring the network from the job of defining and enforcing policy.

  40. SDN and the Cloud • Hide physical complexity • Manage, on-demand virtual networks • Sustain multiple concurrent virtual networks • Provide resilience and flexibility with physical modifications to the network • Automated scaling

  41. Barriers to SDN Network hardware companies do not want to expose access to hardware. We must rely on what features the vendors provide and the way they are provided. There is a lack of consistency in what vendors provide and in the definitions of features.

  42. OpenFlow OpenFlow is an API to the forwarding plane of the network hardware. It separates the control function from the hardware into software controller servers. Any switch implementing the OpenFlow API can be programmatically operated by a separate server. Network hardware supporting OpenFlowships with OpenFlow firmware.

  43. SDN with OpenFlow OpenFlow-enabled switches and routers are configured by OpenFlow controllers. OpenFlow controllers can be simple Python services connected using secure protocols. • Administrators can capture a complete topology of the network from a single controller. • No matter what vendor hardware or software, if the unit supports OpenFlow, the hardware can be abstracted. • Users could be granted permission to define circuits without the aid of IT support. • Compromised nodes can be detected and isolated.

  44. Examples of OpenFlow Use Cases • NEC ProgrammableFlow • Virtualized Physical Network • Automatic Optimization of Network Resources • Alternate route finding for end-to-end reliability • Juniper Junos SDK • Added OpenFlow application to SDK • Customers can implement their own features • Deutsch Telekom • Energy-aware server provisioning and consolidation • Verizon • Deliver bandwidth-on-demand between data centers • Shape Traffic for long-lived flows, avoiding full algorithm

  45. Open Networking Foundation “Founded in 2011 by Deutsche Telekom, Facebook, Google, Microsoft, Verizon, and Yahoo!, the Open Networking Foundation (ONF) is a nonprofit organization whose goal is to rethink networking and quickly and collaboratively bring to market standards and solutions. ONF will accelerate the delivery and use of Software-Defined Networking (SDN) standards and foster a vibrant market of products, services, applications, customers, and users.” — opennetworkingfoundation.org

  46. ONF Vision With the advent of cloud computing, ONF believes the line will continue to blur between the computer and the network. Network innovation must keep pace with demands. The Open Network Foundation aims to create “the most relevant SDN standards.” (Mission Assurance will depend on both the agility and security of the network.)

  47. ONF Members • Board includes: • Deutsche Telekom • Facebook • Google • Microsoft • NTT Communications • Verizon • Yahoo! • More than 50 members as of early 2012

  48. Research: OpenStack + OpenFlow The nova-network daemon supports Flat, FlatDHCP, and VLAN networking modes. Extending nova-network using OpenFlow could lead to much more comprehensive network configurations: • Fully virtualized networks, tailored to VM instances • Control through network isolation and flow-based routing

  49. Research: Virtualized Hadoop • Preserve rack-awareness for Hadoop in a virtual machine environment • Dynamic load balancing • Virtual network consolidation and optimization

  50. Research: Current ACC Projects • Dynamic Policy Monitoring With Inference in Clouds • Novel representation of abstract base policies with formal ontologies. • An Inference Engine uses the base policies and the dynamic information of the system configuration to deduce actions and refine the policies. • Open-flow enabled network switches are utilized to extract network configuration and allow network-level enforcement of policies. • Attack Incident Data • Incorporate new defense strategies into the design of testbeds based on attack data research. • Analyze costs vs. benefits tradeoffs of new detection mechanisms.

More Related