1 / 23

Common Execution Environment

Common Execution Environment. Kate Keahey OOI Cyberinfrastructure Life Cycle Objectives Milestone Review, Release 1 San Diego, CA February 23-25, 2010. Overview. Use Cases Architecture Overview CEI Components in R1 CEI Elements Elastic Provisioning Unit Infrastructure as a Service

adara-pugh
Download Presentation

Common Execution Environment

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Common Execution Environment Kate Keahey OOI Cyberinfrastructure Life Cycle Objectives Milestone Review, Release 1 San Diego, CA February 23-25, 2010

  2. Overview • Use Cases • Architecture Overview • CEI Components in R1 • CEI Elements • Elastic Provisioning Unit • Infrastructure as a Service • Deployable Type management OOI CI LCO Review, Feb 2010

  3. Use Case Overview • Highly Available Services • Rapidly provision resources • - Scale to demand OOI CI LCO Review, Feb 2010

  4. Architecture Overview VM (Deployable Unit) VM (Deployable Unit) Application Software (Deployable Type) VM (Deployable Unit) EPU Worker (Operational Unit) EPU Worker (Operational Unit) EPU HA Service (OOI Application) EPU Worker (Operational Unit) Availability and Scaling Strategy OOI CI LCO Review, Feb 2010

  5. Architecture Overview OOI CI LCO Review, Feb 2010

  6. Taskable Resources Deployable Type e.g., a description of an environment in rBuilder Deployable Unit e.g., a VM image Start/Stop Monitorable Controllable Taskable Resources HA Service Operational Unit Process e.g., a core or application service Simple or complex deployed appliance, e.g., a deployed VM instance, a virtual cluster, platforms for workflow execution e.g., an executing program or a long-running service OOI CI LCO Review, Feb 2010

  7. Architecture Overview OOI CI LCO Review, Feb 2010

  8. Architecture Overview OOI CI LCO Review, Feb 2010

  9. CEI Service Interactions Registry Planner Fault Monitoring and Compensation Provisioner Controller (“Executive”) Resource (Taskable) OOI CI LCO Review, Feb 2010 2970-00001 OV2 CEI

  10. CEI Service Interaction Deployable Type Registry Monitoring Information Planner VM info Other Info Exchange Proc Info Provisioner Controller IaaS Adapter IaaS Context Broker OOI CI LCO Review, Feb 2010

  11. Elastic Processing Unit • Elastic Processing Unit (EPU): High-Availability Services with policy based scaling to demand EPU Worker N1 EPU Worker N2 EPU Worker NX … EPU Controller [N, Deployable Unit] Register HA service Msg to N OOI CI LCO Review, Feb 2010

  12. Elastic Processing Unit Deployable Type Registry Monitoring Information Planner Will this whole thing work (even in a small way)? VM info Other Info Exchange Proc Info Provisioner Controller IaaS Adapter IaaS Context Broker OOI CI LCO Review, Feb 2010

  13. EPU: Prototype 1 • EPU design and prototype: • Policy-configurable planner/scheduler • Scaling policy: size of work queue • All components integrated with Magnet/AMQP OOI CI LCO Review, Feb 2010

  14. EPU: Prototype 2 • Updates technology choices • Uses Redis to distribute work to VM instances and as Information Aggregator • New policy • load • Demonstration later OOI CI LCO Review, Feb 2010

  15. EPU: Technology Choices • IaaS Adapter: still evaluating • Deltacloud, jcloud, libcloud, dasein, fog • Standards: OCCI-WG • Nimbus Context Broker • Redis for Information storage OOI CI LCO Review, Feb 2010

  16. EPU: Elaboration Iterations • Atomic HA for EPU controller • Detailed design on interaction with COI • Scalability, scalability, scalability • Finalize technology choices • Prototype end-to-end EPU • Stress testing and harness • Investigate storage/compute/network • Refine implementation OOI CI LCO Review, Feb 2010

  17. Infrastructure-as-a-Service Deployable Type Registry Monitoring Information Planner VM info Other Info Exchange Proc Info Can we adapt existing resources? Provisioner Will it work with COI? Controller Where are we going to get IaaS resources? IaaS Adapter IaaS Context Broker OOI CI LCO Review, Feb 2010

  18. Infrastructure-as-a-Service • Targeted deployment platforms • Commercial (AWS, rackspace) • Scientific (FutureGrid, Magellan, …) • IaaS for science • The Nimbus Workspace Service • Evaluate Ganeti/SGE/Condor • Nimbus • Make implementation technology-independent • Make implementation WSRF-independent • Provide user management functions OOI CI LCO Review, Feb 2010

  19. Deployable Types Deployable Type Registry Monitoring Information Planner complexity? VM info Other Info Exchange Proc Info Provisioner Controller IaaS Adapter IaaS Will it work with COI? scalability? Context Broker OOI CI LCO Review, Feb 2010

  20. Deployable Types • Complexity • Document requirements and use cases • Web App, Hadoop, Job Queue, NFS, Torque • Design overall approach • Preliminary roundup and evaluation of relevant technologies • Puppet, Chef, CohesiveFT, rBuilder • Working choices: rBuilder and Chef • Prototype integration of Chef with the Nimbus Context Broker OOI CI LCO Review, Feb 2010

  21. Deployable Types • Protocol-independence • Make the Context Broker WSRF-independent • REST interfaces • Scalability and Sustainability • Prototype scalability harness • Scalability tests • Contextualizing up to 500 nodes • Challenges in harness design and efficiency OOI CI LCO Review, Feb 2010

  22. DT: Elaboration Iterations • Finalize technology choices for management of Deployable Types/Units • Deployable Type Registry • More comprehensive harness and more contextualization scalability tests • Understanding performance OOI CI LCO Review, Feb 2010

  23. Thanks ! OOI CI LCO Review, Feb 2010

More Related