1 / 19

Agile Infrastructure Project Overview : Status and How to use it

Agile Infrastructure Project Overview : Status and How to use it. V ítor Gouveia, vitor.gouveia@cern.ch IT-PES-PS. Agenda. Agile Infrastructure (AI) Overview Current Status of the AI project Schedule of the AI project How to use the AI : Creation and configuration of VMs

darcie
Download Presentation

Agile Infrastructure Project Overview : Status and How to use it

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Agile Infrastructure Project Overview:Status and How to use it Vítor Gouveia, vitor.gouveia@cern.ch IT-PES-PS

  2. Agenda • Agile Infrastructure (AI) Overview • Current Status of the AI project • Schedule of the AI project • How to use the AI : • Creation and configuration of VMs • Current support • Questions ? 2/19

  3. Agile Infrastructure (AI) Overview • CERN IT is changing strategy for machine provision and configuration. • Cloud Infrastructure as a Service (IaaS) • virtual delivery of computing resources in the form of hardware, networking, and storage services. • It may also include the delivery of operating systems and virtualization technology to manage the resources. • The goal of the Agile Infrastructure Project is to: • Provide and efficient and flexible infrastructure service which enables new applications and supports existing ones while exploiting increased capacity and multi-site facilities within the current man power resources 3/19

  4. Agile Infrastructure (AI) Overview • User documentation being written at: • http://information-technology.web.cern.ch/book/cern-puppet-development-user-guide • Internal documentation about the AI project: • https://twiki.cern.ch/twiki/bin/view/AgileInfrastructure/WebHome • Agile Infrastructure Components: • Configuration: • Puppet, Foreman, Hiera • Provision: • OpenStack • Other Services • GIT, Koji 4/19

  5. Agile Infrastructure (AI) Overview • Agile Infrastructure Configuration Components: • Puppet • Open source configuration management tool like Quattor • Declarative language to describe system configuration • Puppet programs are called “manifests” • Conceptually like a cdb template • The core of the Puppet language is the resource declaration file { 'testfile': path => '/tmp/testfile', ensure => present, mode => 0640, content => "I'm a test file.", } A singleton collections of resources in puppet is called class • Tutorial about puppet: http://docs.puppetlabs.com/learning/ • CERN puppet user guide (work in progress): http://information-technology.web.cern.ch/book/cern-puppet-development-user-guide 5/19

  6. Agile Infrastructure Configuration Components (Cont.): • Puppet ( Cont.) • Facter • Puppet tools which examines our system and normalizes all the information about it • creates a set of variables and passes them off to puppet • Source of information about the machines (CPUs, RAM, network etc.) • Functionally similar to CDB hardware/network profiles, but provides information dynamically from various data sources • Hiera • Data store used by puppet • hierarchical-based lookup system 6/19

  7. Agile Infrastructure (AI) Overview • Agile Infrastructure Configuration Components (Cont.): • Foreman • Current web front-end for Puppet and OpenStack • Groups hosts (node) into hostgroups of similar configuration • Hostgroup • Essentially the same a cdb cluster • In a hostgroup several features are aggregated together • Ex: Cluster of a LAMP server ( should include the puppet classes responsible for apache, mysqland php) • Can be hierarchical Ex: 1st level ~= cdb cluster, 2nd/3rd level ~= cdb customization • Real time information about hosts status (node) • Rebuild machines from scratch • High-level machine configuration (~= CDB profile) • Possible to configure a hostgroup or just a single host https://judy.cern.ch/users/login 7/19

  8. Agile Infrastructure Provision Component: • OpenStack (Hamster) • Software in use for the CERN private cloud. • Principal organizational structure • Tenants • Resource container for a project/VO/… Tenant users can freely create volumes, VMs etc. within a certain quota of total Disk, Cores, RAM, etc… https://hamster.cern.ch/dashboard 8/19

  9. Agile Infrastructure (AI) Overview • Other Services • GIT • Distributed version control system • GIT in the AI project: • GIT branches map to Foreman environments ( ~= cdb stages) • Inside a branch it is possible to find manifests and modules Modules: • Collection of configuration components Ex: apache, afs, ssh, sssd • Similar to cdb services and cdb components • Can use other modules • Stored in repository root in modules/ 9/19

  10. Other Services (Cont.) • GIT ( Cont.) Manifests: • They are specific for each hostgroup • Should not be reusable • May reference several modules • Conceptually similar to cdb cluster/customization templates • Describe which modules/puppet classes/puppet resources should be applied to a set of machines (hostgroup) Ex: lxplus, lxBatch • Stored in repository root in manifest/ https://gitgw.cern.ch/gitweb/ 10/19

  11. Other Services (Cont.) • KOJI • Creates RPMs and publish to YUM repositories • Instead of the swrepsrv 11/19

  12. Current Status of the AI project • Pre-Production Servicerunning: • OpenStack Essex, Foreman 1.0 and puppet 2.7 • Several IT services running on the AI • EX: SLC6 Batch machines • Deploy puppet configured VMs on OpenStack • Configuration and Management through web portal (Foreman) • VM creation still requires using command-line scripts for the time being 12/19

  13. Schedule of the AI project • Deploy into production: • End of January 2013 • Folsom (Openstack) • suitable for implementing the 'Cattle‘ • machines that may die without impact on production Ex: batch machines • Not suitable yet for ‘pet’ machines • Even less if High-availability is necessary • Ex: VOBOX • Eventually Openstack will be integrated in Foreman (together with Puppet) • It will be possible to create and manage VMs through Foreman • Only one website! • Performance and stability will improve over time 13/19

  14. How to use the AI • From now on you can start using the AI: • Attention: Everything is in pre-production • Things will change!! • What are you going to see? • How to create a VM in the AI • How to configure it More info: https://twiki.cern.ch/twiki/bin/view/AgileInfrastructure/VOCGettingStarted 14/19

  15. How to use the AI • Mailings lists and egroups to join: • lxvoadm-Authorized-Users • Gives Access to [aiadm02.cern.ch | aiadm03.cern.ch] server • Aiadm is for AI what lxvoadm is for Quattor • puppet-users • CERN puppet discussion • ai-vobox-migration-discussion: • Gives you access to create VMs in the AI project • Best way to provide feedback 15/19

  16. 16/19

  17. Current support • This is not a production service so official support lines don't exist yet. • All Feedback about the VOBox migration to AIshould be reported through the mailing list ai-vobox-migration-discussion 17/19

  18. Current support • Upcoming meetings… • Topics to be covered • GIT, branches • Partitions + resize • Monitoring + custom sensors • Koji + RPMs • Accounting • etc… • Starting from January every two weeks… • Thursdays afternoon 18/19

  19. Questions??? 19/19

More Related