1 / 24

OpenStack upgrades

OpenStack upgrades. OpenStack Summit 2013 Hong Kong. Whoarewe. Sébastien Han Frédéric Lepied Mehdi Abaakouk W orking for eNovance Company blog: http://techs.enovance.com/. Worldwide offices coverage. We design, build and run clouds – anytime - anywhere. Problems. State of arts.

hart
Download Presentation

OpenStack upgrades

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. OpenStack upgrades OpenStack Summit 2013 Hong Kong

  2. Whoarewe • Sébastien Han • FrédéricLepied • Mehdi Abaakouk Working for eNovance Company blog: http://techs.enovance.com/ Worldwide offices coverage We design, build and run clouds – anytime - anywhere

  3. Problems State of arts

  4. Key principles • do not log in to the servers • do not manually install packages • do not manually edit configuration files • do not manually restart daemons Puppet, in combination with GIT is the only one managing the configuration.

  5. It’s all about architecture design We need redundancy to perform upgrades without downtime: • Active / active setup – Loadbalancer • Active / passive setup • Databases must be replicated : Galera / MongoDB

  6. Rollback Even with a good QA system, problems might rise in production thus we need a rollback mechanism.

  7. Solution Well, ours 

  8. Breaking point Puppet doesn’t install packages anymore. It only manages configurations. So operating systems are shipped with all the packages installed.

  9. eDeploy solution • Change the abstraction level • Manage updates using sub-trees • 2 kinds of sub-trees : data and programs • Data is not updated only programs Example : • Data: /var/lib/mysql, /var/log... • Program : /usr, /lib...

  10. Consequences • Prepare trees before installation or upgrade • Debootstrap/yum + chroot magic • Install in 3 phases : • Hardware detection • Hardware configuration • Tree copy • Update : • RsyncProg sub-trees. • Script to adapt Data and restore config

  11. eDeploy - Overview • Manage system provisioning by software role and hardware profile • Reproduce provisioning easily • Manage upgrades and rollbacks • Efficient in term of expressiveness and performance

  12. QA Testing systems

  13. Principles Everything is versioned: • Jenkins jobs • Puppet modules/manifests • eDeploy system images • Ansible recipes

  14. This is what we get The upgrade process becomes: • Reproducible • Automated • Testable

  15. Methodology Upgrades with (almost) no downtime

  16. Things that you must consider • Architecture design • MySQL schemas • Do backups!

  17. Configuration management and Orchestration Puppet is responsible for the configuration of a node: • Upgrade the configuration files only (no packages upgrades!) • Restart services Ansible orchestrates the process upgrade.

  18. Components dependency

  19. Base of the process

  20. Database schema But what if the database schema needs to be updated? Just upgrade the database schemas at the end of the orchestration! … Not yet (Icehouse?)

  21. But with DB schema upgrade

  22. Summary • Follow best practices • Architecture matters • Automation is mandatory • Tests, tests, tests

  23. Many thanks! Questions? Contact: sebastien.han@enovance.com frederic.lepied@enovance.com mehdi.abaakouk@enovance.com

More Related