1 / 33

Meetup Fairfax Lean Agile Engineering Practices CI/CD Continuous Integration Continuous Delivery

Meetup Fairfax Lean Agile Engineering Practices CI/CD Continuous Integration Continuous Delivery. Ramakrishnan Srinivasan, AOL ( devfactor@gmail.com ) Thursday, September 18, 2014 *Everything I say reflects my own opinion only. CI/CD: We will cover. Why CI/CD?

jael-little
Download Presentation

Meetup Fairfax Lean Agile Engineering Practices CI/CD Continuous Integration Continuous Delivery

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. MeetupFairfax Lean Agile Engineering PracticesCI/CDContinuous IntegrationContinuous Delivery Ramakrishnan Srinivasan, AOL (devfactor@gmail.com) Thursday, September 18, 2014 *Everything I say reflects my own opinion only.

  2. CI/CD: We will cover • Why CI/CD? • The development process & its goals • Industry & business demands • Discuss DevOps, CI, CD and available software • The CI/CD setup • Opscode Chef terminology & basics • Chef Project structure & show a few screens • References • Q & A

  3. We will speak to • Operations folks • Quality Assurance analysts • Business managers & analysts • Software engineers

  4. DevOps • Its about increasing efficiencies in the development/QA/deployment process • Its about increasing agility and output of the total system • Emphasizes collaboration of Dev, QA and Ops • QA personnel need to learn some software development in order to automate testing • Operations folks will find their tasks to be much easier • There is an increasing demand for these skills

  5. Typical workflow in s/w shop • Developer works on a story to satisfy specific requirements • Code is checked in to the repository • Developer initiates a build • Upon a successful build, QA team is notified • QA installs the build in their environment • QA tests the delivered functionality • Bugs are reported & resolved • Once cleared, its installed in production

  6. The problems & motivation • Code breaks. Defects are inevitable. • No code change, but an external system has changed - breaking our functionality • We forget to run our unit tests before checking in code • We want to know about issues sooner rather than later • We want to be faster to market, deliver rapidly and often • We want to automate deployment in a configurable way

  7. Business demand: Efficiency • We are constantly striving to improve productivity and efficiency • Developers • Make functions generic • Write unit tests • Code with a view to make debugging easy • Only checkin code when • you are actually done, or • you are sure of not breaking existing functionality

  8. Efficiency .. 2 • Testing: Automate where possible • Goals • Reduce costs • Decrease number of defects • Increase productivity • Spend more time building the product and creating value for the customer • Spend less time on functions/process that can be automated • Do more with less

  9. CI: A solution to improve efficiency • Write unit tests with sufficient coverage for the important parts • Trigger a code build on every checkin or maybe every 5 minutes • Run unit tests as part of the build • Write integration tests for both UI and backend • Run integration tests frequently • Tests must be high quality & cover major functions • Detect issues early, and as they happen

  10. CD: Continuous Deployment • Code and tests must be deployed before it can run • CD is really part of the CI solution • Adap.tv does several deployments to production on a daily basis

  11. CI Practices – Martin Fowler • Maintain a Single Source Repository • Automate the Build • Make Your Build Self-Testing • Everyone Commits To the Mainline Every Day • Every Commit Should Build the Mainline on an Integration Machine • Keep the Build Fast • Test in a Clone of the Production Environment • Make it Easy for Anyone to Get the Latest Executable • Everyone can see what's happening • Automate Deployment • Reference: http://martinfowler.com/articles/continuousIntegration.html

  12. CI/CD: Benefits • Technical • Continuous delivery • Standardized environments • Less complex problems since they are identified quickly • Faster resolution • Business • Faster delivery of features • More stable, controlled, and predictable operating environments • Focus on adding business value rather than fixing issues • Increased process effectiveness • Increased agility

  13. CI/CD: Summary • Automate testing as part of the build • Automate deployment to staging, test or production • Automate rollback • Deploy continuously to production • Incremental new features initially enabled for small set of users, then turned on for a wider audience

  14. Software alternatives for CI/CD • Plenty of alternatives available • Opscode Chef • Puppet • Ansible • Salt • Cobbler • Here are a few: http://alternativeto.net/software/chef/

  15. Software used in my project • need to install Ruby • Berkshelf – used to manage dependencies in Chef • Chef Server on the server • chef-client • Jenkins • SonarQube • git

  16. Servers used in a CI/CD setup • Build server • Deployment server • Chef Server for provisioning & deployment • Chef Client on each participating node • Jenkins for scheduling and coordinating automated build pipelines • SonarQube: continuously analyze and measure code quality • A single server can be used for many of the above • Of course, servers VMs can be started up on demand in the cloud

  17. Chef topology

  18. Chef basics/terminology - I • Environment: settings, properties maintained in each environment like Dev, QA and Production • Bootstrap: first time setup • Server: hub for configuration data • Client/Node: a computer/laptop/server that can run chef-client • Workstation: its configured to use the knife command • knife command: Used to activate Chef functionality

  19. Chef components

  20. Chef basics/terminology - II • Based on the Ruby language • Attributes: properties, key-value pairs • Resources: commands, e.g. to run a shell script • Templates • Recipes • Cookbooks • Databags: JSON data accessible in a Chef environment that can be encrypted. • Libraries • Overview: http://docs.getchef.com/chef_overview.html

  21. Chef Attributes • key-value pairs • default values can be set • Some values can be overridden in specific environments • e.g. Connecting to Salesforce • Dev, QA and Production instance URLs can be overridden in respective environments • Reference: https://docs.getchef.com/chef_overview_attributes.html

  22. Attribute locations

  23. Example project • Java Web application • Apache Tomcat • MySQL or Oracle • Dev, QA and Production environments • CI/CD pipeline

  24. CI/CD Pipeline • build the project • execute unit tests • for each build, create new linux RPM package • deploy to CI server • run smoke tests • automated backend integration tests • automated UI testing jobs (Selenium and CasperJS for automated UI testing) • deploy to production if delivery criteria are met

  25. Chef Resources • Template • Package • Directory • bash, csh • Service • Cron • Deploy • Reference: http://docs.getchef.com/chef/resources.html

  26. Templates • these are static text files • with variables embedded in them • Node attributes can be referenced • to dynamically generate environment-specific files like scripts and config files

  27. Recipes & cookbooks • basic Ruby syntax • the programming part of Chef • has standard programming constructs like loops, conditionals, functions, etc. • a Recipe resides in a Cookbook - collection of recipes that belong together • uses resources (like bash and service) to generate and install software • download new package versions only when available • Just enough Ruby for Chef • Reference: https://docs.getchef.com/essentials_cookbook_recipes.html

  28. Data bags • global variables Chef-wide • stored as JSON data • Can be encrypted • Can be accessed by recipes • Reference: http://docs.getchef.com/essentials_data_bags.html

  29. Libraries • Contain Ruby functions • Provides the ability to execute arbitrary Ruby code • Use to define custom resources, connect to a database, etc. • Use to define template helper modules • Reference: https://docs.getchef.com/essentials_cookbook_libraries.html

  30. Chef Project structure • Project • attributes • environments • Ruby classes • Contain Ruby hashes (KV pairs) • like a JSON dictionary or associative array • Different format than attributes • templates (static text, config files) • data_bags (encryptable JSON data) • libraries (any Ruby code, helper functions)

  31. Chef Development lifecycle • Setup project (one-time) • Create recipes, organized inside cookbooks • Test cookbooks locally using Test Kitchen (good option) or Vagrant • Run cookbooks/recipes • Check the target server • If, e.g. a file is not in place as expected, fix recipe and repeat • Once its working, push to git (or other repository) • Run a recipe: • chef-client -o recipe[my_cookbook::my_recipe]

  32. References • Learn Chef: http://learn.getchef.com/ • Test Kitchen: https://github.com/test-kitchen/test-kitchen • Vagrant: https://www.vagrantup.com/ • ThoughtWorksTreehouse blog • ThoughtWorks: ThoughtWorks: http://www.thoughtworks.com/continuous-integration • How DevOps is killing the Developer • Etsy • Their story • Continuous delivery with product/feature flags: • Continuously deliver with confidence • Etsy deploys over 50 times a day • Adap.tv engineering blog

  33. Thank you!!! • Questions • Discuss • Feedback

More Related