Choosing the containerized cloud provisioning tool that best suits your need

1. Choosing the containerized cloud provisioning tool that best suits your need Smruti Soumitra Khuntia, Solution Architect Pradeep Kumar K S, Solution Architect Hemanth Kumar Nakkina, Solution Architect Ericsson

2. Agenda • Why containers for cloud provisioning? • Containerized OpenStack deployment • Key attributes of Containerized Deployments • Airship overview • Kolla overview • TripleO overview • Comparison

3. Why containers for cloud provisioning? • Microservices help to decompose complex operations into smaller composable parts • Quicker to bring them up and down • Containers can scale, upgrade and monitored independently • Increasing efficiency, reliability and scalability • Opensource and thus fosters innovation and rapid development

4. Containerised OpenStack deployment

5. Key features of Containerized Cloud Provisioning tools • Container technologies/tools • Deployment strategies • Infrastructure Configurations • Logging, Monitoring, Health & Auditing • Bare metal provisioning • Container Networking • Security • Storage Support • Day 2 Operations & Upgrades

6. Containerized Cloud Deployment Solutions

7. Introduction to Airship • Airship is a collection of loosely coupled but interoperable open source tools that declaratively automate cloud provisioning. • Airship manages the full lifecycle of data center infrastructure to deliver a production-grade Kubernetes cluster with Helm deployed artifacts, including OpenStack-Helm from raw bare metal infrastructure. • Airship allows operators to manage their infrastructure deployments and lifecycle through the declarative YAML documents that describe an Airship environment.

8. Architecture

9. Airship Workflow

10. Airship Workflow 4) Manually provision ”genesis” node central site site • 1) Specify all info in YAML files • Application templates • Openstack helm charts • Kubernetes helm charts • Security • Generate Passphrases, certificatesPKI structure • Site info • Node hardwareDisks, storage (ceph) setup • Ports and VLANs (Oob/PXE/OAM/Data) site 5) Run the seed script Self hosted Kubernetes cluster Install all undercloud components- Bare metal deployer Install applications- Openstack control plane- (Other applications) Transform node to first cluster controller Genesis/Controller1 Controller2 ~350 Controller3 3) Generateseed script 2) Combine & validate Controller4 Worker Genesis.sh(40MB) site.yaml 2) Combine & validate Worker

11. Kolla

12. Kolla Consists of three main projects Kolla-OpenStack (Kolla) • Provides Dockerfiles for each service and each process in the services • Container external configurations Kolla Images • Ansible code to build those containers • Lets you build your own images on your own private registry or use public images Kolla-Ansible • Kolla provides production-ready containers and deployment tools • Ansible code to deploy, start, configure and re-configure/upgrade a system • Allows for complete customization • Uses lightweight Docker containers to deploy OpenStack services using Ansible.

13. Kolla images build process

14. Kolla Deployment Model/Workflow • Setup the baseline OS • Get OS Installed • Install required tools • Clone or pip install the Kolla tools • Configure and install Docker • Ensure shared mounts are enabled • Configure an insecure registry or pass TLS certificates information for local registry

15. Kolla Deployment Model/Workflow • Setup globals.yml and passwords.yml files • Globals is a way to turn off and on services • Define network and storage interactions with the underlying system. • Passwords defines default passwords, keys, UUIDs etc. • Can be generated automatically using kolla-genpwd tool • Setup multi node inventory Kolla-ansible bootstrap-servers –i inventory_file Kolla-ansible deploy –i inventory_file

16. TripleO Deployment Overview Deployment and Management Cloud • Knowns as Undercloud • Responsible for deployment of overcloud • Provide infrastructure command and control • Only applicable for cloud operator • Conducts all lifecycle management Deploys, Updates Monitors Overcloud OpenStack Undercloud OpenStack TripleO • Tenant Facing Cloud • Knowns as Overcloud • The cloud responsible to run tenant workloads • Visible to Tenant administrator • Provides cloud resources to run tenant • Apps/VNFs.

17. Physical View

18. TripleO Deployment Flow

19. Components of Containerized TripleO • Pre-Built Docker Images • Kolla • Configuration Generation • docker-puppet.py • Container management • Paunch • Updates and Upgrades • Ansible with TripleO Heat templates

20. Config generation for containerized services docker-puppet.json Bootstrap container Puppet Generates configs docker-puppet.py Docker-puppet.py generates configs for each service and available through puppet hiera

21. Kolla Start • Kolla start is responsible for: • Copying configuration files • Setting permissions • Starting the Service process config.json Service Container copy config set permission Kolla Start Configdir Start Process

22. Comparisons

23. Container Technologies/Tools • Airship • Uses Kubernetes • Helm charts ( from OpenStack Helm ) • Loci for image builds • TripleO • Paunch • Podman/Docker • Kolla-ansible • Kolla for image builds

24. Infrastructure configurations • Airship • Defines YAML manifests at global, type, site level • Reuse the files defined at global and type • Only site level manifests get modified for each site • Everything is a YAML configuration. Complex initially to understand the YAML manifests • YAML manifests are released by Airship community as part of Treasuremap project every month • On the brighter side, the manifests provide you control to define settings at a very low level • TripleO • Defines manifests in TripleO-heat-templates under • puppet/services • docker/services • extraconfig/service • YAML files with custom keys to modify

25. Deployment strategy • Airship • Deployment is performed via Genesis node which bootstraps the kubernetes cluster with control and compute nodes • Genesis node itself becomes one of the control nodes post the initial bootstrap process • This would mean that under cloud components are always running on the actual site. • By this virtue, the site manifests remain on Deckhand database registry • This would provide the current/actual state of under cloud within the site itself • Thus not needing a separate hardware for managing under-cloud LCM. • TripleO • Deployment of the under cloud happens via seed node which bootstraps other nodes using an under-cloud OpenStack • Under-cloud OpenStack will be used to deploy the OpenStack overcloud using heat templates and Kolla • Under cloud would be more of a plug & play and can be shut-off and turned-on whenever required.

26. Bare metal server management • Airship • MAAS Plugin for Drydock • iDRAC/iLO management using IPMI plugin, redfish plugin for drydock • Integration with Kubernetes cluster-api with Metal 3‘ with Airship ( WIP) • TripleO • Uses Ironic project for bootstrapping bare metal servers • Update configuration in heat templates

27. Networking • Airship • Uses calico by default for control plane networking • Calico provides ability to set network policies • Can deploy any Kubernetes CNI plugins by changing the Airship manifests, provided helm charts are available for CNI • TripleO • Networking in Containerized TripleO is similar to Non Containerized TripleO • Host based networking is leveraged for all containerized services • Docker *Host* Driver is utilized.

28. Security • Airship • Utilizes all the Kubernetes native security features • Calico network policies • Setting security context for POD/container ( privileges/non-privileged, linux capabilities) • AppArmor • Seccomp • TripleO • Audit rules • Firewall management rules • AIDE ( File directory integrity checker)

29. Logging, Monitoring & Auditing • Logging • Airship • Uses EFK stack ( Fluentd, ElasticSearch, Kibana) • TripleO • Uses EFK stack ( Fluentd*, ElasticSearch, Kibana) • Monitoring • Airship • Uses Prometheus, ElasticSearch, Grafana • TripleO • Uses Sensu* *might be deprecated in future releases

30. Storage • Airship • Supports Ceph and NFS storage drivers • TripleO • Support Ceph and other NFS storage drivers • Under-cloud can be configured to use Swift/Cinder Object Storage as image backend

31. Updates & Upgrades • Airship • Typical process is to use the same set of Under cloud tools to do Day 2 operations and upgrades • Update the YAML manifests and deploy the new manifests with Shipyard • Host impacting components ( Kubelet, Docker, kernel upgrades) using Argo Workflows ( WIP ) • Upgrades for most of the services are handled via helm charts • TripleO • Ansible scripts to perform updates, pre/post update operations triggered via OpenStack overcloud command • Ansible scripts for the under cloud upgrade (using FFU)