1 / 18

Project – C IaaS – Resource Management

Project – C IaaS – Resource Management. Akshay Patekar Nikhil Bharadwaj Ramesha Sharath Chandra Kanaka R Suthan Venkataramanaiah Vidya Ravivarma. AGENDA. Project Overview Background Study Architecture Q Clouds (Paper Exploration) Openstack Setup Ganglia Setup Instance Management

alissa
Download Presentation

Project – C IaaS – Resource Management

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Project – CIaaS – Resource Management AkshayPatekar Nikhil Bharadwaj Ramesha Sharath Chandra Kanaka R Suthan Venkataramanaiah VidyaRavivarma

  2. AGENDA • Project Overview • Background Study • Architecture • Q Clouds (Paper Exploration) • Openstack Setup • Ganglia Setup • Instance Management • Algorithm Design • Dynamic VM Resize and Live Migration • Issues faced • What Next?

  3. Project Overview • Continuously monitor the cloud environment • Process the input and take decisions based on resource utilization metrics • An algorithmic approach • Perform load balancing • Dynamically adjust resources and live migrations of VMs

  4. Background Study- Architecture Openstack Master Ganglia Master Gmond collects the monitoring data and sends it to the Gmetad which is running on Ganglia Master VM VM VM VM It directs VM s to migrate to another physical server in case of application degradation Load Balancing algorithm runs here on top of Ganglia monitored data and sends the decisions for VM migrations. A programs continuously runs by polling for the decisions from the Load Balancing Algorithm and calls upon Openstack APIs for VM migrations

  5. Background Study - QClouds • A QoS aware framework that tunes resource allocations to mitigate performance interference effects • Three main Components in this framework • Scheduler : Module which polls and feeds ganglia monitored data to resource mitigation control • Resource Mitigation Control: Core module to arrive at optimized resource allocation values for performance degraded VMs • Dynamic Migrator: Module in charge of dynamic VM migration based on optimized allocation values • VMs are migrated based on Q-State • A custom performance metric • Our algorithm is based on this paper

  6. Openstack Setup • Setting up the environment for Openstack • Openstack – multinode installation • Openstack master installed on host : Openstack • Openstack slave installed on host Compute1 and Compute2 • Installation done using devstack • $ >./stack.sh • Shell script is used for installing Openstack

  7. Openstack – Modules • Modules configured • Nova, Keystone, Glance, Horizon • In Openstack, each VM has two IPs: fixed and floating • VMs are configured using these IPs • Nova, the compute component allows to manage multiple VM instances • Nova Manage is used to manage Openstack services • $ nova-manage service list • $ nova-manage service disable host=<host> service=<service_name>

  8. Ganglia Setup - Architecture

  9. Ganglia Setup • Ganglia master is run on a dedicated VM • Gmetad, Gmond , Ganglia web frontend installed and configured • Ganglia slaves (Gmond) run on each VMs • VMs running benchmark application • Configuration to specify Gmond representative • $> data_source “Cluster openstack" 192.168.0.131 • Master polls to fetch data from this Gmond • Metrics monitored are CPU_module, Disk_module, Load_module, Mem_module, Proc_module. • rrd file created for each of the metric

  10. VM Instance Creation • Using nova command we instantiated VM on compute nodes • First we need to create key-pair for each compute node nova keypair-add --pub_key ~/.ssh/id_rsa.pub KEY_NAME • Creating Instance nova boot image <image name> flavor <flavor id> key-pair <keypair>

  11. VM Instance Snapshot

  12. Algorithm Design GANGLIA MASTER OPENSTACK MASTER GMETAD VM1 VM2 VM3 Data Reader() Nova Module APIs DecisionMsg() Load Balancer() DecisionPolling()

  13. Dynamic Resize • Nova resize command used to change the flavor of the running VM • nova resize <instance name> <flavor> • Flavor gives the new set of metrics for the VM • VM is resized while it is running • VM resize fails if it is in any other state • Nova Client Python used to write the python script to do resize • The script accepts instance to be resized and changes the flavor of the running VM • Our Load Balancing algorithm uses this script for dynamic resizing

  14. Dynamic Resize Code This script takes one parameter – Source VM Name From novaclient.v1_1 import client from credentials import get_nova_creds import os import sys import time creds=get_nova_creds() nova=client.Client(**creds) instance_name=sys.argv[1] instances= nova.servers.find(name=instance_name) print instances.name flavor=instances.flavor print instances.id,instances.status,flavor['id'] instances.reset_state(state='active') flavor_id=int(flavor['id'])+1 instances.resize(flavor_id) print "Resizing" time.sleep(50) print "Confirm resize" instances.confirm_resize() time.sleep(20) instances.reboot(reboot_type='SOFT') print "Resize Done!!!"

  15. Dynamic Live Migration • Set up NFS server in controller and NFS-common slaves • Shared storage based live migration • Ex.: /opt/stack/data/nova/instances • Mounted on all servers for visibility across all compute nodes • Initially tested with nova command • nova live_migration <instance_id> <host_ip> • Migration managed by the controller • Using nova client python wrapper functions, migration script was written • Instance ID and destination Host will provided as the arguments • Migration managed by the Controller

  16. Dynamic Live Migration This script takes two parameters - source VM Name and Destination_server from novaclient.v1_1 import client from credentials import get_nova_creds import os import sys import time creds=get_nova_creds() nova=client.Client(**creds) source=sys.argv[1] #instance_id=sys.argv[2] destination=sys.argv[2] instance=nova.servers.find(name=source) print "Migration started" instance.live_migrate(destination,False,False) print "Migration Done!!!"

  17. Issues faced and solved • Not all Services come up after the machine is restarted • Need to run rejoin_stack.sh script • Nova network services enabling issue solved using NTP server • To keep controller and all compute nodes in synch • No proper documentation for usage of Nova Client python wrappers • Referred the source code to write the dynamic resizing and migration scripts

  18. What Next? • Install benchmark applications and configure Ganglia for these instances • Gmond monitors the application resource usage • Design of load balancing algorithm based on Q-Clouds • Implement and Test the algorithm by using different sets of VMs in the Openstack cloud environment • Improve the algorithm to meet scalability

More Related