1 / 38

Cloud Computing lecture 10

Cloud Computing lecture 10. Using AWS Keke Chen. Outline. Using EC2 Preparation Run instance from command line tools Use web console Use boto Using S3 Use boto. Preparation. Sign up for AWS Coupon codes worth $100 per code , redeem at “account” Security credentials

pier
Download Presentation

Cloud Computing lecture 10

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Cloud Computinglecture 10 Using AWS Keke Chen

  2. Outline • Using EC2 • Preparation • Run instance from command line tools • Use web console • Use boto • Using S3 • Use boto

  3. Preparation • Sign up for AWS • Coupon codes • worth $100 per code , redeem at “account” • Security credentials • Access Key ID • Security access key • X.509 certificate • “create a certificate” • Download the private key and the certificate (i.e., the public key) and save them to ~/.ec2/

  4. preparation • Methods for accessing EC2 • Command line tools • Web console • boto python library

  5. preparation • Ec2 command line tools have been installed at /usr/local/ec2 at nimbus17 • You should set up env varialbes • JAVA_HOME • EC2_HOME • Add $EC2_HOME/bin to PATH • EC2_PRIVATE_KEY=~/.ec2/pk-XXXXX.pem • EC2_CERT=~/.ec2/cert-XXXXXXX.pem Both pk*.pem and cert*.perm are from the x.509 certificate you downloaded from your account)

  6. Sample setup in .profile export EC2_HOME=/usr/local/ec2/ export PATH=/usr/local/gae/:/opt/matlab/bin/:/usr/local/hadoop/bin:$EC2_HOME/bin:/usr/local/pig/bin:$PATH export EC2_PRIVATE_KEY=~/.ec2/pk-xxxxxxx.pem export EC2_CERT=~/.ec2/cert-xxxxxxx.pem export AWS_ACCESS_KEY_ID=xxxxxxxxxx export AWS_SECRET_ACCESS_KEY=xxxxxxxx export AWS_ACCOUNT_ID=xxxxxxxxx

  7. Ready to start! • Check AMIs ec2-describe-images –o self –o amazon | grep machine|less Looking for … IMAGE ami-3c47a355 ec2-public-images/getting-started.manifest.xml amazon available public i386

  8. Generate key pair 1. ec2-add-keypair gsg-keypair 2. Paste the following part to the file ~/.ec2/id_rsa-gsg-keypair -----BEGIN RSA PRIVATE KEY----- …. -----END RSA PRIVATE KEY----- 3. chmod 600 ~/.ec2/id_rsa-gsg-keypair

  9. Run an instance • ec2-run-instances ami-3c47a355 -k gsg-keypair • ec2-describe-instances i-395bf151 RESERVATION r-29f61541 425984194090 default INSTANCE i-395bf151 ami-3c47a355 pending gsg-keypair 0 m1.small 2009-10-13T05:16:54+0000 us-east-1b aki-a71cf9ce ari-a51cf9cc monitoring-disabled RESERVATION r-29f61541 425984194090 default INSTANCE i-395bf151 ami-3c47a355 ec2-67-202-28-87.compute-1.amazonaws.com domU-12-31-39-06-AC-33.compute-1.internal running gsg-keypair 0m1.small 2009-10-13T05:16:54+0000 us-east-1b aki-a71cf9ce ari-a51cf9cc monitoring-disabled 67.202.28.87 10.208.179.193

  10. Get connected • Authorize accesses to ports • ec2-authorize default –p 22 • ec2-authorize default –p 80 -- enable ssh and web • Or start with some security group • ec2-run-instances ami-xxxxx -g apache • Connect to your instance • http://ec2-67-202-28-87.compute-1.amazonaws.com • ec2-get-console-output i-395bf151 • ssh -i ~/.ec2/id_rsa-gsg-keypair root@ec2-67-202-28-87.compute-1.amazonaws.com

  11. Clean up • Terminate the instance • ec2-terminate-instances i-395bf151 • Or in the instance, run shutdown –h now

  12. Using AWS console

  13. Use boto to access EC2 • Create connection >>> from boto.ec2.connection import EC2Connection >>> conn = EC2Connection('<aws access key>', '<aws secret key>') Or if you have set the keys in AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY >>> import boto >>> conn = boto.connect_ec2()

  14. Images >>> images = conn.get_all_images() >>> images >>> for i in range(len(images)): ... print i, images[i].location

  15. Run instance >>> image = images[xxx] # some selected image >>> reservation = image.run() # have various parameter settings, such as key, security group, instance type, etc. Or reservation=conn.run_instances('<ami-image-id>') >>> reservation.instances [Instance:i-6761850e] >>> instance = reservation.instances[0] >>> instance.state u'pending‘ >>> instance.update() >>> instance.state u'pending' >>> # wait a few seconds to minutes >>> instance.update() >>> instance.state u'running'

  16. Retrieve information of instance >>> instance.dns_name u'ec2-72-44-40-153.z-2.compute-1.amazonaws.com' >>> instance.public_dns_name u'ec2-72-44-40-153.z-2.compute-1.amazonaws.com' >>> instance.private_dns_name u'domU-12-31-35-00-42-33.z-2.compute-1.internal'

  17. Run multiple instances >>> reservation.image.run(2,2,'gsg-keypair') >>> reservation.instances [Instance:i-5f618536, Instance:i-5e618537] >>> for i in reservation.instances: ... print i.status u'pending' u'pending' >>>

  18. Terminate instances >>> instance.stop() >>> instance.update() >>> instance.state u'shutting-down' >>> # wait a minute >>> instance.update() >>> instance.state u'terminated' For multiple instances >>> reservation.stop_all() >>> instances = conn.get_all_instances() >>># then check each instance

  19. Security • Set launch permission for private AMIs image.get_launch_permission() image.set_launch_permission(list_of_AWS_user_IDs) image.remove_launch_permission(list_of_AWS_user_IDs) Image.reset_launch_permission()

  20. Security • Security groups • For network accesses to service ports • A collection of access rules >>> rs = conn.get_all_security_groups() >>> print rs [SecurityGroup:appserver, SecurityGroup:default, SecurityGroup:vnc, SecurityGroup:webserver] >>>

  21. >>> sg = rs[1] >>> sg.name u'default' >>> sg.rules [IPPermissions:tcp(0-65535), IPPermissions:udp(0-65535), IPPermissions:icmp(-1--1), IPPermissions:tcp(22-22), IPPermissions:tcp(80-80)] >>>

  22. Create a security group >>> web = conn.create_security_group('apache', 'Our Apache Group') >>> web SecurityGroup:apache >>> web.authorize('tcp', 80, 80, '0.0.0.0/0') True >>> web.authorize(ip_protocol='tcp', from_port=22, to_port=22, cidr_ip='192.168.1.130/32') True

  23. Revoke permission >>> web.rules [IPPermissions:tcp(80-80), IPPermissions:tcp(22-22)] >>> web.revoke('tcp', 22, 22, cidr_ip='192.168.1.130/32') True >>> web.rules [IPPermissions:tcp(80-80)] >>>

  24. Regions >>> import boto.ec2 >>> regions = boto.ec2.regions() >>> regions [RegionInfo:eu-west-1, RegionInfo:us-east-1] Bind to specific regions >>> eu = regions[0] >>> conn_eu = eu.connect()

  25. Copy resources to new region • Supporting EC2 objects • User created • SecurityGroups, KeyPairs, Addresses, Volumns, Images, and SnapShots • Local to a particular region • Copy to a new region Example – SecurityGroup: >>eu_group = us_group.copy_to_region(eu)

  26. S3 – quick review • Objects are organized in a two-level directory • Bucket • container of objects • Global unique name • Key • Like file names • Unique in the same bucket • Object • Indexed by (bucket, key) • http://bucket.s3.amazonaws.com/key

  27. S3 • Programming tools

  28. Check out AWS Developer Resource Center, for more programming examples • We will take a look at boto library • It is already installed with python at nimbus17

  29. Create a connection >>> from boto.s3.connection import S3Connection >>> conn = S3Connection('<aws access key>', '<aws secret key>') These two keys can be found in your security credentials

  30. If you have set the keys in AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY >>> import boto >>> conn=boto.connect_s3()

  31. Creating a bucket >>> bucket = conn.create_bucket(‘mybucket’) Note that mybucket is globally (in the entire S3 system) uniuqe

  32. Storing data >>> from boto.s3.key import Key >>> k = Key(bucket) >>> k.key = 'foobar' >>> k.set_contents_from_string('This is a test of S3')

  33. Retrieve data >>> import boto >>> c = boto.connect_s3() >>> b = c.create_bucket('mybucket') # substitute your bucket name here >>> from boto.s3.key import Key >>> k = Key(b) >>> k.key = 'foobar' >>> k.get_contents_as_string() 'This is a test of S3'

  34. Work on files >>> k = Key(b) >>> k.key = 'myfile' >>>k.set_contents_from_filename('foo.jpg') >>> k.get_contents_to_filename('bar.jpg')

  35. Check all created buckets >>> rs = conn.get_all_buckets() Rs is a list of buckets >>> len(rs) >>> for b in rs: … print b.name … Listing of all available buckets

  36. Set access control • Set public readable for entire bucket >>> b.set_acl('public-read') • For one object >>> b.set_acl('public-read‘, ‘foobar’) Or if k is a Key >>>k.set_acl(‘public-read’)

  37. Check ACL >>> acp = b.get_acl() >>> acp <boto.acl.Policy instance at 0x2e6940> >>> acp.acl <boto.acl.ACL instance at 0x2e69e0> >>> acp.acl.grants [<boto.acl.Grant instance at 0x2e6a08>] >>> for grant in acp.acl.grants: ... print grant.permission, grant.grantee ... FULL_CONTROL <boto.user.User instance at 0x2e6a30>

  38. Meta data with objects >>> k = Key(b) >>> k.key = 'has_metadata' >>> k.set_metadata('meta1', 'This is the first metadata value') >>> k.set_metadata('meta2', 'This is the second metadata value') >>>k.set_contents_from_filename('foo.txt') >>> k = b.lookup('has_metadata) >>> k.get_metadata('meta1') 'This is the first metadata value'

More Related