Cloud Computing lecture 10

1. Cloud Computinglecture 10 Using AWS Keke Chen

2. Outline Using EC2 Preparation Run instance from command line tools Use web console Use boto Using S3 Use boto

3. Preparation Sign up for AWS Coupon codes worth $100 per code ?, redeem at �account� Security credentials Access Key ID Security access key X.509 certificate �create a certificate� Download the private key and the certificate (i.e., the public key) and save them to ~/.ec2/

4. preparation Methods for accessing EC2 Command line tools Web console boto python library

5. preparation Ec2 command line tools have been installed at /usr/local/ec2 at nimbus17 You have to set up env varialbes JAVA_HOME EC2_HOME Add $EC2_HOME/bin to PATH EC2_PRIVATE_KEY=~/.ec2/pk-XXXXX.pem EC2_CERT=~/.ec2/cert-XXXXXXX.pem Both pk*.pem and cert*.perm are from the x.509 certificate you downloaded from your account)

6. Ready to start! Check AMIs ec2-describe-images �o self �o amazon | grep machine|less Looking for � IMAGE ami-3c47a355 ec2-public-images/getting-started.manifest.xml amazon available public i386

7. Generate key pair 1. ec2-add-keypair gsg-keypair 2. Paste the following part to the file ~/.ec2/id_rsa-gsg-keypair -----BEGIN RSA PRIVATE KEY----- �. -----END RSA PRIVATE KEY----- 3. chmod 600 id_rsa-gsg-keypair

8. Run an instance ec2-run-instances ami-3c47a355 �k gsg-keypair ec2-describe-instances i-395bf151

9. Get connected Authorize accesses to ports ec2-authorize default �p 22 ec2-authorize default �p 80 -- enable ssh and web Or start with some security group ec2-run-instances ami-xxxxx -g apache Connect to your instance http://ec2-67-202-28-87.compute-1.amazonaws.com ec2-get-console-output i-395bf151 ssh -i ~/.ec2/id_rsa-gsg-keypair root@ec2-67-202-28-87.compute-1.amazonaws.com

10. Clean up Terminate the instance ec2-terminate-instances i-395bf151 Or in the instance, run shutdown �h now

11. Using AWS console

12. Use boto to access EC2 Create connection >>> from boto.ec2.connection import EC2Connection >>> conn = EC2Connection('<aws access key>', '<aws secret key>') Or if you have set the keys in AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY >>> import boto >>> conn = boto.connect_ec2()

13. Images >>> images = conn.get_all_images() >>> images >>> for i in range(len(images)): ... print i, images[i].location

14. Run instance >>> image = images[xxx] # some selected image >>> reservation = image.run() # have various parameter settings, such as key, security group, instance type, etc. >>> reservation.instances [Instance:i-6761850e] >>> instance = reservation.instances[0] >>> instance.state u'pending� >>> instance.update() >>> instance.state u'pending' >>> # wait a few seconds to minutes >>> instance.update() >>> instance.state u'running'

15. Retrieve information of instance >>> instance.dns_name u'ec2-72-44-40-153.z-2.compute-1.amazonaws.com' >>> instance.public_dns_name u'ec2-72-44-40-153.z-2.compute-1.amazonaws.com' >>> instance.private_dns_name u'domU-12-31-35-00-42-33.z-2.compute-1.internal'

16. Run multiple instances >>> reservation.image.run(2,2,'gsg-keypair') >>> reservation.instances [Instance:i-5f618536, Instance:i-5e618537] >>> for i in reservation.instances: ... print i.status u'pending' u'pending' >>>

17. Terminate instances >>> instance.stop() >>> instance.update() >>> instance.state u'shutting-down' >>> # wait a minute >>> instance.update() >>> instance.state u'terminated' For multiple instances >>> reservation.stop_all() >>> instances = conn.get_all_instances() >>># then check each instance

18. Security Set launch permission for private AMIs image.get_launch_permission() image.set_launch_permission(list_of_AWS_user_IDs) image.remove_launch_permission(list_of_AWS_user_IDs) Image.reset_launch_permission()

19. Security Security groups For network accesses to service ports A collection of access rules >>> rs = conn.get_all_security_groups() >>> print rs [SecurityGroup:appserver, SecurityGroup:default, SecurityGroup:vnc, SecurityGroup:webserver] >>>

20. >>> sg = rs[1] >>> sg.name u'default' >>> sg.rules [IPPermissions:tcp(0-65535), IPPermissions:udp(0-65535), IPPermissions:icmp(-1--1), IPPermissions:tcp(22-22), IPPermissions:tcp(80-80)] >>>

21. Create a security group >>> web = conn.create_security_group('apache', 'Our Apache Group') >>> web SecurityGroup:apache >>> web.authorize('tcp', 80, 80, '0.0.0.0/0') True >>> web.authorize(ip_protocol='tcp', from_port=22, to_port=22, cidr_ip='192.168.1.130/32') True

22. Revoke permission >>> web.rules [IPPermissions:tcp(80-80), IPPermissions:tcp(22-22)] >>> web.revoke('tcp', 22, 22, cidr_ip='192.168.1.130/32') True >>> web.rules [IPPermissions:tcp(80-80)] >>>

23. Regions >>> import boto.ec2 >>> regions = boto.ec2.regions() >>> regions [RegionInfo:eu-west-1, RegionInfo:us-east-1] Bind to specific regions >>> eu = regions[0] >>> conn_eu = eu.connect()

24. Copy resources to new region Supporting EC2 objects User created SecurityGroups, KeyPairs, Addresses, Volumns, Images, and SnapShots Local to a particular region Copy to a new region Example � SecurityGroup: >>eu_group = us_group.copy_to_region(eu)

25. S3 � quick review Objects are organized in a two-level directory Bucket container of objects Global unique name Key Like file names Unique in the same bucket Object Indexed by (bucket, key) http://bucket.s3.amazonaws.com/key

26. S3 Programming tools

27. Check out AWS Developer Resource Center, for more programming examples We will take a look at boto library It is already installed with python at nimbus17

28. Create a connection >>> from boto.s3.connection import S3Connection >>> conn = S3Connection('<aws access key>', '<aws secret key>') These two keys can be found in your security credentials

29. If you have set the keys in AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY >>> import boto >>> conn=boto.connect_s3()

30. Creating a bucket >>> bucket = conn.create_bucket(�mybucket�) Note that mybucket is globally (in the entire S3 system) uniuqe

31. Storing data >>> from boto.s3.key import Key >>> k = Key(bucket) >>> k.key = 'foobar' >>> k.set_contents_from_string('This is a test of S3')

32. Retrieve data >>> import boto >>> c = boto.connect_s3() >>> b = c.create_bucket('mybucket') # substitute your bucket name here >>> from boto.s3.key import Key >>> k = Key(b) >>> k.key = 'foobar' >>> k.get_contents_as_string() 'This is a test of S3'

33. Work on files >>> k = Key(b) >>> k.key = 'myfile' >>>k.set_contents_from_filename('foo.jpg') >>> k.get_contents_to_filename('bar.jpg')

34. Check all created buckets >>> rs = conn.get_all_buckets() Rs is a list of buckets >>> len(rs) >>> for b in rs: � print b.name � Listing of all available buckets

35. Set access control Set public readable for entire bucket >>> b.set_acl('public-read') For one object >>> b.set_acl('public-read�, �foobar�) Or if k is a Key >>>k.set_acl(�public-read�)

36. Check ACL >>> acp = b.get_acl() >>> acp <boto.acl.Policy instance at 0x2e6940> >>> acp.acl <boto.acl.ACL instance at 0x2e69e0> >>> acp.acl.grants [<boto.acl.Grant instance at 0x2e6a08>] >>> for grant in acp.acl.grants: ... print grant.permission, grant.grantee ... FULL_CONTROL <boto.user.User instance at 0x2e6a30>

37. Meta data with objects >>> k = Key(b) >>> k.key = 'has_metadata' >>> k.set_metadata('meta1', 'This is the first metadata value') >>> k.set_metadata('meta2', 'This is the second metadata value') >>>k.set_contents_from_filename('foo.txt') >>> k = b.lookup('has_metadata) >>> k.get_metadata('meta1') 'This is the first metadata value'