1 / 35

Amazon Web Services: Building Highly Scalable Web Applications Institutional Web Management Workshop July 2007

Amazon Web Services: Building Highly Scalable Web Applications Institutional Web Management Workshop July 2007. Jeff Barr Senior Web Services Evangelist jbarr@amazon.com. Today . Introduction Amazon Web Services overview Amazon EC2 Amazon S3 Amazon SQS Application Architecture Questions.

Gabriel
Download Presentation

Amazon Web Services: Building Highly Scalable Web Applications Institutional Web Management Workshop July 2007

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Amazon Web Services:Building Highly Scalable Web ApplicationsInstitutional Web Management WorkshopJuly 2007 Jeff Barr Senior Web Services Evangelist jbarr@amazon.com

  2. Today • Introduction • Amazon Web Services overview • Amazon EC2 • Amazon S3 • Amazon SQS • Application Architecture • Questions

  3. Who am I? • Software development background • 5 Years with Amazon: • Senior Developer • Web Services Evangelist • Senior Web Services Evangelist • Veteran of several startups • Microsoft .Net Team • Visix Software co-founder

  4. Amazon? Consumers (Buy) Associates, Sellers & Merchants (Sell) Developers (Innovate)

  5. Issues Facing Developers • 70% of Web Development Effort is “Muck”: • Data Centers • Bandwidth / Power / Cooling • Operations • Staffing • Scaling is Difficult and Expensive: • Large Up-Front Investment • Invest Ahead of Demand • Load is Unpredictable

  6. Dream or Nightmare? • Slashdot/Digg/TechCrunch Effect • Rapid, unexpected customer demand/growth

  7. Seasonal Spikes

  8. More Seasonal Spikes “Every year, we take the busiest minute of the busiest hour of the busiest day and build capacity on that, we built our systems to (handle that load) and we went above and beyond that.” * -- Scott Gulbransen Intuit Spokesman * http://news.com.com/2100-1038_3-6177341.html

  9. Solution: Web-Scale Computing • Scale capacity on demand • Turn fixed costs into variable costs • Always available • Rock-solid reliability • Simple APIs and conceptual models • Cost-effective • Reduced time to market • Focus on product & core competencies

  10. Amazon Web Services Are... A set of APIs and business models which give developer-level access to Amazon’s infrastructure and content: • Search As A Service • Alexa Web Information Service • Alexa Top Sites • Alexa Site Thumbnail • Alexa Web Search Platform • Data As A Service • Amazon E-Commerce Service • Amazon Historical Pricing • Infrastructure As A Service • Amazon Simple Queue Service • Amazon Simple Storage Service • Amazon Elastic Compute Cloud • People As A Service • Amazon Mechanical Turk

  11. The Commercial Side • Standard licensing terms • Commercially usable • Aggressive pricing • Monthly credit card billing • Self-serve model: • Sign up as developer • Choose services • Agree to service licenses • Enter payment info • Start coding

  12. Web Services Billing

  13. Infrastructure Services Elastic Compute Cloud Compute Simple Storage Service Simple Queue Service Store Message

  14. Amazon Simple Storage ServiceS3

  15. Amazon Simple Storage Service $.01 for 1000 to 10000 requests $.15 per GB per month storage • Object-Based Storage • 1 B – 5 GB / object • Fast, Reliable, Scalable • Redundant, Dispersed • 99.99% Availability Goal • Private or Public • Per-object URLs & ACLs • BitTorrent Support $.10 - $.18 per GB data transfer

  16. Amazon S3 Concepts • Objects: • Opaque data to be stored (1 byte … 5 Gigabytes) • Authentication and access controls • Buckets: • Object container – any number of objects • 100 buckets per account / buckets are “owned” • Keys: • Unique object identifier within bucket • Up to 1024 bytes long • Flat object storage model • Standards-Based Interfaces: • REST and SOAP • URL-Addressability – every object has a URL

  17. S3 SOAP/Query API • Service: • ListAllMyBuckets • Buckets: • CreateBucket • DeleteBucket • ListBucket • GetBucketAccessControlPolicy • SetBucketAccessControlPolicy • GetBucketLoggingStatus • SetBucketLoggingStatus • Objects: • PutObject • PutObjectInline • GetObject • GetObjectExtended • DeleteObject • GetObjectAccessControlPolicy • SetObjectAccessControlPolicy

  18. Amazon Simple Queue ServiceSQS

  19. Amazon Simple Queue Service • Scalable Queuing • Elastic Capacity • Reliable, Simple, Secure • Inter-process messaging, data buffering, architecture component $.10 per 1000 messages $.10 - $.18 per GB data transfer

  20. Amazon SQS Concepts • Queues: • Named message container • Persistent • Messages: • Up to 256KB of data per message • Peek / Lock access model • Scalable: • Unlimited number of queues per account • Unlimited number of messages per queue

  21. Amazon SQS At Work

  22. SQS SOAP/Query API • Queues: • ListQueues • DeleteQueue • SetVisibilityTimeout • GetVisibilityTimeout • Messages: • SendMessage • ReceiveMessage • DeleteMessage • PeekMessage • Security: • AddGrant • ListGrants • RemoveGrant

  23. Amazon Elastic Compute CloudEC2

  24. Amazon Elastic Compute Cloud $.10 per server hour • Virtual Compute Cloud • Elastic Capacity • 1.7 GHz x86 • 1.7 GB RAM • 160 GB Disk • 250 MB/Second Network • Network Security Model • Time or Traffic-based Scaling, Load testing, Simulation and Analysis, Rendering, Software as a Service Platform, Hosting $.10 - $.18 per GB data transfer

  25. Amazon EC2 Concepts • Amazon Machine Image (AMI): • Bootable root disk • Pre-defined or user-built • Catalog of user-built AMIs • OS: Fedora, Centos, Gentoo, Debian, Ubuntu, Windows Server • App Stack: LAMP, mpiBLAST, Hadoop • Instance: • Running copy of an AMI • Launch in less than 2 minutes • Start/stop programmatically • Network Security Model: • Explicit access control • Security groups • Inter-service bandwidth is free

  26. Amazon EC2 At Work • Startups • Cruxy – Media transcoding • GigaVox Media – Podcast Management • Fortune 500 clients: • High-Impact, S hort-Term Projects • Development Host • Science / Research: • Hadoop / MapReduce • mpiBLAST • Load-Management and Load Balancing Tools: • Pound • Weogeo • Rightscale

  27. Images: RegisterImage DescribeImages DeregisterImage Instances: RunInstances DescribeInstances TerminateInstances GetConsoleOutput RebootInstances Keypairs: CreateKeyPair DescribeKeyPairs DeleteKeyPair Image Attributes: ModifyImageAttribute DescribeImageAttribute ResetImageAttribute Security Groups: CreateSecurityGroup DescribeSecurityGroups DeleteSecurityGroup AuthorizeSecurityGroupIngress RevokeSecurityGroupIngress EC2 SOAP/Query API

  28. Sample Web-Scale ArchitectureGigaVox Media

  29. Web-Scale Architecture

  30. GigaVox Economics • Implemented Amazon S3, Amazon EC2 and Amazon SQS in November 2006 • Created an infinitely scalable infrastructurefor less than $100 - building the same infrastructure themselves would have cost thousands of dollars • Reduced staffing requirements - far less responsibility for 24x7 operations

  31. Q & A

  32. Thank You! Pointers Portal http://aws.amazon.com Blog http://aws.typepad.com EC2 http://aws.amazon.com/ec2 S3 http://aws.amazon.com/s3 Resource Center http://aws.amazon.com/resources Forums http://aws.amazon.com/forums Jeff Barr Senior Web Services Evangelist jbarr@amazon.com

More Related