The Use of Microsoft based Technologies for the benefit of the Community

1. The Use of Microsoft based Technologies for the benefit of the Community Prof. Avi Mendelson – Microsoft & Technionmendlson@cs.technion.ac.il - Taub 643 Prof. Ron Friedman – CS Technionroy@cs.technion.ac.il – Taub 605

2. אדמיניסטרציות • הגדרת דרישות • מעבר לתכנון

3. תהליך גיבוש פרויקט • מסמך דרישות • המסמך הריק - ..\Assist material\General\Top_Level_Design.empty.doc • דוגמא למסמך מלא - ..\Assist material\General\Scriptoexample\TLD.Group6v0.03.doc • מסמך Top Level Design

4. azure • Installation: http://www.microsoft.com/windowsazure/sdk/

5. What is the cloud? An approach to computing that’s about internet scale and connecting to a variety of devices and endpoints

6. Cloud Services Infrastructure-as-a-Service “PaaS” “IaaS” “SaaS” Platform-as-a-Service Software-as-a-Service consume build host

7. IaaS, PaaS and SaaS • Infrastructure as a Service • Platform as a Service • Software as a Service SaaS ApplicationsPackaged Software PaaS PlatformOS & Application Stack PlatformOS & Application Stack IaaS InfrastructureServers · Storage · Network InfrastructureServers · Storage · Network InfrastructureServers · Storage · Network

8. Common Factors • Pay per use • Instant Scalability • Security • Reliability • APIs

9. Advantages • Lower cost of ownership • Reduce infrastructure management responsibility • Allow for unexpected resource loads • Faster application rollout

10. What Is Windows Azure? • It is an operating system for the cloud

11. Introducing Windows Azure Windows Azure is an internet-scale cloud services platform hosted in Microsoft data centers around the world, proving a simple, reliable and powerful platform for the creation of web applications and services.

12. Windows Azure Platform Data Centers North America Region Europe Region Asia Pacific Region N. Europe N. Central – U.S. W. Europe S. Central – U.S. E. Asia S.E. Asia • 6 datacenters across 3 continents Simply select your data center of choice when deploying an application

13. So, What Is Windows Azure? • It is an operating system for the cloud • It is designed for utility computing • It provides facilities to: • Write your apps (developer experience) • Host your apps (compute) • Manage your apps (service management) • Store your data (storage)

14. Windows Azure Platform Service Guarantee • Storage servicewill be available/ reachable (connectivity) • Your storage requests willbe processed successfully • .NET Service Bus endpoint willhave external connectivity • Message operation requests willbe processed successfully Serviceavailability Storage availability Database availability • Role instance monitoring and restart Compute connectivity Database is connected to the internet gateway Availability monitoring every 5-minute interval All runningroles will be continuously monitored If role is unhealthy, we will detect and initiate corrective state Your service is connected and reachable via web Internet facing roles will have external connectivity Automated Systems Management >99.9% >99.95% >99.9% >99.9%

15. Windows Azure for Application Developers

16. Windows Azure Web & Worker Roles • Windows Server 2008 x64 (R2 coming soon) • .NET Framework – 3.5 SP1 and 4.0 • Native Code, PHP, Java, etc. – Bring your own runtime • Supports Full Trust today (admin mode coming soon) • Fabric manages role lifecycle Worker Role Web Role • Role is hosted on IIS • HTTP/HTTPS • ASP.NET • Fast CGI + PHP • Role is an executable • Create your own web server, host a database, etc. • Inbound on • Any TCP Port • HTTP/HTTPS

17. Roles - cont • We also have VM-Role

18. Roles and Instances • Roles are defined in a Service Model • May define one or more Roles per Service • A role definition specifies • VM size • Communication Endpoints • Local storage resources • Etc… • At runtime each Role will execute on one or more instances (up to 20 per subscription) • A role instance is a set of code, configuration, and local data, deployed in a dedicated VM

19. Windows Azure Service Architecture The Internet The Internet via TCP or HTTP Tables LB LB LB Storage Queues Web Site (ASPX, ASMX, WCF) Worker Service Web Site (ASPX, ASMX, WCF) Worker Service Web Role IIS as Host Worker Role Managed Interface Call Blobs Windows Azure Data Center

20. Role Programming Model • Inherits RoleEntryPoint • OnStart() Method • Called by Fabric on startup, allows you to perform initialization tasks. • Reports Busy status to load balancer until you return true. • Run() Method • Main logic is here – can do anything, typically infinite loop. Should never exit. • OnStop() Method • Called when role is to be shutdown, graceful exit. • 30 Seconds to tidy up

21. Worker Role Patterns • Queue Polling Worker • Poll and Pop Messages within while(true) loop • E.g. Map/Reduce pattern, background image processing • Listening Worker Role • Create TcpListener or WCF Service Host • E.g. Run a .NET SMTP server or WCF Service • External Process Worker Role • OnStart or Run method executes Process.Start() • Startup Task installs or executes background/foreground process • E.g. Run a database server, web server, distributed cache

22. Web Role • All features of a worker role + IIS 7 or 7.5 • ASP.NET 3.5 SP1 or 4.0 – 64bit • Hosts • Webforms or MVC • FastCGI applications (e.g. PHP) • Multiple Websites • Http(s) • Web/Worker Hybrid • Can optionally implement RoleEntryPoint

23. Service Definition • Describes the shape of your Windows Azure Service • Defines Roles, Ports, Certificates, Configuration Settings, Startup Tasks, IIS Configuration, and more… • Can only be changed by upgrades or new deployments

24. Service Configuration • Supplies Runtime Values (Scale, Config Settings, Certificates to use, VHD, etc.) • Can be updated any time through Portal or API

25. VM Size in Windows Azure • Windows Azure • Supports Various VM Sizes • Size set on Role in Service Definition • Service can have multiple roles • Balance of Performance per node vs. High Availability from multiple nodes • Set in Service Model definitionAll instances of role will be of equal size

26. demo Hello Windows Azure

27. First Phone cloud application • Cloud side

28. First output

30. Phone side

32. Windows Azure Storage

33. Windows Azure Storage • Scalable storage in the cloud • 100tb per storage account • Auto-scale to meet massive volume and throughput • Accessible via RESTful Web Service API • Access from Windows Azure Compute • Access from anywhere via internet • Supporting .NET Client Library • Various storage types • Table - group of entities (name/value pairs) • Queue - Simple non-transactional message queue • Blob - Large binary storage • Drives - NTFS VHD mounted into Compute instance

34. Storage in the Development Fabric • Provides a local “Mock” storage • Emulates storage in cloud • Allows offline development • Requires SQL Express 2005/2008 or above There are some differences between Cloud and Dev Storage.http://msdn.microsoft.com/dd320275 A good approach for developers:To test pre-deployment, push storage to the cloud first Use Dev Fabric for compute connect to cloud hosted storage.Finally, move compute to the cloud.

35. Blobs Blobs Containers Photo1.png Photo2.png Pictures Photo3.png

36. Account Container Blob Pages/ Blocks Blob Storage Concepts http://<account>.blob.core.windows.net/<container>/<blobname> PIC01.JPG Block/Page images VID1.AVI PIC02.JPG contoso Block/Page videos

37. Blob Details • Main Web Service Operations • PutBlob • GetBlob • DeleteBlob • CopyBlob • SnapshotBlob • LeaseBlob • Associate Metadata with Blob • Standard HTTP metadata/headers (Cache-Control, Content-Encoding, Content-Type, etc) • Metadata is <name, value> pairs, up to 8KB per blob • Either as part of PutBlob or independently • Blob always accessed by name • Can include ‘/‘ or other delimeter in name e.g. /<container>/myblobs/blob.jpg

38. Blob Containers • Multiple Containers per Account • Special $root container • Blob Container • A container holds a set of blobs • Set access policies at the container level • Associate Metadata with Container • List the blobs in a container • Including Blob Metadata and MD5 • NO search/query. i.e. no WHERE MetadataValue = ? • Blobs Throughput • Effectively in Partition of 1 • Target of 60MB/s per Blob

39. Enumerating Blobs • GET Blob operation takes parameters • Prefix • Delimiter • Include= (snapshots, metadata etc…) http://adventureworks.blob.core.windows.net/ Products/Bikes/SuperDuperCycle.jpg Products/Bikes/FastBike.jpg Products/Canoes/Whitewater.jpg Products/Canoes/Flatwater.jpg Products/Canoes/Hybrid.jpg Products/Tents/PalaceTent.jpg Products/Tents/ShedTent.jpg GET http://.../products?comp=list&prefix=Tents&delimiter=/ <Blob>Tents/PalaceTent.wmv</Blob> <Blob>Tents/ShedTent.wmv</Blob> GET http://.../products?comp=list&delimiter=/ <BlobPrefix>Bikes</BlobPrefix> <BlobPrefix>Canoes</BlobPrefix> <BlobPrefix>Tents</BlobPrefix>

40. Two Types of Blobs Under the Hood • Block Blob • Targeted at streaming workloads • Each blob consists of a sequence of blocks • Each block is identified by a Block ID • Size limit 200GB per blob • Optimistic Concurrency via ETags • Page Blob • Targeted at random read/write workloads • Each blob consists of an array of pages • Each page is identified by its offset from the start of the blob • Size limit 1TB per blob • Optimistic or Pessimistic (locking) concurrency via Leases

41. Uploading a Block Blob THE BLOB • Uploading a large blob blobName = “TheBlob.wmv”; PutBlock(blobName, blockId1, block1Bits); PutBlock(blobName, blockId2, block2Bits); ………… PutBlock(blobName, blockIdN, blockNBits); PutBlockList(blobName, blockId1,…,blockIdN); 10 GB Movie Block Id 2 Block Id N Block Id 1 Block Id 3 • Benefit • Efficient continuation and retry • Parallel and out of order upload of blocks Windows Azure Storage TheBlob.wmv TheBlob.wmv

42. Page Blob – Random Read/Write • Create MyBlob • Specify Blob Size = 10 Gbytes • Sparse storage - Only charged for pages with data stored in them • Fixed Page Size = 512 bytes • Random Access Operations • PutPage[512, 2048) • PutPage[0, 1024) • ClearPage[512, 1536) • PutPage[2048,2560) • GetPageRange[0, 4096) returns valid data ranges: • [0,512) , [1536,2560) • GetBlob[1000, 2048) returns • All 0 for first 536 bytes • Next 512 bytes are data stored in [1536,2048) 0 512 1024 1536 2048 10 GB Address Space 2560 10 GB

43. Windows Azure Drives • Durable NTFS volume for Windows Azure Instances • Use existing NTFS APIs to access a network attached durable drive • Use System.IO from .NET • Benefits • Move existing apps using NTFS more easily to the cloud • Durability and survival of data on instance recycle • A Windows Azure Drive is a NTFS VHD Page Blob • Mounts Page Blob over the network as an NTFS drive • Local cache on instance for read operations • All flushed and unbuffered writes to drive are made durable to the Page Blob

44. Windows Azure Drive Capabilities • A Windows Azure Drive is a Page Blob formatted as a NTFS single volume Virtual Hard Drive (VHD) • Drives can be up to 1TB • A Page Blob can be mounted: • On one instance at a time for read/write • Using read-only snapshots to multiple instances at once • An instance can dynamically mount up to 16 drives • Remote Access via standard BlobUI • Can’t remotely mount drive. • Can upload the VHD to a Page Blob using the blob interface, and then mount it as a Drive • Can download the VHD to a local file and mount locally

45. Tables

46. Table Storage Concepts Account Table Entity Name =… Email = … customers Name =… EMailAdd= … contoso Photo ID =… Date =… photos Photo ID =… Date =…

47. Table Details • Not an RDBMS! • Table • Create, Query, Delete • Tables can have metadata • Entities • Insert • Update • Merge – Partial update • Replace – Update entire entity • Delete • Query

48. Entity Properties • Entity can have up to 255 properties • Up to 1MB per entity • Mandatory Properties for every entity • PartitionKey & RowKey (only indexed properties) • Uniquely identifies an entity • Defines the sort order • Timestamp • Optimistic Concurrency. Exposed as an HTTP ETag • No fixed schema for other properties • Each property is stored as a <name, typed value> pair • No schema stored for a table • Properties can be the standard .NET types • String, binary, bool, DateTime, GUID, int, int64, and double

49. No Fixed Schema Fav Sport Canoeing

50. Queues