Windows Azure Storage: Building applications that scale

1. Windows Azure Storage: Building applications that scale Jai Haridas & Joe Giardino 4-004

2. Agenda • Introduction • Scalability Targets • Best Practices • Demo – How to build scalable apps? • Questions

3. Introduction

4. Windows Azure Storage Abstractions • Blobs: Store files and metadata associated with it • Queues: Durable asynchronous messaging system to decouple components • Tables: A strongly consistent NoSQL structured store that auto scales • Drives and Disks: Network mounted durable drives available for applications in the cloud

5. Windows Azure Storage Characteristics • A “pay for what you use” cloud storage system • Durable: Store multiple replicas of your data • Local replication: • Synchronous replication before returning success • Geo replication: • Replicated to data center at least 400+ miles apart • Asynchronous replication after returning success to user. • Available: Multiple replicas are placed to provide fault tolerance • Scalable: Automatically partitions data across servers to meet traffic demands • Strong consistency: Default behavior is consistent reads once data is committed

6. Windows Azure Storage Characteristics • All abstractions backed by same store • Same feature set across all abstractions (geo, durability, strong consistency, auto scale, monitoring, partitioning logic etc.) • Reduce costs by blending different characteristics of each abstraction • 880K requests/s at peak & 4+ Trillion objects • Great performance for low transaction costs! • Easy to use and open REST APIs • Client libraries in Java, Node.js, PHP, .NET etc.

7. Windows Azure Storage – How is it used? Xbox: Uses Windows Azure Blobs, Tables & Queues for applications like Cloud Game Saves, Halo multiplayer, Music, Kinect data collection etc. SkyDrive: Uses Windows Azure Blobs to store pictures, documents etc. Bing:Uses Windows Azure Blobs, Tables and Queues to implement an ingestion engine that consumes Twitter and Facebook public status feeds and provides it to Bing search And many more…

8. Running on Windows Azure Storage Bing Realtimefacebook/twitter search ingestion engine Bing Ingestion Engine (Azure Service) • Facebook/Twitter data stored into blobs • Ingestion engine process blobs • Annotate with auth/spam/adult scores, content classification , expands links, etc • Uses Tables heavily for indexing • Queues to manage work flow • Results stored back into blobs • Bing takes resulting blobs and folds into search index Index Facebook/Twitter data within 15 seconds of update VM VM VM VM User postings Status updates ………… Windows Azure Blobs Windows Azure Queues Windows Azure Tables peak 40,000 Requests/sec 2~3 billion Requests per day Took 1 dev 2 months to design, build and release to production

9. Windows Azure Storage North America Region Europe Region Asia Pacific Region N. Central – U.S. Sub-region N. Europe Sub-region West – U.S. Sub-region W. Europe Sub-region East – U.S. Sub-region E. AsiaSub-region S. Central – U.S. Sub-region S.E. Asia Sub-region Major datacenter CDN PoPs

10. Scalability Targets

11. Windows Azure Flat Network Storage Flat networkstorage design “Quantum 10” network Non-blocking 10Gbpsbased fully meshed network Move to software based Load Balancer Provides an aggregate backplane in excess of 50 Tbpsbandwidth per Datacenter Enables high bandwidth scenarios such as Windows Azure IaaS disks, HPC, Map Reduce etc.

12. Scalability Targets -Storage Account • Storage Account level targets by end of 2012 • Applies to accounts created after June 7th 2012 • Capacity – Up to 200 TBs • Transactions– Up to 20,000 entities/messages/blobs per second • Bandwidthfor a Geo Redundant storage account • Ingress - up to 5 Gibps • Egress - up to 10 Gibps • Bandwidth for a Locally Redundant storage account • Ingress - up to 10 Gibps • Egress - up to 15 Gibps

13. Scalability Targets – Partition • Partition level Targets by end of 2012 • Applies to accounts created after June 7th2012 • Single Queue – Account Name + Queue Name • Up to 2,000 messages per second   • Single Table Partition – Account Name + Table Name + PartitionKey value • Up to 2,000 entities per second   • Single Blob– Account Name + Container Name + Blob Name • Up to 60 Mibps

14. Best Practices

15. Common Design & Scalability • Common Settings Turn off Nagling & Expect 100 (.NET – ServicePointManager) Set connection limit (.NET – ServicePointManager.DefaultConnectionLimit) Turn off Proxy detection when running in cloud (.NET – Config: autodetect setting in proxy element) • Design you application that allows distributing requestsacross your range of partition keys to avoid hotspots • Avoid Append/Prepend pattern: Access pattern lexically sorted by Partition Key • values • Perform one time operations at startup rather than every request • Creating containers/tables/queues which should always exist • Setting required constant ACLs on container/table/queue

16. Common Design & Scalability • Turn on analytics & take control of your investigations– Logging and Metrics • Who deleted my container? – Look at the client IP for delete container request • Why is my request latency increased? - Look at E2E vs. Server latency • What is my user demographics? – Use client request id to trace requests & client IP • How can I tune my service usage? – Use metrics to analyze API usage & peak traffic stats • And many more… • Use appropriate retry policy for intermittent errors • Storage client uses exponential retry by default

17. Storage Accounts • Collocate storage accountswith your compute roles as egress is free within same region • Use multiple storage accountsto: • achieve targets that exceed a single storage • achieve client proximity • Map multiple clients to same storage account Use different containers/tables/queues instead an account for each customer

18. Storage Accounts • Design to add more accounts as needed • Use different account for Windows Azure Diagnostics • Choose local redundant storageif • Data can be restored on major disasters • Geographical boundary constraints on where data can be stored

19. Windows Azure Blobs - Scalability • How to upload a single large blob as fast as possible? • Use parallel block upload and then commit • Storage client library – CloudBlobClient’sSingleBlobUploadThresholdInBytes and ParallelOperationThreadCount • How to upload multiple blobs as fast as possible? • Use single thread for each blob but upload multiple blobs in parallel • Future Service Update:Accommodate better “Append/Prepend” blob writes

20. Windows Azure Blobs - Scalability • How to migrate/backup blobs to a different account? • Use Async Copy using REST version 2012-02-12 and above • Pipeline all copies at once and then poll for completion • In partial reads scenarios, use 64KB block size when primary scenario is to read small chunks

21. Table Design & Scalability • Critical Queries:Select PartitionKey, RowKey to avoid hotspots • Batch Processing:Entities that need to be updated together will have same PartitionKey • Schema-less:Store multiple types in same table • Single Index – {PartitionKey, RowKey}: If needed, concatenate columns to form composite keys • Entity Locality: {PartitionKey, RowKey} determines sort order Store related entites togetherto reduce IO and improve performance

22. Table Design & Scalability • Avoid Large Number of Scans: Cost depends on selectivity - number of entities that server should read vs. returned in result set • Store data in required pivots in same partition i.e. Partition Level Index • Build index table to provide eventually consistent index i.e. Cross Partition Level • Index • Cache relatively constant and frequently scanned result sets • Low latency requirement: • If scenario allows, use async pattern to queue up request when online operation is expensive or fails • Do not reuse DataServiceContext/TableServiceContext across logical operations

23. Queue Design & Scalability • Make message processing idempotent • Use “Update Message” API to save intermittent processing state • Extend visibility time based on message. Allows processors to set smaller lease times • Save interim processing state • Use “Message Count” to scale workers • Use “Dequeue Count” on message to handle poison messages • Batch Get - increase message processing throughput • Use blobs to store messages that exceed 64KB or to increase throughput • Use multiple queues to scale beyond the published targets

24. Shared Access Signatures (SAS) • Use HTTPS to securely use/transport SAS tokens • Use minimum permissionsneeded and restrict time period for access • Clock Skew - Clients should renew SAS token with sufficient leeway • Revocable SAS- Use policy to store SAS permissions, expiry etc. • Only 5 policies can be associated with container • Change policy IDs when removing and recreating policies • REST version 2012-02-12 allows expiry > 1 hour even without using revocable SAS

25. How to build scalable apps? - Social Graffiti Demo A Windows 8 App & Windows Azure Service - Scales using Tables and Queues

26. Social Graffiti • Windows 8 App allows users to select a wall • Wall has tiles that allows users to draw • Users can zoom into any tile and draw shapes • Users can see what other users are drawing at the same time • Goals • Allow a single wall to be used by many users and share graffiti • Number of active walls is not a concern as service controls its creation

27. Social Graffiti Demo - Demo

28. Design Choices - #1 • Graffiti Service will need to scale like Windows Azure Storage Service • Need to accommodate large throughput • Latency overhead as all calls routed via service Windows Azure Storage Service Read/Write Read/Write Graffiti Service Devices

29. Design Choices - #2 • Graffiti Service scales as it is only responsible for handing SAS • Devices read/write directly to Windows Azure Storage service using SAS – no routing overhead • Frequent table scans to retrieve all shapes drawn by other users – scalability targets may exceed & scans should be avoided Get SAS Windows Azure Storage Service SAS Read/Write Graffiti Service Devices

30. Design Choices - #3 • Graffiti Service scales as it is only responsible for handing SAS and cached result of shapes • Devices write directly to Windows Azure Storage service using SAS – no routing overhead • Checkpoint workers coalesces shapesinto a single record cached by service – reduces the number of records to scan • Devices read from cache from Graffiti service which caches to reduce scans Get SAS & Shapes to show Cache Changes SAS Write Windows Azure Storage Service Checkpoint changes to image Checkpoint Workers Devices Graffiti Service

31. Data Model • Master Wall Table • List of available walls to draw onPartitionKey = Ticks i.e. Append only pattern but since we do not require high request throughput, this works • Wall Table • Each wall gets its own table to store all the shapes drawn and store checkpointed records • ParitionKey = Tile Location i.e. x, y, z coordinates on wall • Stores checkpoint and individual shape records. RowKey prefix determines type • Dirty Tile Table • Each tile in a wall gets a row to mark it as dirty which the checkpoint worker role monitors to checkpoint • PartitionKey = Tile Location i.e. x, y, z coordinates on wall • RowKey = Wall Table name

32. Windows Azure Table Client - Service Layer • Option 1 – WCF Data Services • Good for fixed schema used like relational tables • Do not require control on serialization/deserialization • Option 2 – Table Service Layer’s Dynamic Table Entity • Entity containing a Dictionary of Key-Value properties • Used when schema is not known example: Explorers • Performance! • Option 3 – Table Service Layer’s POCO • POCO derives from ITableServiceEntityor TableServiceEntity • Control over serialization and deserialization – make your data dance to your tune! • ETag maintained with Entities - easy to update! • Performance!

33. Work Flow - Device • 1 - Device asks Graffiti service a list of walls • 2- Once user selects a wall, device sends request to service to indicate interest in a wall • Service returns SAS for Wall & Dirty Tile tables for a restricted range of PartitionKey • 3 - Device also upserts a record in Dirty Tile table to indicate that it is dirty. Note – all users working on same tile will have only one record in Dirty Tile table • 4- When user draws, device caches it and then sends a batch request to insert into Wall table 2 1 Dirty Tile Table 3 4 Wall Table Devices Graffiti Service

34. Social Graffiti Demo - Code

35. Work Flow - Checkpoint • 1 - A master checkpoint worker queries the Dirty table records to retrieve all tiles that are dirty • 2- It queues up work for checkpoint workers to checkpoint dirty tiles across all walls • Uses multiple queues to scale out • 3 – Checkpoint worker will process the message in which it • a) Retrieves all shape records for a tile • b) Generates an image object for the shapes it retrieved and stores it as checkpoint record • c) Deletes unconditionally all the shape records retrieved and previous checkpoint record • d) Deletes the dirty indicator from Dirty Table only if Etag matches 2 Dirty Tile Table 1 Wall Table 3 Checkpoint Workers Checkpoint Queues

36. Social Graffiti Demo – Code & Analytics

37. Performance with Storage Client Library 2.0 • Faster NoSQL table access • Upto72.06% reduction in execution time • Upto31.92% reduction in processor time • Upto69-90% reduction in latency

38. Performance with Storage Client Library 2.0 • Faster uploads and downloads • 31.46% reduction in processor time • Upto22.07% reduction in latency

39. Future Features • JSONfor Tables • CORS support for Blobs, Tables and Queues • Content-Dispositionheader for blobs • Queue batching for PUT & DELETE • Provide Geo failover control to user • Windows Phone library

40. Questions?

41. Resources • Storage team blogs @ http://blogs.msdn.com/b/windowsazurestorage/ • Getting Started @ https://www.windowsazure.com/en-us/develop/overview/ • Pricing information @ https://www.windowsazure.com/en-us/pricing/details/

42. Resources • Follow us on Twitter @WindowsAzure • Get Started: www.windowsazure.com/build Please submit session evals on the Build Windows 8 App or at http://aka.ms/BuildSessions