virtual techdays

INDIA │ 22-24 november2010 virtual techdays Windows Azure Storage – Deep Dive Gaurav Mantri│ Founder, Cerebrata Software

INDIA │ 22-24 november2010 virtual techdays • Overview of Windows Azure Storage • Blob Containers & Blobs • Tables & Entities • Queues & Messages • API & Tools • Resources • Q&A S E S S I O N A G E N D A

INDIA │ 22-24 november2010 virtual techdays • Please provide feedback for the session. Again, the feedback console for this session will be live till 5:00pm today. Your feedback is considered in deciding the content for the next event. • In case you are facing any technical problems, please change your status on the top-right corner of your screen, and someone will assist you. • If you have questions in between the presentation, use the “Q&A panel” on Live Meeting console to ask your question. I will try to answer as many questions as possible. Housekeeping

INDIA │ 22-24 november2010 virtual techdays Windows Azure Storage Storage Options Blob Storage Table Storage Queue Storage SQL Azure

INDIA │ 22-24 november2010 virtual techdays • Think of it as file system in the cloud. • Similar to Amazon S3. • 2 Level hierarchy. • No limit of number of blob containers & blobs. • Size limited to the size of Storage Account (currently 100 TB). Blob Storage – Containers & Blobs Overview

INDIA │ 22-24 november2010 virtual techdays • Think of it as “Folders” in Cloud file system. • Similar to “Buckets” in Amazon S3. • Scoped by storage account. • No limit on number of blobs. • Nested blob containers not supported. • Supports metadata (up to 8K in size). • Supports ACL & Signed URL. • Typical URL: http://<myaccount>.blob.core.windows.net/<containername> • “$root” blob container: http://<myaccount>.blob.core.windows.net • Use “$root” for storing client access policy file. Blob Containers Overview

INDIA │ 22-24 november2010 virtual techdays • Create • Delete • Get/set metadata • Get/set ACL & access policies • List blobs Blob Containers Operations

INDIA │ 22-24 november2010 virtual techdays • The blob container name must be a valid DNS name, confirming to the following naming rules: • Blob container names must start with a letter or number, and can contain only letters, numbers, and the dash (-) character. • Every dash (-) character must be immediately preceded and followed by a letter or number e.g. my-blob-container. • All letters in a container name must be lowercase e.g. siteimages. • Blob container names must be from 3 through 63 characters long. • Valid blob container names: abc, myblobcontainer, videos-and-images • Invalid blob container names: a, MyBlobContainer, videos_and_images Blob Containers Naming

INDIA │ 22-24 november2010 virtual techdays • ACL & Access Policies provide a mechanism to provide additional access security on blob containers and blobs. • Account owner will always have complete control over blob container. • Provide 3 kinds of access control over blob container: • Full public read access (Container) • Public read access for blobs only (Blob) • No public read access (Private) • Up to 5 access policies can be defined for a blob container. • Provide read/write/delete/list access on blob container and blobs without giving away account credentials. • Use “Signed URL” to provide time limited access to blob container and blobs. Blob Containers ACL & Access Policies

INDIA │ 18-20 august2010 virtual techdays DEMO: Blob Container ACL

INDIA │ 22-24 november2010 virtual techdays • Ability to assign name/value pairs as blob container metadata. • Metadata size is limited to 8K. • Metadata name must be a valid C# identifier. Blob Containers Metadata

INDIA │ 22-24 november2010 virtual techdays • Think of them as “Files” in the cloud. • URI accessible e.g. http://<myaccount>.blob.core.windows.net/<blobcontainer>/myblob.txt • Scoped by blob container. • Flat hierarchy however an illusion of folder hierarchy is achievable. • Size limited by blob type and storage account capacity. • Supports metadata (up to 8K in size). • Two kinds of blobs – Block blob & Page blob. Blobs Overview

INDIA │ 22-24 november2010 virtual techdays • Create/Upload/Download • Delete • Copy • Get/Set Metadata • Snapshot • Lease Blobs Operations

INDIA │ 22-24 november2010 virtual techdays • Ideal for streaming purposes e.g. images, documents, videos etc. • Each block blob can be up to 200GB in size. • Consists of a sequence of blocks where each block can be of a maximum of 4 MB size. Blobs Block Blobs

INDIA │ 22-24 november2010 virtual techdays • Ideal for random read/write workloads. • Each page blob can be up to 1TB in size. • Consists of a sequence of pages where each page is of 512 bytes. • Pay only for the pages used. Empty pages are not charged. Blobs Page Blobs

INDIA │ 22-24 november2010 virtual techdays • Azure drive is a page blob mounted as an NTFS drive. • Provides a durable NTFS volume for Windows Azure applications. • Mounted by one VM at a time for read/write. • Contents of an Azure drive is only visible to the VM which mounted it. Blobs Azure Drive

INDIA │ 22-24 november2010 virtual techdays • Use it to store structured data in the cloud. • NOT a relational database rather it’s a NO-SQL implementation by Microsoft. • Similar to Amazon SDB. • Possible to store huge amounts of data. • No limit of number of tables & entities. • Size limited to the size of Storage Account (currently 100 TB). Table Storage – Tables & Entities Overview

INDIA │ 22-24 november2010 virtual techdays • NOT a relational table. • Think of it as a property bag where disparate data can be stored. • For example, in one table you can store Customer and Order data. • Data in a table is “Partitioned” to support massive storage. Tables Overview

INDIA │ 22-24 november2010 virtual techdays • Create • Delete • Query entities • Entity group transaction • Insert • Update – Merge/Replace Tables Operations

INDIA │ 22-24 november2010 virtual techdays • The table name must be a valid DNS name, confirming to the following naming rules: • Table names may contain only alphanumeric characters. • A table name may not begin with a numeric character. • Table names are case-insensitive. • Table names must be from 3 through 63 characters long. • Valid table names: Abc, Customer, Account1 • Invalid blob container names: A, 1Temp Tables Naming

INDIA │ 22-24 november2010 virtual techdays • NOT a row in a table. • Think of it as a property bag. • A maximum of 255 properties per entity. • Mandatory properties – PartitionKey, RowKey, and Timestamp (system generated). • Each entity can be of a maximum of 1MB in size. • Each property can be of following data type Entities Overview

INDIA │ 22-24 november2010 virtual techdays • PartitionKey & RowKey are of “String” type. • Once created, PartitionKey & RowKey are not editable. • An entity can be uniquely identified by it’s PartitionKey and RowKey value. • Timestamp value is system generated and is always in UTC. Entities PartitionKey, RowKey and Timestamp

INDIA │ 22-24 november2010 virtual techdays • Supports the query options defined by the ADO.NET Data Services Framework. • Not all query options supported. • Querying table will return all attributes of matching entities i.e. you can’t select attributes to return. • For best performance, one must always include “PartitionKey” in query criteria. • Example (PartitionKeyeq ‘MyPartitionKey’) and (RowKeyeq ‘MyRowKey’) Tables Querying tables

INDIA │ 18-20 august2010 virtual techdays DEMO: Tables & Entities

INDIA │ 22-24 november2010 virtual techdays • Use it as reliable messaging infrastructure in the cloud. • Similar to Amazon SQS. • No limit of number of queues and messages. • Set a limit for message expiration (currently a message can stay for a maximum of 7 days). Queues & Messages Overview

INDIA │ 22-24 november2010 virtual techdays • The queue name must be a valid DNS name, confirming to the following naming rules: • A queue name must start with a letter or number, and may contain only letters, numbers, and the dash (-) character. • The first and last letters in the queue name must be alphanumeric. The dash (-) character may not be the first or last letter. • All letters in a queue name must be lowercase. • A queue name must be from 3 through 63 characters long. • Valid queue names: abc, myqueue, poison-messages • Invalid queue names: Abc, MYQUEUE, -poison_messages Queues Naming

INDIA │ 22-24 november2010 virtual techdays • A message can be 8K in size. For larger messages use Blob or Table storage. • No limit on a number of messages in a queue. • Message expiration date can be specified. A message can stay in a queue for a maximum of 7 days. • Up to a maximum of 32 messages can be fetched at a time. • Peek/Get message semantics. Messages Overview

INDIA │ 22-24 november2010 virtual techdays • Azure Storage is accessible via REST API. • Microsoft has provided a .Net based storage client library which is built on top of REST API. • Open source libraries are available for Java & PHP support. API & Tools API

INDIA │ 22-24 november2010 virtual techdays • Storage can also be managed by a number of tools both from Microsoft & ISVs. • Tools range from open source to commercial. • Partial list of available tools: • Cerebrata Cloud Storage Studio - $70.00 • ClumsyLeaf Azure Explorer - $50.00 • Microsoft MMC Plugin – Free • Microsoft Visual Studio Plugin – Free • Microsft MyAzureStorage.com – Free • Azure Storage Explorer – Free • http://blogs.msdn.com/b/windowsazurestorage/archive/2010/04/17/windows-azure-storage-explorers.aspx API & Tools Tools

INDIA │ 22-24 november2010 virtual techdays • Windows Azure Home Page: • http://www.microsoft.com/azure • Windows Azure Storage Team Blog: • http://blogs.msdn.com/b/windowsazurestorage/ • Windows Azure MSDN Forums: • http://social.msdn.microsoft.com/Forums/en-US/category/windowsazureplatform • Neil Mackenzie Blog: • http://convective.wordpress.com • MSDN Documentation: • http://msdn.microsoft.com/en-us/library/dd163896.aspx Resources Useful links

THANKS│22-24 november2010 virtual techdays Questions??

THANKS│22-24 november2010 virtual techdays gmantri@cerebrata.com│ @gmantri│ www.cerebrata.com

virtual techdays