the use of microsoft based technologies for the benefit of the community n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
The Use of Microsoft based Technologies for the benefit of the Community PowerPoint Presentation
Download Presentation
The Use of Microsoft based Technologies for the benefit of the Community

Loading in 2 Seconds...

play fullscreen
1 / 76

The Use of Microsoft based Technologies for the benefit of the Community - PowerPoint PPT Presentation


  • 117 Views
  • Uploaded on

The Use of Microsoft based Technologies for the benefit of the Community. Prof. Avi Mendelson – Microsoft & Technion mendlson@cs.technion.ac.il - Taub 643 Prof. Ron Friedman – CS Technion roy@cs.technion.ac.il – Taub 605. אדמיניסטרציות . הגדרת דרישות מעבר לתכנון. תהליך גיבוש פרויקט.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'The Use of Microsoft based Technologies for the benefit of the Community' - cassia


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
the use of microsoft based technologies for the benefit of the community

The Use of Microsoft based Technologies for the benefit of the Community

Prof. Avi Mendelson – Microsoft & Technionmendlson@cs.technion.ac.il - Taub 643

Prof. Ron Friedman – CS Technionroy@cs.technion.ac.il – Taub 605

slide2
אדמיניסטרציות
  • הגדרת דרישות
  • מעבר לתכנון
slide3
תהליך גיבוש פרויקט
  • מסמך דרישות
    • המסמך הריק - ..\Assist material\General\Top_Level_Design.empty.doc
    • דוגמא למסמך מלא - ..\Assist material\General\Scriptoexample\TLD.Group6v0.03.doc
    • מסמך Top Level Design
azure
azure
  • Installation: http://www.microsoft.com/windowsazure/sdk/
what is the cloud
What is the cloud?

An approach to computing that’s about internet scale and connecting to a variety of devices and endpoints

cloud services
Cloud Services

Infrastructure-as-a-Service

“PaaS”

“IaaS”

“SaaS”

Platform-as-a-Service

Software-as-a-Service

consume

build

host

iaas paas and saas
IaaS, PaaS and SaaS
  • Infrastructure as a Service
  • Platform as a Service
  • Software as a Service

SaaS

ApplicationsPackaged Software

PaaS

PlatformOS & Application Stack

PlatformOS & Application Stack

IaaS

InfrastructureServers · Storage · Network

InfrastructureServers · Storage · Network

InfrastructureServers · Storage · Network

common factors
Common Factors
  • Pay per use
  • Instant Scalability
  • Security
  • Reliability
  • APIs
advantages
Advantages
  • Lower cost of ownership
  • Reduce infrastructure management responsibility
  • Allow for unexpected resource loads
  • Faster application rollout
what is windows azure
What Is Windows Azure?
  • It is an operating system for the cloud
introducing windows azure
Introducing Windows Azure

Windows Azure is an internet-scale cloud services platform hosted in Microsoft data centers around the world, proving a simple, reliable and powerful platform for the creation of web applications and services.

windows azure platform data centers
Windows Azure Platform Data Centers

North America Region

Europe Region

Asia Pacific Region

N. Europe

N. Central – U.S.

W. Europe

S. Central – U.S.

E. Asia

S.E. Asia

  • 6 datacenters across 3 continents

Simply select your data center of choice when deploying an application

so what is windows azure
So, What Is Windows Azure?
  • It is an operating system for the cloud
  • It is designed for utility computing
  • It provides facilities to:
    • Write your apps (developer experience)
    • Host your apps (compute)
    • Manage your apps (service management)
    • Store your data (storage)
windows azure platform service guarantee
Windows Azure Platform Service Guarantee
  • Storage servicewill be available/ reachable (connectivity)
  • Your storage requests willbe processed successfully
  • .NET Service Bus endpoint willhave external connectivity
  • Message operation requests willbe processed successfully

Serviceavailability

Storage availability

Database availability

  • Role instance monitoring and restart

Compute connectivity

Database is connected to the internet gateway

Availability monitoring every 5-minute interval

All runningroles will be continuously monitored

If role is unhealthy, we will detect and initiate corrective state

Your service is connected and reachable via web

Internet facing roles will have external connectivity

Automated Systems Management

>99.9%

>99.95%

>99.9%

>99.9%

windows azure web worker roles
Windows Azure Web & Worker Roles
  • Windows Server 2008 x64 (R2 coming soon)
  • .NET Framework – 3.5 SP1 and 4.0
  • Native Code, PHP, Java, etc. – Bring your own runtime
  • Supports Full Trust today (admin mode coming soon)
  • Fabric manages role lifecycle

Worker Role

Web Role

  • Role is hosted on IIS
  • HTTP/HTTPS
  • ASP.NET
  • Fast CGI + PHP
  • Role is an executable
    • Create your own web server, host a database, etc.
  • Inbound on
    • Any TCP Port
    • HTTP/HTTPS
roles cont
Roles - cont
  • We also have VM-Role
roles and instances
Roles and Instances
  • Roles are defined in a Service Model
    • May define one or more Roles per Service
    • A role definition specifies
      • VM size
      • Communication Endpoints
      • Local storage resources
      • Etc…
  • At runtime each Role will execute on one or more instances (up to 20 per subscription)
    • A role instance is a set of code, configuration, and local data, deployed in a dedicated VM
windows azure service architecture
Windows Azure Service Architecture

The Internet

The Internet via TCP or HTTP

Tables

LB

LB

LB

Storage

Queues

Web Site

(ASPX, ASMX, WCF)

Worker Service

Web Site

(ASPX, ASMX, WCF)

Worker Service

Web Role

IIS as Host

Worker Role

Managed Interface Call

Blobs

Windows Azure Data Center

role programming model
Role Programming Model
  • Inherits RoleEntryPoint
  • OnStart() Method
    • Called by Fabric on startup, allows you to perform initialization tasks.
    • Reports Busy status to load balancer until you return true.
  • Run() Method
    • Main logic is here – can do anything, typically infinite loop. Should never exit.
  • OnStop() Method
    • Called when role is to be shutdown, graceful exit.
    • 30 Seconds to tidy up
worker role patterns
Worker Role Patterns
  • Queue Polling Worker
    • Poll and Pop Messages within while(true) loop
    • E.g. Map/Reduce pattern, background image processing
  • Listening Worker Role
    • Create TcpListener or WCF Service Host
    • E.g. Run a .NET SMTP server or WCF Service
  • External Process Worker Role
    • OnStart or Run method executes Process.Start()
    • Startup Task installs or executes background/foreground process
    • E.g. Run a database server, web server, distributed cache
web role
Web Role
  • All features of a worker role + IIS 7 or 7.5
  • ASP.NET 3.5 SP1 or 4.0 – 64bit
  • Hosts
    • Webforms or MVC
    • FastCGI applications (e.g. PHP)
    • Multiple Websites
  • Http(s)
  • Web/Worker Hybrid
    • Can optionally implement RoleEntryPoint
service definition
Service Definition
  • Describes the shape of your Windows Azure Service
    • Defines Roles, Ports, Certificates, Configuration Settings, Startup Tasks, IIS Configuration, and more…
  • Can only be changed by upgrades or new deployments
service configuration
Service Configuration
  • Supplies Runtime Values (Scale, Config Settings, Certificates to use, VHD, etc.)
  • Can be updated any time through Portal or API
vm size in windows azure
VM Size in Windows Azure
  • Windows Azure
    • Supports Various VM Sizes
    • Size set on Role in Service Definition
    • Service can have multiple roles
    • Balance of Performance per node vs. High Availability from multiple nodes
    • Set in Service Model definitionAll instances of role will be of equal size
windows azure storage1
Windows Azure Storage
  • Scalable storage in the cloud
    • 100tb per storage account
    • Auto-scale to meet massive volume and throughput
  • Accessible via RESTful Web Service API
    • Access from Windows Azure Compute
    • Access from anywhere via internet
    • Supporting .NET Client Library
  • Various storage types
    • Table - group of entities (name/value pairs)
    • Queue - Simple non-transactional message queue
    • Blob - Large binary storage
    • Drives - NTFS VHD mounted into Compute instance
storage in the development fabric
Storage in the Development Fabric
  • Provides a local “Mock” storage
  • Emulates storage in cloud
  • Allows offline development
  • Requires SQL Express 2005/2008 or above

There are some differences between Cloud and Dev Storage.http://msdn.microsoft.com/dd320275

A good approach for developers:To test pre-deployment, push storage to the cloud first

Use Dev Fabric for compute connect to cloud hosted storage.Finally, move compute to the cloud.

blobs
Blobs

Blobs

Containers

Photo1.png

Photo2.png

Pictures

Photo3.png

blob storage concepts

Account

Container

Blob

Pages/

Blocks

Blob Storage Concepts

http://<account>.blob.core.windows.net/<container>/<blobname>

PIC01.JPG

Block/Page

images

VID1.AVI

PIC02.JPG

contoso

Block/Page

videos

blob details
Blob Details
  • Main Web Service Operations
    • PutBlob
    • GetBlob
    • DeleteBlob
    • CopyBlob
    • SnapshotBlob
    • LeaseBlob
  • Associate Metadata with Blob
    • Standard HTTP metadata/headers (Cache-Control, Content-Encoding, Content-Type, etc)
    • Metadata is <name, value> pairs, up to 8KB per blob
    • Either as part of PutBlob or independently
  • Blob always accessed by name
    • Can include ‘/‘ or other delimeter in name e.g. /<container>/myblobs/blob.jpg
blob containers
Blob Containers
  • Multiple Containers per Account
    • Special $root container
  • Blob Container
    • A container holds a set of blobs
    • Set access policies at the container level
    • Associate Metadata with Container
    • List the blobs in a container
      • Including Blob Metadata and MD5
      • NO search/query. i.e. no WHERE MetadataValue = ?
  • Blobs Throughput
    • Effectively in Partition of 1
    • Target of 60MB/s per Blob
enumerating blobs
Enumerating Blobs
  • GET Blob operation takes parameters
    • Prefix
    • Delimiter
    • Include= (snapshots, metadata etc…)

http://adventureworks.blob.core.windows.net/

Products/Bikes/SuperDuperCycle.jpg

Products/Bikes/FastBike.jpg

Products/Canoes/Whitewater.jpg

Products/Canoes/Flatwater.jpg

Products/Canoes/Hybrid.jpg

Products/Tents/PalaceTent.jpg

Products/Tents/ShedTent.jpg

GET http://.../products?comp=list&prefix=Tents&delimiter=/

<Blob>Tents/PalaceTent.wmv</Blob>

<Blob>Tents/ShedTent.wmv</Blob>

GET http://.../products?comp=list&delimiter=/

<BlobPrefix>Bikes</BlobPrefix>

<BlobPrefix>Canoes</BlobPrefix>

<BlobPrefix>Tents</BlobPrefix>

two types of blobs under the hood
Two Types of Blobs Under the Hood
  • Block Blob
    • Targeted at streaming workloads
    • Each blob consists of a sequence of blocks
      • Each block is identified by a Block ID
    • Size limit 200GB per blob
    • Optimistic Concurrency via ETags
  • Page Blob
    • Targeted at random read/write workloads
    • Each blob consists of an array of pages
      • Each page is identified by its offset from the start of the blob
    • Size limit 1TB per blob
    • Optimistic or Pessimistic (locking) concurrency via Leases
uploading a block blob
Uploading a Block Blob

THE BLOB

  • Uploading a large blob

blobName = “TheBlob.wmv”;

PutBlock(blobName, blockId1, block1Bits);

PutBlock(blobName, blockId2, block2Bits);

…………

PutBlock(blobName, blockIdN, blockNBits);

PutBlockList(blobName,

blockId1,…,blockIdN);

10 GB Movie

Block Id 2

Block Id N

Block Id 1

Block Id 3

  • Benefit
    • Efficient continuation and retry
    • Parallel and out of order upload of blocks

Windows Azure Storage

TheBlob.wmv

TheBlob.wmv

page blob random read write
Page Blob – Random Read/Write
  • Create MyBlob
    • Specify Blob Size = 10 Gbytes
    • Sparse storage - Only charged for pages with data stored in them
  • Fixed Page Size = 512 bytes
  • Random Access Operations
    • PutPage[512, 2048)
    • PutPage[0, 1024)
    • ClearPage[512, 1536)
    • PutPage[2048,2560)
  • GetPageRange[0, 4096) returns valid data ranges:
    • [0,512) , [1536,2560)
  • GetBlob[1000, 2048) returns
    • All 0 for first 536 bytes
    • Next 512 bytes are data stored in [1536,2048)

0

512

1024

1536

2048

10 GB Address Space

2560

10 GB

windows azure drives
Windows Azure Drives
  • Durable NTFS volume for Windows Azure Instances
    • Use existing NTFS APIs to access a network attached durable drive
    • Use System.IO from .NET
  • Benefits
    • Move existing apps using NTFS more easily to the cloud
    • Durability and survival of data on instance recycle
  • A Windows Azure Drive is a NTFS VHD Page Blob
    • Mounts Page Blob over the network as an NTFS drive
    • Local cache on instance for read operations
    • All flushed and unbuffered writes to drive are made durable to the Page Blob
windows azure drive capabilities
Windows Azure Drive Capabilities
  • A Windows Azure Drive is a Page Blob formatted as a NTFS single volume Virtual Hard Drive (VHD)
    • Drives can be up to 1TB
  • A Page Blob can be mounted:
    • On one instance at a time for read/write
    • Using read-only snapshots to multiple instances at once
  • An instance can dynamically mount up to 16 drives
  • Remote Access via standard BlobUI
    • Can’t remotely mount drive.
    • Can upload the VHD to a Page Blob using the blob interface, and then mount it as a Drive
    • Can download the VHD to a local file and mount locally
table storage concepts
Table Storage Concepts

Account

Table

Entity

Name =…

Email = …

customers

Name =…

EMailAdd= …

contoso

Photo ID =…

Date =…

photos

Photo ID =…

Date =…

table details
Table Details
  • Not an RDBMS!
  • Table
    • Create, Query, Delete
    • Tables can have metadata
  • Entities
    • Insert
    • Update
      • Merge – Partial update
      • Replace – Update entire entity
    • Delete
    • Query
entity properties
Entity Properties
  • Entity can have up to 255 properties
      • Up to 1MB per entity
  • Mandatory Properties for every entity
      • PartitionKey & RowKey (only indexed properties)
      • Uniquely identifies an entity
      • Defines the sort order
      • Timestamp
      • Optimistic Concurrency. Exposed as an HTTP ETag
  • No fixed schema for other properties
      • Each property is stored as a <name, typed value> pair
      • No schema stored for a table
      • Properties can be the standard .NET types
      • String, binary, bool, DateTime, GUID, int, int64, and double
no fixed schema
No Fixed Schema

Fav Sport

Canoeing

queue storage concepts
Queue Storage Concepts

Message

Queue

Account

customer ID order ID http://…

order processing

adventureworks

customer ID order ID http://…

loosely coupled workflow with queues
Loosely Coupled Workflow with Queues
  • Enables workflow between roles
    • Load work in a queue
      • Producer can forget about message once it is in queue
    • Many workers consume the queue
    • For extreme throughput (>500 tps)
      • Use multiple queues
      • Read messages in batches
      • Multiple work items per message

Input Queue (Work Items)

Queue

Worker Role

Web Role

Worker Role

Web Role

Worker Role

Web Role

Worker Role

what is a queue
What is a queue?
  • Queues are used to store messages
  • They are FIFO, and one way
  • A queue name must be lower case name, and URL friendly
messages
Messages
  • A queue can hold an unlimited number of messages
  • Messages must be serializable as XML
  • Limited to 8KB in size
  • Commonly use the work ticket pattern
queue details
Queue Details
  • Simple asynchronous dispatch queue
    • No limit to queue length subject to storage limit
    • 8kb per message
    • ListQueues- List queues in account
  • Queue operations
    • CreateQueue
    • DeleteQueue
    • Get/Set Metadata
    • Clear Messages
  • Message operations
    • PutMessage– Reads message and hides for time period
    • GetMessages – Reads one or more messages and hides them
    • PeekMessages – Reads one or more messages w/o hiding them
    • DeleteMessage – Permanently deletes messages from queue
queue s reliable delivery
Queue’s Reliable Delivery
  • Guarantee delivery/processing of messages (two-step consumption)
    • Worker Dequeues message and it is marked as Invisible for a specified “Invisibility Time”
    • Worker Deletes message when finished processing
    • If Worker role crashes, message becomes visible for another Worker to process
    • More on this pattern in the Async Workloads session

Worker Role

Web Role

Input Queue (Work Items)

Worker Role

Queue

Web Role

Worker Role

Web Role

Worker Role

message lifecycle
Message lifecycle

RemoveMessage

GetMessage (Timeout)

HTTP/1.1 200 OK

Transfer-Encoding: chunked

Content-Type: application/xml

Date: Tue, 09 Dec 2008 21:04:30 GMT

Server: Nephos Queue Service Version 1.0 Microsoft-HTTPAPI/2.0

<?xml version="1.0" encoding="utf-8"?>

<QueueMessagesList>

<QueueMessage>

<MessageId>5974b586-0df3-4e2d-ad0c-18e3892bfca2</MessageId>

<InsertionTime>Mon, 22 Sep 2008 23:29:20 GMT</InsertionTime>

<ExpirationTime>Mon, 29 Sep 2008 23:29:20 GMT</ExpirationTime>

<PopReceipt>YzQ4Yzg1MDIGM0MDFiZDAwYzEw</PopReceipt>

<TimeNextVisible>Tue, 23 Sep 2008 05:29:20GMT</TimeNextVisible>

<MessageText>PHRlc3Q+dG...dGVzdD4=</MessageText>

</QueueMessage>

</QueueMessagesList>

Worker Role

PutMessage

Queue

Msg 1

Msg 2

Msg 2

Msg 1

Web Role

POST http://myaccount.queue.core.windows.net/myqueue/messages

DELETE

http://myaccount.queue.core.windows.net/myqueue/messages/messageid?popreceipt=YzQ4Yzg1MDIGM0MDFiZDAwYzEw

Worker Role

Worker Role

Msg 3

Msg 4

Msg 2

working with queues
Working with Queues
  • Create a CloudQueueClient
peek a message
Peek a message
  • Peeking lets you see the contents without ‘getting’ the message.
  • Handy when trying to determine how to handle a message without consuming it.
get a message
Get a message
  • The visibility is 30 seconds and can be as long as 2 hours.
deleting a message
Deleting a Message
  • The Message Id and the Pop Receipt must match for a successful delete.
polling
Polling
  • Most queues are polled in an infinite loop
  • Make your processes idempotent
sql azure
SQL Azure

Extending SQL Server to the Cloud

Reporting

Database

Data Sync

sql azure database the first and only true database as a service
SQL Azure DatabaseThe first and only true database as a service

Subtitle color

Managed Service

Scale On Demand

Innovate Faster

  • Build cloud-based database solutions on consistent relational model
  • Leverage existing skills through existing ecosystem of developer and management tools
  • Database utility; pay as you grow
  • Business-ready SLAs
  • Enable multi-tenant solutions
  • World-wide presence
  • Easy provisioning and deployment
  • Auto high-availability and fault tolerance
  • No need for server or VM administration
windows azure appfabric
Windows Azure AppFabric

Building Block Services for developing connected applications

Access Control

Service Bus

Caching

how to get started
How to Get Started?
  • Go to http://www.microsoft.com/azure
    • Sign up for a cloud
    • Download the SDK
  • Community involvement
    • Collaborate online
    • Find/contribute cloud apps on CodePlex
links
Links
  • Windows Azure Developer Platformhttp://www.microsoft.com/windowsazure/developers/
  • Channel 9 videoshttp://channel9.msdn.com/azure
  • Windows Azure Platform Training Kithttp://www.azure.com
  • Microsoft PDC Conferencehttp://microsoftpdc.com/
  • Windows Azure SDKhttp://www.microsoft.com/windowsazure/developers/
windows azure november 2010 update
Windows Azure November 2010 Update
  • Windows Azure SDK & Tools 1.3
  • New Windows Azure Platform Management Portal
  • Remote Desktop
  • Multiple Service Administrators
  • Windows Server 2008 R2 & IIS 7.5
  • Full IIS
  • Elevated Privileges (Admin mode)
  • Windows Azure Virtual Machine Role
  • Windows Azure Connect (formerly “Sydney”)
  • Extra Small Virtual Machine Instances
resources starting point
Resources (starting point)
  • Kinect
    • Start at: http://research.microsoft.com/kinectsdk
    • Demo: http://www.xbox.com/en-GB/Kinect/Kinect-Effect
    • OpenNI - http://www.openni.org/
  • C#
    • C# book 2011 - http://www.robmiles.com/c-yellow-book/
  • WP7
    • Starting point- HUB- http://create.msdn.com/en-US/
    • Downloads: http://create.msdn.com/en-us/home/getting_started
    • Windows Phone 7 Development for Absolute Beginners - http://channel9.msdn.com/Series/Windows-Phone-7-Development-for-Absolute-Beginners
    • Mango Jumpstart Videos - http://create.msdn.com/en-us/news/mango-jumpstart-videos
    • WP7 (Mango) book 2011 - http://www.robmiles.com/c-yellow-book/
    • Charles Petzold Book http://www.charlespetzold.com/phone/
future classes
Future classes
  • AZURE and Cloud services
  • XNA
  • SQL and data related services