Project "Velocity" - A First Look

1. TL14 Project "Velocity"- A First Look  Muralidhar Krishnaprasad Principal Architect Microsoft Corporation

2. This session.. • Distributed Caching • Types of Cached data • What is Velocity? • When can I use it? • How does it help my application? • Performance • Scale • High Availability • Features of Velocity

3. Most composite applications designed for serious scale and availability are using mid-tier, distributed caching technologies today.

4. Caching Reference Data Scenario: Social Networking Clients Web Tier Distributed Cache Servers Local Cache (in Proc) Usernames, Name-> ID Mapping Data Tier Friend Lists Usernames

5. Caching Activity-Oriented Data Clients Scenario: Enterprise LOB Application Web Tier Application Tier Integration Hub Distributed Cache Aggregated Vendor Catalogs Order, Invoice, Payment Data Tier Vendor services Pricing Vendor Sources

6. Caching Resource-Oriented Data Scenario: Flight Inventory and Pricing Booking Service Application Tier Flight Itinerary Seat Inventory Distributed Cache Flight Segment Flight Price American Airlines

7. Sample Application Inventory Shopping Cart Catalog

8. What Is "Velocity"? • An explicit, distributed, in-memory application cache for all kinds of data (CLR objects, rows, XML, Binary data etc.) • Fuse "memory" across machines into a unified cache Clients can be spread across machines or processes Clients Access the Cache as if it was a large single cache Unified Cache View Cache Layer distributes data across the various cache nodes

9. Where Does It Fit? Users Application / Web Tier … Application Application Velocity ClientLocal Cache Velocity ClientLocal Cache Application Velocity ClientLocal Cache Cache Tier Velocity Service Velocity Service Velocity Service Server 2 Server 1 Server 3 Server Side Callbacks Data Tier Database Cloud

10. Why Velocity? Make Application Available Protect from machine failures Scale Out Add More Machines to Scale out your Application Increase Performance Cache Objects close to Application Do all this at low cost Use Cheap Commodity Hardware Velocity V1 will be FREE out of band release for the .Net framework

11. When Can I Get Velocity? • CTP1 • Teched • CTP2 • Now • CTP3 • Mix 2009 • RTM • Mid 2009

12. Velocity Concepts

13. Velocity Logical Hierarchy • Host • Physical processes hosting Velocity instance. • Named Caches • Can span across machines • Defined in the configuration file • Regions • Physically co-located Container of Cache Items • May be implicit or explicitly created • Cache Item • Key, Payload (Object ), Tags, TTL, Timestamps, Version Velocity Service Velocity Service Velocity Service Velocity Service Named Cache : Product Catalog Named Cache : Electronics Inventory Regions Key Payload Tags Region A 121 xxxx “Toy” “Child” 123 yyyy “Toy” “Chair”.. • Machine -> Cache Host -> Named Caches -> Regions -> Cache Items -> Objects

14. Example Usage Of Velocity API // Create instance of cachefactory (reads appconfig) CacheFactoryfac = new CacheFactory(); // Get a named cache from the factory Cache catalog = fac.GetCache("catalogcache"); // Simple Get/Put catalog.Put("toy-101", new Toy("Puzzle", .,.)); // From the same or a different client Toy toyObj = (Toy)catalog.Get("toy-101"); // Region based Get/Put catalog.CreateRegion("toyRegion"); // Both toy and toyparts are put in the same region catalog.Put("toyRegion", "toy-101", new Toy( .,.)); Catalog.Put("toyRegion", "toypart-100", new ToyParts(…)); Toy toyObj = (Toy)catalog.Get("toyRegion", "toy-101");

15. demo Install Demo

16. Velocity V1 Features • Simple Access Patterns – Get/Add/Put/Remove • Tag Searching • Partitioned & Local Client Cache • High Availability • Multiple Concurrency Control Options • ASP.Net Integration • Management & Monitoring • Security • Cache Event Notifications • Server Side Callbacks Post CTP2

17. Access APIs – Tagging Items • Hot or Discount Items in catalog • Use Tagging • List All Items / Hot Items - Enumeration • Use Enumeration // Add Tags Tag hotItem = new Tag("hotItem"); Tag discItem = new Tag("discountItem"); catalog.Put("toyRegion", "toy-101", new Toy("Puzzle"), new Tag[]{hotItem}); catalog.Put("toyRegion", "toy-102", new Toy("Bridge"), new Tag[]{hotItem,discItem}); // From the same or a different client List<KeyValuePair<string, object>> toys = catalog.GetAnyMatchingTag("toyRegion", hotItem);

18. Cache Types • Scale the Catalog and Shopping Cart Partitioned Cache • Data Partitioned across all nodes in the named cache • Backups for shopping cart for Availability • Client Side Local Cache for Higher Performance • Use Local Cache in client to speed up catalog access • Payload kept in object form • Synchronization using expiry or notifications *- Post CTP2

19. Partitioned Cache Users Application Velocity ClientLocal Cache Application / Web Tier … Application Application Velocity ClientLocal Cache Velocity ClientLocal Cache G H I D E F A B C Cache Tier Velocity Service Velocity Service Velocity Service Server 2 Server 1 Server 3

20. Partitioning Increases Scale And Performance • Scale on Data Size • More machines => More memory to cache • Scale on Cache Throughput • Cache Throughput: Operations/sec from entire cache • More machines => keys distributed across more machines => better throughput • Performance • High Performance by scaling out data and processing • Increased memory • Increased processing • Workload distributed across multiple cache nodes

21. demo Scale Demo

22. Basic Layout Load Controller Spawns Test Clients Load Test Controller … Client Machines run test Cache Client n Cache Client 2 Cache Client1 Cache Service Cluster Server 2 Server 3 Server 1 PartitionedCache

23. Demo Output Server 3 Added Throughput Increases Latency Decreases Load Server 2 Added Throughput Increases Latency Decreases Until Server Saturation Single Server Throughput Increases with Increasing Load Until Server Saturation Throughput Latency

24. Partitioned Cache + HA Option Users Application Velocity ClientLocal Cache Application / Web Tier … Application Application Velocity ClientLocal Cache Velocity ClientLocal Cache G H I D E F A B C Cache Tier D I Velocity Service H G B C Velocity Service A Velocity Service Server 2 Server 1 F Server 3 E

25. HA Option – Failure Scenario Users Application Velocity ClientLocal Cache Application / Web Tier … Application Application Velocity ClientLocal Cache Velocity ClientLocal Cache Cache Tier D H C C I B G Velocity Service A Velocity Service F F Velocity Service E E Server 2 Server 1 Server 3 B A A F E H H D D G C I

26. Session Cart Integration With ASP.Net Load Balance Requests No more sticky routing Application Velocity SessionStoreProvider <sessionState mode="Custom" customProvider="SessionStoreProvider"> <providers> <add name="SessionStoreProvider" type="System.Data.Caching.SessionStoreProvider, ClientLibrary" cacheName="<YourNamedCache>"/> </providers> </sessionState> Drop in Velocity SessionStoreProvider … Application Application Velocity SessionStoreProvider Velocity SessionStoreProvider Session State stored in Velocity Scale your Session Store Dynamically Velocity Service Velocity Service Velocity Service Highly Available

27. demo Availability Demo

28. Availability Demo Cache Cluster Velocity Embedded in ASP.Net Application CacheViewer Application Northwind Application Northwind Application ASP.net Session Cache ASP.net Session Cache VELOCITY VELOCITY VELOCITY Velocity Session Store Provider Velocity Session Store Provider Partitioned Cache

29. Administration And Monitoring • Installation • MSI to install on each box • Silent/Remote Install • Logging • Provider model; Default ETW, support for file logs • Administration • PowershellCommandlets • Start and stop cluster or service, configure named caches .. • Monitor Cache statistics • Perfmon integration * Post CTP2

30. Configuration Store • Used to store • Dynamic Global Partitioning Map • Enables Global Partition Manager to restart fast • Velocity Configuration & Policies • Enables global change of policies • Options - Network Share or SQL Server Server 3 Server 2 Server 1 Velocity Service Velocity Service Cache Servers Velocity Service Global Partition Manager One of the Velocity Service Hosts the Global Partition Manager Configuration Store (Can be database , File share etc.) Stores Named Cache Policies Stores Global Partitioning Map

31. Putting It All Together

32. Caching Reference Data Scenario: Social Networking Partitioned Cache Scale Clients Web Tier Velocity Cache Service ASP.Net Application Farm Velocity Local Cache Usernames, Name-> ID Mapping Data Tier Local Client Cache Higher Performance Friend Lists Usernames

33. Caching Activity-Oriented Data Scenario: Enterprise LOB Application Clients ASP.Net Integration Session Load Balancing Web Tier Partitioned Cache + HA Scale + Availability Application Tier ASP.Net Application Farm Session Store Provider Integration Hub Velocity (with HA on) Aggregated Vendor Catalogs Order, Invoice, Payment Data Tier Vendor services Pricing Vendor Sources

34. Caching Resource-Oriented Data Scenario: Flight Inventory and Pricing Partitioned Cache + HA + Concurrency Control Scale + Availability + Performance Booking Service Application Tier Flight Itinerary Seat Inventory Velocity Flight Segment Flight Price American Airlines

35. Looking Just Ahead

36. More PerformancePost CTP2 • CTP2 delivers • 22K Ops per second on Quad Core for 4K sized Objects • @ 0.3 – 0.5ms per access without serialization (byte[]) • @ around 2 ms with serialization overheads • Session Store • 50K sessions stored and accessed • Same as SQLServer temporary table performance • Bulk Access APIs • Bulk fetch or puts to improve throughput

37. SecurityPost CTP2 • Open Security Option • No security enforced • Token Based Security Option • Register Simple Token Id at named cache level • Application must call GetCache() with that token • App-id Based Security Option • Domain Account based Authentication • Register the domain account with the named cache • Transport Level Security • Use Standard WCF security options • E.g., Encryption

38. June 2008 (CTP1)(TechEd 2008) Schedule • Oct 2008 (CTP2)(PDC 2008) • CTP3 • (Mix 2009) • RTM • Mid 2009

39. Recap Velocity Http://msdn.microsoft.com/data

40. Evals & Recordings Please fill out your evaluation for this session at: This session will be available as a recording at: www.microsoftpdc.com

41. Q&A Please use the microphones provided

42. © 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.