1 / 55

Load Balancing and Scaling your WCF Services Today and Tomorrow

THE REGION: my code is 805. Load Balancing and Scaling your WCF Services Today and Tomorrow. Michele Leroux Bustamante, Chief Architect IDesign SOA309. Michele Leroux Bustamante. Chief Architect IDesign: www.idesign.net. Microsoft Regional Director, MVP Connected Systems.

grant
Download Presentation

Load Balancing and Scaling your WCF Services Today and Tomorrow

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. THE REGION: my code is 805 Load Balancing and Scaling your WCF Services Today and Tomorrow Michele Leroux Bustamante, Chief ArchitectIDesign SOA309

  2. Michele Leroux Bustamante Chief ArchitectIDesign: www.idesign.net Microsoft Regional Director, MVPConnected Systems Published Author:ASP.NET Pro (WCF Column), MSDN, CoDeLearning WCF (O’Reilly 2007/2008/2009) Speaker:Tech Ed, PDC, Dev Connections, other international events Blogs:www.dasblonde.net (main)www.thatindigogirl.com (book)

  3. Agenda • Scalability, throughput, and performance • Hosting and threading model • Throttling requests • Asynchronous models • Channel creation overhead and caching options • Session expiry • Number of service hops • Load balancing and failover • SSL processing routers • Related “Dublin” features

  4. Performance, Throughput, and Scalability • Performance: Time to complete a request • Execution time, response time • Throughput: Concurrent request processing • Requests per unit of time (usually/second) • Scalability: Ability to handle overall system load • Ability to perform as resources are added • Influenced by: • Server resources • Communication overhead • Application design, server affinity

  5. WCF Considerations • Hosting and threading model • Throttling requests • Asynchronous models • Channel creation overhead and caching options • Session expiry • Number of service hops • Load balancing and failover • SSL processing routers

  6. Server Hosting Model • Windows Server 2003 • IIS 6 for HTTP services • Windows Services for non-HTTP • Windows Server 2008 • IIS 7/Windows Process Activation Service (WAS) for all protocols • Windows Services may also be used

  7. Hosting Features Dublin improves these aspects significantly

  8. Windows Services Threading Model WCF IO Thread Windows Service • Requests allocated to WCF thread • Concurrent threads controlled by throttling behavior App Domain Service

  9. ServiceThrottleBehavior • Behavior setting per service type • MaxConcurrentCalls:Limits concurrent requests (16) • MaxConcurrentInstances: Limits service instances (int.MaxValue) • MaxConcurrentSessions: Limits active transport sessions (10)

  10. demo Service Throttling

  11. IIS 6 Threading Model Application Pools Worker Process (w3wp.exe) Worker Process (w3wp.exe) Worker Process (w3wp.exe) App Domain App Domain App Domain Service Service Service HttpHandler HttpHandler HttpHandler WWWService HttpModule HttpModule HttpModule Aspnet_isapi.dll Aspnet_isapi.dll Aspnet_isapi.dll User Mode Kernel Mode Kernel Level Queue http.sys

  12. IIS 6 Threading Model WCF IO Thread • IIS IO thread released for CLR thread • CLR thread held until WCF thread completes • Concurrent threads controlled by WCF and ASP.NET throttling Worker Process (w3wp.exe) App Domain App Domain CLR Thread App Level Queue Service Service HttpModule HttpModule User Mode Kernel Mode http.sys IIS IO Thread

  13. IIS 6 Threading Model WCF IO Thread • No concurrent request throttling • Limit requests by application queue Worker Process (w3wp.exe) App Domain CLR Thread Service App Level Queue HttpHandler (maxWorkerThreads*CPU)-minWorkerThreads HttpModule User Mode Kernel Mode http.sys IIS IO Thread

  14. IIS 6 Threading Model WCF IO Thread • Can rely on autoConfig=true (since .NET 2.0) • Increase value for IO-intensive applications Worker Process (w3wp.exe) App Domain CLR Thread Service App Level Queue HttpHandler HttpModule User Mode Kernel Mode http.sys IIS IO Thread

  15. IIS 7 Threading Model (HTTP) WCF IO Thread • Process-wide request queue now supports throttling concurrent requests • Less overhead than application queue, no managed code/memory Worker Process (w3wp.exe) App Domain App Domain CLR Thread Service Service HttpHandler HttpHandler HttpModule HttpModule Process-wide request queue User Mode Kernel Mode http.sys IIS IO Thread

  16. IIS 7 Threading Model (HTTP) • Defaults to 12 • Set to 0 to use IIS IO thread • Increase for IO intensive applications Worker Process (w3wp.exe) App Domain App Domain Service Service HttpHandler HttpHandler HttpModule HttpModule Process-wide request queue User Mode Kernel Mode http.sys IIS IO Thread

  17. IIS 7 / WAS Processing Model Worker Process (w3wp.exe) App Domain HttpHandler TcpProcess ProtocolHandler WindowsActivationService(WAS) Protocol Handlers NamedPipeProcess ProtocolHandler MsmqProcess ProtocolHandler HttpModule w3svc inpas itcpas imsmqas Listener Adapters HTTP Listener Adapter Named Pipe Listener Adapter Tcp Listener Adapter Msmq Listener Adapter Protocol Listeners http.sys net.pipe net.tcp net.msmq HTTP Named Pipe TCP MSMQ

  18. Asynchronous Operations • WCF supports an asynchronous pattern for service operations • Required asynchronous module and handler • Synchronous operations are invoked if present • Begin operation should behave asynchronously

  19. Asynchronous Pipeline • By default ASP.NET throttles requests to WCF via thread pool or concurrent request limits • Synchronous HttpModule and HttpHandler • Best configuration for IIS 6 • Can configure asynchronous pipeline • ServiceHttpModule and ServiceHttpHandler/Factory • Rely on ASP.NET request throttle in IIS 7 • Optimize calls to WCF service with IO-intensive and high latency operations • Set up handler with WcfAsyncWebUtil.exe

  20. Channel Overhead • Channel construction takes time • Affects client-server architecture where multithreading applies • Caching channel can help • Primarily affects server-server architecture • Caching channel factory can help • May not be able to, depending on factoryinitialization requirements

  21. Client-Server Architecture • Can return to the same machine for session, and scale • Can use the same client channel • Cache the channel factory and channel (or, the proxy) • Proxy must handle faulted channels andrecreate session • May be a new machine

  22. Multithreaded Clients • Cache channel factory and channel • Reuse the same proxy for multiple threads • Improves client-side performance • Requires services to allow multiple threads from the same client

  23. Proxy Without Caching Client Proxy A Proxy B Channel Factory A Channel Factory B Channel A Channel B Service A Service B Service A Service B

  24. Proxy With MRU Caching Client MRU Cache Proxy A Channel Factory A Proxy B Channel A Channel B Channel Factory B Service A Service B Service A Service B

  25. MRU Cache and Client Architecture Client Thread 1 Thread 3 MRU Cache Proxy A Proxy B Channel Factory A Channel A Channel B Channel Factory B Thread 2 Thread 4 Service A Service B Service A Service B

  26. demo Clients and Proxy Caching

  27. Session Expiry • Primarily affects client-server architecture • Sessions can time out, or be faulted by uncaught exceptions • Service channel is torn down • Session is lost • Client channel must be recreated • User may not need to know • Proxy wrapper can help

  28. demo Handling Session Expiry

  29. Server-Server Architecture • ASP.NET or another service as client • Must scale across distributed boundaries • Avoid sticky IP/session • Should not use the same client channel • MAY be able to cache the channel factory • Don’t cache the channel • Session expiry not an issue • New channel for each call

  30. Downstream Services • Behind the firewall, multiple concurrent threads execute at the service • Calls to downstream services require a proxy, thus channel creation overhead • Caching channel factory and channel • Illusion of throughput, will choke with load • Caching channel factory • Improved throughput, still allows distribution among available connections

  31. Server Side Proxies HTTP HTTP Web Server WCF Service ASP.NET Page P P P P P P P P TCP TCP Application Server WCF Service WCF Service

  32. Limiting Service Hops • Crossing process and machine boundariesrequires WCF • Essential to distribution of functionality andapplication scalability • Service-oriented design implies reuse via services, even if invoked in-process • Some applications can’t withstand >2 service hops in a single request • Must benchmark the application

  33. Load Balancing and Failover • Scalability is impacted by ability to balance load across server machines • Sessions influence load balancing • Without sessions, requests can be directed to any machine • Transport session like TCP, requires sticky IP • With reliable and secure sessions, or application sessions, requires sticky sessions • Application sessions can be durable • Failover not supported by default

  34. Binding Considerations • NetTcpBinding • Sticky IP • BasicHttpBinding, WebHttpBinding • Keep-Alive settings • WS[2007]HttpBinding, WSDualHttpBinding, WS[2007]FederationHttpBinding • Reliable, secure sessions • Context-aware bindings

  35. Load Balancing and PerCall Client Proxy Server Machine A Server Machine B Op1 Op2 Op2 Service Service Service

  36. Load Balancing and PerSession Client Client Proxy Proxy Server Machine A Server Machine B Op1 Op2 Op1 Op2 Service Session A Service Session B State State

  37. Load Balancing, PerSession, and Durable Services Client Proxy Server Machine A Server Machine B Op1 Op2 Op1 Op2 Service Session A Service Session B State State Database

  38. Load Balancing and Transport Sessions Client Client Proxy Proxy Socket A Socket B Server Machine A Server Machine B Op1 Op2 Op1 Op2 Service Service Session

  39. Load Balancing and Reliable or Secure Sessions Client Client Proxy Proxy RM SX Server Machine A Server Machine B Op1 Op2 Op1 Op2 Service Service

  40. SSL Processing Load Balancer • Process SSL and forward • F5 BigIP a typical scenario • Service must accept unencrypted messages (including security tokens!) • Not supported by default • Can override default token authentication behavior to support this scenario

  41. Basic Authentication Client BasicHttpBinding+ Transport+ Basic HTTPS BasicHttpBinding+ Transport+ Basic Service

  42. Basic Authentication Client BasicHttpBinding+ Transport+ Basic HTTPS Process SSL+ Forward Authenticate Header+ Forward Message SSL LoadBalancers HTTP BasicHttpBinding+ TransportCredentialOnly+ Basic Service

  43. UserName Authentication Client BasicHttpBinding+ TransportWithMessageCredential+ UserName HTTPS Process SSL+ Forward Message SSL LoadBalancers HTTP BasicHttpBinding+ Message+ UserName Service

  44. demo Supporting SSL Processing Load Balancers

  45. Dublin-Related Enhancements Microsoft Confidential "Dublin" + WCF 4 + WF 4The New Middle Tier Dublin-Related Features Quadrant Visual Studio IIS Manager Model Deployment to Dublin WF and WCF Project Templates WF and WCF Management Modules App Server SCOM Pack Management APIs (PowerShellcmdlets) IIS/WAS Hosting Persistence Monitoring Messaging Availability, Deployment, Control Reliablility, Scalablity Application Monitoring Versioning, Partitioning, Routing System Center WCF and WF Frameworks Instance Restart Behavior Durable Timer Service Discovery Service SQL Persistence Provider WF SQL Tracking WCF SQL Tracking Router Service AutoStart Service Runtime Databases Persistence schema Monitoring schema Windows Server 2008

  46. Hosting Features with "Dublin"

  47. "Dublin" + WCF 4 + WF 4The New Middle Tier • Reliability and availability • Message-based activation • Automatic start improves host responsiveness • Persistence model enhanced • Improved lock management and restart for failed WF Service instances • Scalability • Distribute load with the RouterService • Asynchronous messaging with WF Services • Load balancing through persistence • Better lock management and exception handling

  48. Resources • Learning WCF • Michele Leroux Bustamante, O’Reilly 2007 • Reprinted with updates for VS2008, August 2008 • My Blog: http://www.dasblonde.net • See post-conference blog post forsamples from this and other sessions at Tech Ed! • Book blog: www.thatindigogirl.com

  49. Question & Answer

More Related