1 / 82

JAWS (A Framework for High-Performance Web Servers) TP Version 1.1

Son YoungSu indigoguru@gmail.com Microsoft MVP Devpia Architecture Sysop EvaCast Study Leader ( http://www.EvaCast.net ). JAWS (A Framework for High-Performance Web Servers) TP Version 1.1. What’s the Good Web Server?. Architect. Motivation. Motivation. Motivation. Seed Paper.

snow
Download Presentation

JAWS (A Framework for High-Performance Web Servers) TP Version 1.1

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Son YoungSu indigoguru@gmail.com Microsoft MVP Devpia Architecture Sysop EvaCast Study Leader (http://www.EvaCast.net) JAWS(A Framework for High-Performance Web Servers)TP Version 1.1

  2. What’s the Good Web Server? Architect Motivation http://www.EvaCast.net

  3. Motivation http://www.EvaCast.net

  4. Motivation http://www.EvaCast.net

  5. Seed Paper • James C. Hu, Douglas Schmidt, JAWS: A Framework for High-Performance Web Servers, Domain-Specific Application Frameworks: Frameworks Experience By Industry, John Wiley & Sons, October, 1999. http://www.EvaCast.net

  6. Agenda • Motivation • Applying Patterns & Frameworks to Web Server • Common Pitfalls of Developing Web Server • Overcoming Web Server Pitfalls • Relationship Between Frameworks, Patterns & Other Reuse Technique • The JAWS Web Server • Overview of the JAWS Framework • Overview of the Design Pattern in JAWS • Concurrency Strategy • I/O Strategy • Protocol Pipeline Strategy • File Caching Strategy • Web Server Benchmarking • Concluding Remarks http://www.EvaCast.net

  7. 2. Applying Patterns & Frameworks to Web Server • Common Pitfalls of Developing Web Server Software • Overcoming Web Server Pitfalls with Patterns and Frameworks • Relationship between Frameworks, Patterns, and Other Reuse Techniques http://www.EvaCast.net

  8. 2.1 Common Pitfalls of Developing Web Server SW • Excessive low-level detail • Continuous rediscovery & reinvention of incompatible higher-level programming abstraction. • High potential for errors • Lack of portability • Steep learning curve • Inability to handle increasing complexity http://www.EvaCast.net

  9. 2.2 Overcoming Web Server Pitfalls with Patterns & Frameworks • Limitations of Class Library • do not capture the control flow and collaboration among families of related SW components. • Class Library reuse often re-invent and re-implement the overall Software architecture for each new Application. http://www.EvaCast.net

  10. 2.2 Overcoming Web Server Pitfalls with Patterns & Frameworks • The Benefits of Patterns • Expert’s Design Experiences • Good Methods to use Framework • The infrastructure of Framework • The Benefits of Frameworks • Expert’s intellectual products • Help developers avoid costly re-invention http://www.EvaCast.net

  11. 2.3 Relationship Between Frameworks, Patterns, and Other Reuse Techniques Class Libraries vs Framework Class Library NETWORKING GUI Active Object State DATABASE ADTs App Specific Logic OO Design MATH MATH NETWORKING Invocations Event Loop Reactor GRAPHICS GUI App Specific Logic Event Loop Invocations Callbacks Singleton Strategy Selections ADTs Reactor Adapter DATABASE GRAPHICS Singleton Adapter State Active Object Design Pattern Class Library Component Architecture Application Framework Component Architecture http://www.EvaCast.net

  12. 2.3 Relationship Between Frameworks, Patterns, and Other Reuse Techniques • Frameworks define “semi-complete” application that embody domain-specific object structures and functionality. • Class Libraries don’t offer explicit guidance to system design • Components in a Framework manage the canonical flow of control within App • Frameworks are active and exhibit“IoC(Inversion of Control)” at runtime. • Template Method Pattern (Hollywood Principle) • In Practice, Frameworks, Class Libraries, and Components are complementary technologies. http://www.EvaCast.net

  13. 3. The JAWS Adaptive Web Server • Overview of the JAWS Framework, • Overview Design Pattern in JAWS. • 4 Core Components in JAWS • Concurrency. • I/O. • Protocol/Pipeline. • File Caching. • JAWS Framework Revisited http://www.EvaCast.net

  14. 3.1 Overview of the JAWS Framework http://www.EvaCast.net

  15. 3.2 Overview Design Pattern in JAWS • Strategic Pattern • Message Composition/Decomposition • Pipe & Filter , Layer, Composite Message • Dispatcher • Acceptor –Connector / Reactor / Proactor • Active Object • Service Configurator • Tactical Pattern • Strategy • Adapter • State • Singleton http://www.EvaCast.net

  16. Strategic Pattern Overview Dispatching Data Layer Decomposite Message UnMarshaling Layer Composite Message Messages Marshaling

  17. 3.2.1 Strategic Pattern • Marshaling / UnMarshaling • Composite Message / Layer / Pipe & Filter • Dispatcher • Acceptor-Connector / Reactor / Proactor • Active Object • Service Configurator http://www.EvaCast.net

  18. Composite Message http://www.EvaCast.net

  19. Acceptor/Connector Pattern http://www.EvaCast.net

  20. Acceptor Protocol Handler peer_acceptor_accept() peer_stream_open() Acceptor Pattern • Acceptor is Factory. • Creates, Accepts and Activates a new Protocol Handler Whenever the Event Dispatcher notifies Acceptor that a connection has arrived from a Client. Protocol Handler Task Concurrency Event Dispatcher http://www.EvaCast.net

  21. Reactor Pattern Event Handler handle_event(in type) get_handle() Reactor handle_events() register_handler(in handle) remove_handler(in handle) Concrete Event Handler Concrete Event Handler Synchronous Event Demultiplexer handle_event(in type) get_handle() handle_event(in type) get_handle() select() dispatch handle set owns Handle <<uses>> notifies http://www.EvaCast.net

  22. Reactive IO Handler handle_input() handle_output() Reactor send_file() recv_file() send_data() recv_data() Initiation Dispatcher Timer_Queue handle_event() register_handler(h) remove_handler(h) schdule_timer(h) cancel_timer(h) expire_timers(h) Reactor Pattern • decouples the synch demultiplexing and dispatching logic. • the Single-Thread Web Server Concurrency model InputOutput Handler Protocol Pipeline InputOutput Reactive IO Filecache Handle Synchronous IO Asynchronous IO IO Handle http://www.EvaCast.net

  23. Proactor Pattern Completion Handler handle_event() Asynchronous Event Demultiplexer Asynchronous Operation Processor Asynchronous Operation Proactor get_completion_event() async_operation() execute_async_operation() handle_events() Initiator <<uses>> <<uses>> <<uses>> <<invokes>> is associated with Handle * <<enqueues>> <<demultiplexes & dispatches>> <<executes>> Concrete Completion Handler Completion Event Queue <<dequeues>> http://www.EvaCast.net

  24. Thread Thread Thread 4. Link Socket and Port Proactor Pattern (IOCP) 7. Thread Returns Thread Pool 5. 1 Send Completion Socket Info 3. Create Thread Socket A 2.1 Create Completion Port Completion Port Socket B 5. 2 Send Completion Packet Socket C 1. Create Socket 2.2 Create Completion Queue Thread Pool 6. Process Completion Events http://www.EvaCast.net

  25. Proactive IO Handler Asynch Op open() cancel() handle_read_file() handle_write_file() Proactor send_file() recv_file() send_data() recv_data() Timer_Queue Completion Dispatcher schdule_timer(h) cancel_timer(h) expire_timers(h) handle_event() register_handler(h) remove_handler(h) Proactor Pattern • decouples the Asynch demultiplexing and dispatching logic. • The Asynch variant of the Thread Pool • Use IOCP (IO Completion Port) InputOutput Handler Protocol Pipeline InputOutput Asynchronous IO Filecache Handle Synchronous IO Reactive IO IO Handle http://www.EvaCast.net

  26. Active Object Pattern Scheduler ActivationQueue Proxy Dispatch() Insert() Insert() Remove() method_1() method_n() Servant method_1() method_n() 1 1 1 1 invoke creates creates maintains * MethodRequest Future write to Can_run() Call() obtain result from Concrete MethodRequest N Concrete MethodRequest 1 Client http://www.EvaCast.net

  27. Protocol Handler http_request() ftp_request() snmp_request() Activation Queue insert() remove() Scheduler http_request() ftp_request() snmp_request() dispatch() Active Object Pattern • decouples method invocation from method execution. • Thread-per-Request, Thread Pool, Thread-per-Session Model use Active Object pattern. for(;;){ m = aq->remove(); dispatch(m); } aq->insert(http) delegates Method Object Resource Representation Protocol Pipeline http://www.EvaCast.net

  28. Service Configurator Pattern Component Component Repository +Init() +Fini() +Suspend() +Resume() +Info() Service Configurator CoreComponent CoreComponent http://www.EvaCast.net

  29. Service Configurator Pattern Service +init() +fini() +suspend() +resume() +info() Filecache Service Repository Protocol Handler Filter Repository Cache Strategy Repository DLL Cache Strategy Protocol Pipeline Read Request Filter LRU Strategy LFU Strategy Parse Request Log Request http://www.EvaCast.net • Provides System Configuration at Runtime. • Dynamically optimize, control and reconfigure the behavior of Web server strategies a installation-time or during run-time.

  30. 3.2.2 Tactical Patterns • Strategy • Configure Cache replacement strategies without afffecting the core SW Architecture. • Adapter • Uniformly encapsulate the operation of synch, asynchand reactive I/O. • State • Event Dispatcher use State to seamlessly support different concurrency strategies and both synch and asynch. • Singleton • To provide a global point to access object. • Exist one copy of Cached Virtual File System. http://www.EvaCast.net

  31. Review http://www.EvaCast.net

  32. 3.3 4 Components in JAWS http://www.EvaCast.net

  33. 3.3 Concurrency Strategy • Design Challenge • A large portion of non-I/O related Web server overhead is due to the Web server’s concurrency strategy. • Key Overhead • Synchronization • Thread/Process Creation • Context Switching • Choosing an efficient concurrency strategy is crucial to achieve high-performance http://www.EvaCast.net

  34. 3.3 Concurrency Strategy • Alternative Solution Strategies • Thread-per-Request • Thread-per-Session • Thread Pool • Single-Thread • Experiments • JAWS Concurrency Strategy Framework http://www.EvaCast.net

  35. 3.3.1 Alternative Solution Strategies • Concurreny Strategy of Web Servers • Single Thread • Roxen • Process-Based • Apach • Zeus • MultiThread • PHTTD • JAWS Dynamic Factors # of simultaneous requests Memory Use Machine Load Memory Workload Static Factors Web Server Use Case OS Platform H/W Configuration http://www.EvaCast.net

  36. Thread-per-Request • Allows each client request to run concurrently in a separate thread. • Each Request arrives, a new thread is crated to process request. • This design appropriate synch I/O mechanisms Protocol Handler Protocol Handler Task Acceptor Protocol Handler Protocol Handler Concurrency Event Dispatcher http://www.EvaCast.net

  37. Thread-per-Request (cont’d) • Advantage • Simplicity • Exploit parallelism on multi-processor platform • Limitation • The # of running threads may grow without bound, exhausting available memory and CPU resources • Compatible Fields • Use Light loaded servers with low latency • Don’t use time consuming task based system http://www.EvaCast.net

  38. Thread-per-Session • Allows each client connection to run concurrently • All of these requests are submitted through a connection between each client and a separate thread in the web server process • Advantage • Does not spawn a separate thread for each request. • Limitation • This model still vulnerable to unbounded resource consumption as the # of clients grows. http://www.EvaCast.net

  39. Thread-per-Session (cont’d) • Suited for HTTP 1.1 but not HTTP 1.0 • HTTP 1.0 • support open/operation/close • One URL Fetch per Connection • Needs Reconnection • Transporting Data Size Limitation • URL Size Limitation • HTTP 1.1 • Persistent Connection • Pipeline Functionality • Many Requests -> Support Serial Response • Compressed Data Transporting • Use Proxy Server and Cache http://www.EvaCast.net

  40. Thread Pool • Allows up to N requests to execute concurrently within a pool of threads. • Support prespawning of Thread Creation Protocol Handler Protocol Handler Task Acceptor Protocol Handler Protocol Handler Concurrency Event Dispatcher http://www.EvaCast.net

  41. Thread Pool (cont’d) • Advantage • Lower overhead that Thread-per-Request. • Thread Prespawning • thread creation cost is amortized • Pool Size is Fixed • Resource Consume is bounded • Limitation • Small Pool Size • New incoming requests be dropped or wait indefinitely. • Large Pool Size • Resource Consumption may be no better than using Thread-Per-Request http://www.EvaCast.net

  42. Single-Threaded • All connections and requests are handled by the same thread of control. • Single-Threaded servers process requests iteratively. • Compatible Filed • Asynch or Reactive I/O Multiple Request • Inadequate for high volume server • Subsequent requests are blocked until they are reached http://www.EvaCast.net

  43. Experiments • Static adaptivity • Select the concurrency strategy that best meets The static requirements of the system. • multi-threaded concurrency • More suited to A multiple-processor machinethan a uni-processor machine • Dynamic adaptivity • Adapt dynamically to current server condition • In order to accommodate Unanticipated load • Support to increase the # of available threads in the Pool http://www.EvaCast.net

  44. Event Dispatcher Concurrncy Thread Pool Thread-per-Connection Acceptor Protocol Handler dispatch() dispatch() open() accept() dispatch() dispatch() Task accept() activiate() enque() deque() svc() JAWS Concurrency Strategy Framework server handler server->dispatch() acceptor task acceptor->accept() task->accept() task->activate() task->enque() http://www.EvaCast.net

  45. 3.4 I/O Strategy • Design Challenge • is to devise efficient data retrieval and delivery strategies, collectively referred to as I/O. • Key Overhead • Arranging Multiple I/O operation. • Example • Synch • Web Transaction involving monetary fund transfer • ASynch • Web access to static information (Web page – ASP..) http://www.EvaCast.net

  46. 3.4 I/O Strategy • Alternative Solution Strategies • The Synchronous I/O Strategy • The Reactive I/O Strategy • The Asynchronous I/O Strategy • Experiments • JAWS I/O Strategy Framework http://www.EvaCast.net

  47. The Synch I/O Strategy • Synch I/O describes the model of I/O interactionbetween a Web server process and kernel. • Kernel doesn’t return the thread of control to the server until the requested I/O operation either complete , completes partially, or fails. • Advantage (Easy Use) • Disadvantage (Resource Exhaust) • If combined with a single threaded concurrency strategy, It is not possible to perform multiple synch I/O operation simultaneously. • When using multiple thread, It is still possible for an I/O request to block indefinitely. http://www.EvaCast.net

  48. The Reactive I/O Strategy • Reactive I/O alleviates the blocking problems of synchronous I/O without resorting to polling. • Web Server uses an OS event demultiplexing system call. • select() in UNIX • WaitForMultipleObjects() in Win32 • Refers Reactor Pattern • Disadvantage • May not make effective use of multiple CPU. http://www.EvaCast.net

  49. The Asynch I/O Strategy • Asynch I/O simplifies the de-multiplexing of multiple events in one or more thread of control without blocking the Web server. • Advantage • The Web Server not block on I/O request. • Disadvantage • Asynch I/O is not available on many OS Platform. • more complicated than writing synch program. http://www.EvaCast.net

  50. Experiments • The Result reveal that each I/O strategy behaves differently under different load condition. • JAWS I/O Strategy Framework addresses this issue by allowing the I/O Strategy to adapt dynamically to run-time server condition. http://www.EvaCast.net

More Related