Scalability of Linux Event-Dispatch Mechanisms

1. Scalability of Linux Event-Dispatch Mechanisms David Mosberger Hewlett Packard Labs Palo Alto Abhishek Chandra University of Massachusetts Amherst

2. Motivation Web Server Clients WAN • Large Web and Internet traffic • Heavily Loaded/Accessed Web Servers • cnn.com, britneyspearsfans.com, … • Starr Report, Napster ruling, ... • Challenge: Make Web Servers Scalable

3. Server Scalability Issues • Large number of concurrent/idle connections • Last-mile problem: Slow end-connections • High latency WAN traffic • HTTP/1.1 Persistent Connections • Heavy Request Loads • Need for high throughput • Pure Thread-based vs. Event-based servers • Focus: Scalability of Event-based servers on Linux

4. Outline • Motivation • Event-based Servers • Linux Event-Dispatch Mechanisms • Evaluation: Handling concurrent connections • RT signals and Signal-per-fd enhancement • Evaluation: Handling request load • Concluding Remarks

5. 1. Interest Set Specification 3. Event Notification 4. I/O Handling Interest Set 2. Network Event Event-Based Servers • Server specifies Interest Set to Kernel • Kernel notifies Server of Event on a connection • Server handles I/O on the connection Server Kernel Connections

6. Linux Event-Dispatch Mechanisms • select() system call • poll() system call • /dev/poll interface • POSIX.4 Real-Time Signals

7. Interest Set Ready Set Scan select() system call • Interest Set specified on each call • Notification requires scan of interest set Server Kernel Connections

8. poll() and /dev/poll • Interest Set: • List of pollfd structures • Better for sparse interest sets, worse for dense sets • Notification • Requires scan of Interest Set • /dev/poll: • Interest Set specified incrementally • More compact ready set

9. POSIX.4 Real Time Signals • RT signals are queued • Multiple signals of same type can be delivered • RT signals carry a data payload (siginfo) • Provides the context of the signal • sigwaitinfo() system call: • Dequeues signals • Avoids overhead of calling signal handler • Signal can be blocked

10. 1. Associate RT Signal 4.sigwaitinfo() 3. EnQueue Signal 5. DeQueue Signal 2. Network Event Using Real Time Signals for Network I/O • Interest Set specified incrementally • No scanning of Interest Set required Server Kernel Signal Queue Interest Set Sockets

11. Outline • Motivation • Event-based Servers • Linux Event-Dispatch Mechanisms • Evaluation: Handling concurrent connections • RT signals and Signal-per-fd enhancement • Evaluation: Handling request load • Concluding Remarks

12. Evaluation: Handling Concurrent Connections • Dispatch overhead and latency as a function of number of concurrent connections • Experimental Setup • 400 MHz P3 Linux 2.4.0-test7 server • μ -server using select(),/dev/poll or RT signals • 10 clients running httperf • Fixed request rate, increasing number of connections

13. Server CPU Usage 500 req/s • RT signal overhead independent of no. of concurrent connections

14. Response Time 500 req/s • RT signal response time independent of no. of concurrent connections

15. Drop 3 Network Event Network Event Network Event Limitations of Real Time Signals • Signal Queue Overflow: • New events lost • Can lead to “hung server” • Unfair Allocation of Signal Queue Server 3 3 2 Kernel Signal Queue Interest Set Sockets 1 2 3 4

16. Handling Signal Queue Overflow • Fallback mechanism • select(), poll(), etc. • Reconstruct current state • Issues • Server complexity • Overhead of maintaining explicit interest sets • Potential performance penalty

17. 1 Discard Network Event Network Event Network Event RT Signal Enhancement:Signal-per-fd • Goals: • Avoid signal queue overflows • Fair Allocation of signal queue • Solution: Enqueue only one signal per socket Server 3 2 Kernel Signal Queue Interest Set Sockets 1 2 3 4

18. Signal-per-fd • Idea: • Signal queue length same as fdset size • Bitmap used to efficiently determine presence/absence of signal in queue • Advantages: • Simpler Server Implementation • No signal queue overflows • No need for fallback mechanisms • Fair Allocation of Signal Queue Resource • Avoids too fine-grained event notification • Coalesce multiple events for a socket

19. Outline • Motivation • Event-based Servers • Linux Event-Dispatch Mechanisms • Evaluation: Handling concurrent connections • RT signals and Signal-per-fd enhancement • Evaluation: Handling request load • Concluding Remarks

20. Server Throughput 6000 idle connections • Linear scaling of RT signals, signal-per-fd

21. Server CPU Usage 6000 idle connections • Linear Scaling of RT signals, signal-per-fd

22. Related Work • Event-Delivery API [BMD99] • Performance studies: • select() [BM98], /dev/poll [PL00] • RT signals [PLT00] • Web Servers: • Event-based: Flash [PDZ99], phhttpd [Brown99] • In-kernel: TUX, khttpd, AFPA [JKNRT01] • Future: Linux 2.5 Asynchronous I/O?

23. Summary • Scalability issues with Linux Event-dispatch mechanisms • Real Time Signals are scalable • Performance independent of number of concurrent connections • Signal Queue Overflow Problems • Signal-per-fd enhancement • potentially improves performance • reduces server complexity • provides fairness • Patch available at http://www.netli.com/links