lazy asynchronous i o for event driven servers n.
Skip this Video
Loading SlideShow in 5 Seconds..
Lazy Asynchronous I/O For Event-Driven Servers PowerPoint Presentation
Download Presentation
Lazy Asynchronous I/O For Event-Driven Servers

play fullscreen
1 / 41
Download Presentation

Lazy Asynchronous I/O For Event-Driven Servers - PowerPoint PPT Presentation

selah
109 Views
Download Presentation

Lazy Asynchronous I/O For Event-Driven Servers

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. Lazy Asynchronous I/O For Event-Driven Servers Khaled Elmeleegy, Anupam Chanda and Alan L. Cox Department of Computer Science Rice University, Houston, Texas. Willy Zwaenepoel School of Computer and Communication Sciences EPFL, Lausanne, Switzerland.

  2. Event-Driven Architecture • Event-driven architecture is widely used for servers • Performance • Scalability

  3. Problem • Developing event-driven servers is hard for many reasons • We focus on problems with non-blocking I/O: • Incomplete coverage • Burden of state maintenance

  4. Lazy Asynchronous I/O (LAIO) • Addresses problems with non-blocking I/O • Universality • Covers all I/O operations • Simplicity • Requires less code • High performance for event-driven servers • Meets or exceeds alternatives

  5. Outline • Background • Lazy Asynchronous I/O (LAIO) • Evaluation • Conclusions

  6. Outline • Background • Event-driven servers • Lazy Asynchronous I/O (LAIO) • Evaluation • Conclusions

  7. Event-Driven Servers • Event loop processes incoming events • For each incoming event, it dispatches its handler • Single thread of execution Event Loop Handler #1 Handler #2 Handler #k

  8. Event Handler I/O operation (Network/Disk) To event loop Complete handling event

  9. Event Handler • If the I/O operation blocks • The server stalls I/O operation (Network/Disk) Block To event loop Complete handling event

  10. Outline • Background • Lazy Asynchronous I/O (LAIO) • API • Example of usage • Implementation • Evaluation • Conclusions

  11. LAIO API

  12. laio_syscall() • Lazily converts any system call into an asynchronous call • If the system call doesn’t block • laio_syscall() returns immediately • With return value of system call • Else if it blocks: • laio_syscall() returns immediately • With return value -1 • errno set to EINPROGRESS • Background LAIO operation

  13. laio_gethandle() • Returns a handle representing the last issued LAIO operation • If operation didn’t block, NULL is returned

  14. laio_poll() • Returns a count of completed background LAIO operations • Fills an array with completion entries • One for each operation • Each completion entry has • Handle • Return value • Error value

  15. Event Handler With LAIO • If operation completes • Handler continues execution laio_syscall() Operation completed No To event loop Yes Complete handling event

  16. Event Handler With LAIO • If operation doesn’t complete • laio_syscall() returns immediately • Handler records LAIO handle • Returns to event loop • Completion notification arrives later laio_syscall() Operation completed No Handle= laio_gethandle() To event loop Yes Complete handling event

  17. LAIO Implementation • LAIO uses scheduler activations • Scheduler activations • The kernel delivers an upcall when an operation • Blocks - laio_syscall() • Unblocks - laio_poll()

  18. laio_syscall() – Non-blocking case Application laio_syscall() • Save context • Enable upcalls Issue operation Library System call blocks? No • Disable upcalls • Return retval

  19. laio_syscall() – Blocking case Application laio_syscall() Library • Save context • Enable upcalls Issue operation System call blocks? Yes

  20. laio_syscall() – Blocking case Application laio_syscall() Library • Save context • Enable upcalls Issue operation System call blocks? Yes Kernel Background laio_syscall Upcall on a new thread

  21. laio_syscall() – Blocking case Application laio_syscall() Library • Save context • Enable upcalls Issue operation upcall handler System call blocks? Library Steals old stack using stored context Yes Kernel Background laio operation Upcall on a new thread

  22. laio_syscall() – Blocking case Application laio_syscall() • Disable upcalls • errno = EINPROGRESS • Return -1 Library • Save context • Enable upcalls Issue operation upcall handler System call blocks? Library Steals old stack using stored context Yes Kernel Background laio operation Upcall on a new thread

  23. List of completions is retrieved by the application using laio_poll() upcall handler() Library • Construct completion structure: • laio operation handle. • System call return value. • Error code. • Add completion to list of • completions. Unblocking case Kernel • Background laio operation completes, thread dies • Upcall on the current thread

  24. Outline • Background • Lazy Asynchronous I/O (LAIO) • Evaluation • Methodology • Experiments • LAIO vs. conventional non-blocking I/O • LAIO vs. AMPED • Programming complexity • Conclusions

  25. Methodology • Flash web server • Intel Xeon 2.4 GHz with 2 GB memory • Gigabit Ethernet between machines • FreeBSD 5.2-CURRENT • Two web workloads • Rice 1.1 GB footprint • Berkeley 6.4 GB footprint

  26. Experiments: LAIO vs. conventional non-blocking I/O Compare performance

  27. Performance: Large Workload

  28. Performance: Small Workload

  29. Why Be Lazy? • Most potentially blocking operations don’t actually block • Experiments: 73% - 86% of such operations don’t block • Lower overhead • Micro-benchmark: AIO is 3.2 times slower than LAIO for non-blocking operations

  30. Experiments: LAIO vs. AMPED Compare performance

  31. Asymmetric Multi-process Event-Driven (AMPED) Architecture • Event-driven core • Potentially blocking I/O handed off to a helper • Helper can block as it runs on a separate thread of execution Event Loop Helper #1 Handler #1 Helper #2 Handler #2 Helper #n Handler #k

  32. Flash-NB-AMPED • Stock version of flash • Two relevant helpers • File read: reads a file from disk • Name conversion: checks for a file’s existence and permissions

  33. Performance of LAIO vs. AMPED

  34. Performance of LAIO vs. AMPED

  35. Programming Complexity • Where performance is comparable: Flash-LAIO-LAIO vs. Flash-NB-AMPED • Flash-LAIO-LAIO is simpler • No helpers • No state maintenance

  36. State Maintenance With Non-blocking I/O • If operation does not complete • Handler returns to event loop handler I/O operation Operation completed No To event loop Yes Complete handling event

  37. State Maintenance With Non-blocking I/O • If operation does not complete • Handler returns to event loop • Operation is continued later handler I/O operation Operation completed No To event loop Yes Complete handling event

  38. State Maintenance With Non-blocking I/O • If operation does not complete: • Handler returns to event loop • Operation is continued later • This may require storing state handler I/O operation Operation completed To event loop No Store state Yes Complete handling event

  39. Lines of Code Comparison 9.5% reduction in lines of code

  40. Conclusions • LAIO is universal • Supports all system calls • LAIO is simpler • Used uniformly • No state maintenance • No helpers • Less lines of code • LAIO meets or exceeds the performance of other alternatives

  41. LAIO source can be found at: http://www.cs.rice.edu/~kdiaa/laio