1 / 39

Building Fast and Secure Web Services with OKWS

Building Fast and Secure Web Services with OKWS. Max Krohn - MIT CSAIL krohn@mit.edu. SparkMatch v1 (2000). SparkMatch v1 (2000). ~500,000 signups. WS1. Database. Load Balancer. WS2. WS3. MySQL. Apache ModPerl . SparkMatch Post-Mortem. v1 (2000) DB bottleneck due to table locking.

kalani
Download Presentation

Building Fast and Secure Web Services with OKWS

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Building Fast and Secure Web Services with OKWS Max Krohn - MIT CSAIL krohn@mit.edu

  2. SparkMatch v1 (2000)

  3. SparkMatch v1 (2000) ~500,000 signups WS1 Database Load Balancer WS2 WS3 MySQL Apache ModPerl

  4. SparkMatch Post-Mortem • v1 (2000) • DB bottleneck due to table locking

  5. SparkMatch v2 (2001)

  6. SparkMatch v1 (2000) ~500,000 signups WS1 Database Load Balancer WS2 WS3 MySQL Apache ModPerl

  7. SparkMatch v2 (2001) ~1,000,000 signups Database WS1 Load Balancer WS2 WS3 WS4 MySQL Apache ModPerl WS5 WS6

  8. SparkMatch Post-Mortem • v1 (2000) • DB bottleneck due to table locking • v2 (2001) • too many DB connections • Perl was too slow • difficult to implement efficient caching

  9. SparkMatch v3 (2002) DB1 WS1 Load Balancer WS2 DB2 WS3 MySQL Apache C Modules

  10. SparkMatch Post-Mortem • v1 (2000) • DB bottleneck due to table locking • v2 (2001) • too many DB connections • Perl was too slow • difficult to implement efficient caching • v3 (2002) • development cycle was too long • code was unsafe

  11. Some Desired WS Features • Thin, fast server (no cruft) • Smart gzip support • Small # database connections per WS • Easy to build fast, efficient caches • Leaves free memory for caches • Easy and safe to write C/C++ code

  12. Not Necessary • Support for static content • Images • Styles Sheets • Terms & Conditions • Bottleneck is access link  Apache suffices!

  13. Static/Dynamic Split WS1 ~$1,000 per Mb/s per Month DB1 Load Balancer WS2 DB2 WS3 New York City - Dynamic ~$100 per Mb/s per Month Texas - Static Apache Apache Apache Apache

  14. Apache/PHP For Dynamic Content? • History of buffer overruns in Apache and PHP (see paper). • Users likely to introduce more when writing C extensions. • Poor isolation strategy.

  15. Apache/PHP Isolation Apache/PHP Search Index and Profiles Process 1 Process 2 Search.php Search.php ShowProfile.php ShowProfile.php ChangePW.php ChangePW.php Passwords logger config logger config

  16. Apache/PHP Isolation Apache/PHP Search Index and Profiles Process 1 Process 2 Search.php Search.php ShowProfile.php ShowProfile.php ChangePW.php ChangePW.php Passwords logger config logger config

  17. What We Built • The OK Web Server (OKWS) • Specialized for dynamic content service. • Meets our earlier-stated requirements.

  18. OKWS Design • A Web site consists of many Web services. • e.g., Search, ShowProfile, ChangePW • A and B are distinct services if they access different pools of data. • One-to-one mapping between Web Services and Unix processes.

  19. OKWS Isolation Strategy • Process pool fixed at startup (~10). • Each obeys least-privilege principle. • Isolates processes: • From SQL database access • From each other • From the OS (filesystem in particular) • From DBs they need not access.

  20. How To Build a Web Service ShowProfile ~2000 LOC RPC ~50 LOC R/O In-Memory Profile Cache Server Profile-DB Translator SQL DB

  21. client OKWS Block Diagram GET /ShowProfile HTTP/1.1 launcher demux HTTP Response Pass FD Search ShowProfile ChangePW RPC RPC RPC Profile-DB Translator PW-DB Translator logger (write to FS) SQL pubd (read from FS) DB2 DB1

  22. OKWS Process Isolation Log Jail Docs Jail Run Jail launcher UID=0 demux UID=okd logger UID=oklog Search UID=51001 ShowProfile UID=51002 ChangePW UID=51003 pubd UID=www Web Server Machine Profile-DB Translator PW-DB Translator DB1 DB2 Database Machines

  23. If Service A is Compromised… • cannot access its own DB outside the RPC interface provided. • cannot access setuid executables. • cannot access logs, config files, source files, privileged ports. • cannot send service B signals • cannot trace service B's system calls • cannot access B's database

  24. OKWS Implementation • Developed C++ with the SFS libraries • Single Threaded / Event Driven • Services developed with C++/SFS. • DB translation libraries: • SFS-style asynchronous RPC • Standard threaded client libraries.

  25. Perl-Like Development Tools • Refcounted garbage collection. • Refcounted safe string classes. • RPC Stubs for network communication. • Perl-compatible regular expressions. • OKWS preprocessor for Perl-style heredocs.

  26. 1 Service = 1 Process, 1 Thread • No synchronization required for in-memory caches. • Small, fixed-size process pool  more memory available for caching. • Each service needs one connection to database machine. • No pooling required.

  27. OkCupid (i.e., SparkMatch v4)

  28. OkCupid Deployment • 23 Public Web Services • 12 Database Translators

  29. Site Performance • Pentium Dual 2.4GHz Xeon / FreeBSD • 1M+ users, 100k+ now active • Example throughput observed near 200 req/sec (~20KB each) • one WS handled this load with 7% CPU utilization

  30. OKWS Performance Testing • In paper, compared to: • Apache2/PHP4 • Haboob (JDK1.3) • Flash v1 • Results: OKWS gets better throughput, responsiveness.

  31. Performance Datapoint • C/C++ faster than interpreted code • Methodology: Generate a 10x30 HTML-formatted multiplication table. • (Same LOC for both PHP and OKWS)

  32. OKWS Partial Summary • Thin, fast server (no cruft) • No DB pooling required. • Easy to build fast, efficient caches • Not wasteful of memory. • Easy and safe to write C++ code.

  33. Conclusion • Used to build real Web sites! • Match Service (OkCupid.com) • New Diversion (Movie-Madness.org) • Could Also be Used For • Blogging Sites • Social network sites • HTTP Front end to distributed systems • GPL'ed at: www.okws.org

  34. The End www.okws.org

  35. Does the # of processes matter?

  36. Increased Context Switching

  37. WS Comparison (Single Null Svc)

  38. WS Comparison (Single Null Svc)

  39. WS Comparison ("Realistic")

More Related