An ocaml based network services platform chris waterson waterson@liveops com
1 / 6

An OCaml-based Network Services Platform Chris Waterson [email protected] - PowerPoint PPT Presentation

  • Uploaded on

An OCaml-based Network Services Platform Chris Waterson [email protected] October 4, 2007. Motivation. Highly-interactive self-scheduling system Workers use site to book work throughout the week Non-trivial application logic Extreme peaks

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' An OCaml-based Network Services Platform Chris Waterson [email protected]' - santa

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript


  • Highly-interactive self-scheduling system

    • Workers use site to book work throughout the week

    • Non-trivial application logic

    • Extreme peaks

  • First generation system was LAMP-based and stateless

    • M=MySQL, P=Perl

    • Each interaction generated an HTTPS request

    • Client polled for updates

  • Issues with scaling led to a session-based redesign

    • Client maintains persistent SSL connection with server

    • No need to re-authorize or reconstruct state to handle client requests

    • Two-way communication allows events to be propagated to client

    • do we scale to millions of users? (== millions of SSL connections)

Designing a scalable server
Designing a Scalable Server

  • Conflicting requirements

    • Manageable programming model

    • Scalable implementation

  • Some Alternatives

    • "Naive" threading (one thread per session) over blocking I/O.

      • This is the model the application programmer wants

      • Conventional wisdom says: hard to scale past several hundred threads

      • Arbitrary pre-emption requires synchronization for shared structures

    • Event-based I/O with worker thread pooling

      • Parallelism is still limited by the number of worker threads

      • You still have synchronization issues

    • Event-based I/O with a single thread & explicit state management

      • Hand-coded state machines are difficult to maintain over time

Event based model redux
Event-based Model, Redux

  • Start with this:

    let rec echo fd =

    let str = readln fd in writeln fd str ; echo ()

  • Transform into continuation passing style

    • Each I/O operation takes a continuation routine that will be called when the I/O completes

      let rec echo fd sk =

      readln fd s

      (fun str s -> writeln fd str s (fun _ s -> echo fd sk))

    • The underlying I/O state is maintained in the s argument

    • If readln blocks, it can store k and use s to resume another I/O operation

    • We've encapsulated our application's "state machine" in the lexical closure

  • Transform into monadic style

    • Define bind and sequence operators to combine functions exactly as above

      let echo fd =

      readln fd >>= writeln fd >> echo fd


  • Monadic Communication Library

    • Continuation passing to encapsulate session state

    • An additional argument to maintain underlying I/O state

    • Layered monads to make it manageable

    • Provides basic I/O operations (read, write, connect, accept, ...)

    • Provides basic concurrency operations (fork, wait, notify)

  • Advantages

    • "Feels" like naively threaded code over blocking I/O

    • Runs over scalable I/O primitives (kqueue, epoll)

    • Pre-emption occurs at well-known points

  • OCaml

    • First-class functions to support continuations

    • Function currying to support monadic style

    • Production-quality compiler and runtime


  • Scales well

    • Two Xeon-class dual core servers

    • Supports 5,000 simultaneous users connecting via SSL

    • Sustains 700+ TPS (with bursts of 1,500 TPS) during peaks

    • Two major feature releases since initial deployment (mid-2006)

  • Issues

    • Requires non-blocking protocol implementations

    • Monadic style doesn't mix well with exceptions

  • Conclusion

    • FP techniques allowed us to have our cake...and eat it, too!

    • Hope to release as open source soon