Erlang/OTP

1. Erlang/OTP A framework for building real-world applications

2. Erlang/OTP • Open Telecom Platform • When building a large system • certain designs reappear frequently • differing only in what they do • generic part called “behaviour” • things {processes, hardware} WILL fail • might as well accept it and deal with it • distributed systems easily become large • monitoring + keeping track are the dominating problems • especially when multiple components have to cooperate

3. OTP view of a distributed system S* = Supervisors P* = Processes S1 P1 P2 S2 P3 P4 This is called a supervisiontree

4. But wait, where’s the hardware? • From Erlang Pt 1: • Messagepassing between Processes is transparent • Starting processes as well • Pid = spawn(nodename@hostname, Function) HostA HostB Node1 Node1 Node2 Node2

5. Behaviours in OTP • Repetetive patterns • generic server (gen_server) • models the client/server architecture • generic event handler (gen_event) • models one-way event (does not expect reply) • generic finite state machine (gen_fsm) • state + event = new state • all states and events are user defined! • supervisor • supervises a set of processes which typically are gen_* implementations themselves • Only need to fill in the callbacks!

6. Behaviours built on top of: • many processes is a Good Thing • transparent messagepassing • no difference AT ALL between local/remote processes • pattern-matching • delegating messages/events to appropriate handler • side-effect-freeness • enables hot codeswap without interrupted service or state (momentarily) • registered processes: • register(ProcessId, someatom). • someatom ! {message, Argument}.

7. The supervisor • Responsible for managing processes • Hierarchy is important and maintained • Assures its configured children are running • Restarts them if terminated unexpectedly • Depending on “child-specification” • {Module, Function, Args} • restartpolicy, eg: • one-for-one: one process dies, the same one gets restarted • one-for-all: one process dies, the whole set gets killed and restarted • rest-for-one: one process dies, it and the processes following it will be restarted • transient: will not be restarted upon normal exit • maximum restartfrequency, to prevent infinite tight restartloops because of a bug/failure. if threshold passed, whole supervisiontree will be terminated.

8. Example: gen_server -module(my_server). -behaviour(gen_server). % compiler issues warnings/errors about missing callbacks! % Define your client API test_server(Argument) -> gen_server:call(myserver_instance, {test, Argument}). % Start the server, typically from a supervisors’ ChildSpec start_link(Args) -> gen_server:start_link({local, myserver_instance), my_server, Args, _Options). % The callbacks init(Args) -> {ok, stateFromArgs(Args)}. handle_call({test, Argument}, From, State) -> {reply, {ok, Argument}, State}. code_change(OldVsn, State, Extra) -> NewState = modify(State), {ok, NewState}. terminate(Reason, State) -> cleanup(State), ok.

9. Example: gen_server (using it) Terminal 2 $> erl -sname client@localhost (client@localhost)1> my_server:test_server(“aap”). “aap” (client@localhost)2> Terminal 1 $> erl -sname server@localhost (server@localhost)1> my_server:start_link(). {ok,<0.43.0>} handle_call({test, "aap") (server@localhost)2>

10. Erlang and the outside world * Linking to external programs *strongly* discouraged * Instead: "ports" - launch remote program - communicate over streams (e.g., stdin/stdout) - binary or line-based protocols possible * Used to talk to gnuplot and ssh managers

11. Why do we need ssh managers? * Long-running command (ssh <user@host> tail -f <file>) never stops * Ports can be closed (usually on eof) * Doesn't stop the associated Unix process! * Need wrapper that can stop program

12. Implications * Communication with external programs is low-tech * Very unlike scripting languages (Python et al.) * Additional library support for C programs, but * Pushes you towards an Erlang-centric architecture

13. In practice: log monitoring/plotting Motivations * Python had threading issues and * We needed an Erlang demonstrator application Decision: * Rewrite in Erlang * Erlang spawns "ssh tail -f"

14. Overview: log scraping * Script at stations copies logs to mk5s * Erlang/Python process copies logs to JIVE * Erlang process scans logs for selected events * Events stored to database * Erlang/gnuplot process makes plots (.png)

15. Overview: realtime monitoring * Schedules converted to database * PHP queries db (via URL+json) for active stations * Erlang maintains (Mnesia) db of webcam images * Erlang maps URLs to plot images & webcam pics * PHP queries (Erlang-coded) URLs for webcam and status graphs

16. Key features * Erlang-to-erlang distributed programming - DMS's code makes graphs; HV's code will serve them - Communicate via Erlang calls/messages across nodes * Reliability - Webcam code written to OTP standards - recovers from everything - Log scraper ditto

17. Key features (2) * Mnesia - The Erlang database: Erlang’s native point of state * RDBMS interface - Log grovelling uses Erlang/MySQL driver * Erlang <-> Unix process communication - Log groveller runs python subprocess (with wrapper) - Erlang uses gnuplot (no wrapper required)

18. Key features (3): Mnesia • The transparently distributed (replicated) DB of erlang • client doesn’t know which node actually executes query/commit • as long as there’s a node alive the system will stay up, rejoining nodes sync • It stores tuples with named fields (the columns); records • The columntype is always the same: “ANY” • Can store any Erlang term in a column • e.g. binary jpeg data; no intelligence required, it just works! • Contents of a “column” may vary between rows • some may consider this an asset, some won’t • It is NOT a RDBMS • see above note • first “column” =:= PrimaryKey, store/retrieve rows based on this • Two table types • set: PrimaryKey => record mapping is 1:1 • bag: PrimaryKey => record mapping is 1:n • Access to the db MUST be done in transactions (which is a good thing)