Control flow structures in Distributed programs

1. Control flow structures in Distributed programs Philippe Demaecker Programming Technology Lab (PROG)Departement Informatica (DINF)Vrije Universiteit Brussel (VUB)

2. About distributed programs • Wide area networks  programs on remote machines can’t be altered • Component wise (application consists of interconnected components) • Components are active

3. About • Examples: • metacrawler  several searchengines • dispatching within HTTPD daemons (to cgi’s) • ... these kind of control flow handlers are often more needed than anticipated

4. Goal • Event handlers are often needed • control flow code hardcoded in program • insert components that were written by other programmers • Which control flow structures are needed to write distributed programs in an easier way? • No AOP (You don’t posses the remote code) • Purpose is to go a level higher than concurrency primitives

5. Working method • Study the control flow of some ‘simple’ programs to extract primitives (e.g. pipeline). • Programs written in Cborg

6. Cborg • Asynchronous sends in Cborg: syntax • <agent>.<callee>(<pars>) • But … no result is returned this way • This can be solved using callbacks:

7. Callbacks in Cborg tecra/ses1 tecra/ses2 a:remotedict(“tecra/ses2”) answer(b)::display(b); a.calcdet([[1,8],[5,6]], agentself()) calcdet(matrx,cb):: cb.answer(matrx)

8. Example SieveComp(next):{ prime: 0; Receive(aNumber)::{if (prime = 0, {SetPrime(aNumber); display(prime)}, {if((not ((aNumber\\prime)=0)), next.Receive(aNumber),void)})}; agentclone(clone())}

9. Example (ctd) c: Collector(); s3: Sievecomp(c); s2: Sievecomp(s3); s1: Sievecomp(s2); g: Generator(s1); g.Start() Problem: what to do if we want to extend the functionality (e.g display between components)? • Write a component that understands Receive, with the same parameter structure. • Creationstructure must be altered

10. Problems … • Components have to understand Receive(...) • Building the structure is too explicit • Collector should be a control flow component • Parameter passing convention is hardcoded ==> control flow too explicit

11. Example (with pipeline) • Primecheck PrimeCheck():{prime: 0; Receive(aNumber,primeTo,nonPrimeTo)::{ if(prime = 0, {SetLocalPrime(aNumber); primeTo.GetResult(prime)}, {if((not ((aNumber\\prime)=0)), nonPrimeTo.GetResult(aNumber), void)})} agentclone(clone());}

12. Example (ctd) • Generator NumberGenerator(from,to, name):{ next: 0; SendInfo(target,info):: target.GetResult(info); Start(target)::{ for(t:from, t:=t+1, t<to, SendInfo(target,t))}; agentclone(clone(), name) }

13. Example (proposed solution) • components:[g,s1,s2,s3] • p:Pipeline(components)) • components:[g,disp,s1,disp,s2,disp,s3,disp] • p:Pipeline(components) >> p.Start(console, ”Receive”,”GetResult”)

14. Pipeline structure

15. FFT using same pipeline CalcFFT(name, stageNr) :{ Calc(coeffs, displayTo, resultTo)::{ <compute coefficients>; resultto.SendToNext(info)} agentclone(clone(), name)} • components:[g, FFT1, FFT2, ...]; • pipeline(components,”Calc”,”SendToNext”); • Same pipeline structure !

16. Addendum • Good use of migration: Assume: components are in Zimbabwe 1. You can declare the pipeline locally and 2. send the pipes to Zimbabwe. => more performant

17. Future Work & Thesis (1) • Study other applications and designs • HTTPD • MVC • Distributed Namespaces • Whiteboards • Design Patterns • Message Queues • Event Dispatchers

18. Future Work & Thesis (2) • Checking out what happens when working with other natives. • Synchronisation natives (sync, send, recv) • Main goal is to detect appropriate control flow structures: • star structure (many to one) • multi casting (one to many) • others (many to many, ...)

19. Pipe (internals) Pipe(n, posPipe, nextAgent,nextNegPipe)::{ SendResultToNext@args::{ info : args[1]; mss : read("rcv." + n + "()"); mss[1] := nextAgent; app : mss[2]; app[2] := [info, posPipe, nextNegPipe]; eval(mss)}; cst : make(16); cst[1] := read(callbackName+"@args"); cst[2] := SendResultToNext[3]; eval(cst); agentclone(clone(), name)};