Concurrent Programming with Futures

1. Concurrent Programming with Futures Presented by Michael Hicks Oregon Summer School 2006 on Concurrent and Distributed Software

2. This Lecture • Introducing Futures • Programming model • Implementation in Multilisp (Halstead 1985, Mohr et al 1991, Flanagan and Felleisen 1995) • Futures in Java • Java.util.concurrent • Transparency with static typing (Pratikakis et al, 2004) • Safety (Welc et al, 2005)

3. Thanks • Adam Welc at Purdue for most of the Safe Futures slides

4. Scheme Merge Sort (define (split x) …) (define (merge x y) … (car x) …) (define (mergesort x) (let ((y,z) (split x)) (merge (mergesort y) (mergesort z)))) How to make parallel?

5. Explicit Approach • Threads, Message Passing • Problems • Message passing requires partitioning the data among different address spaces • Must write code to exploit resources of underlying platform • Significant code changes

6. Implicit Approach • Rely on the compiler to figure out opportunities for parallelism • Problems • Really hard! • Instruction-level and loop-level parallelism can be inferred, but • Inferring larger “subroutine”-level parallelism has had less success.

7. Middle Ground: Futures • Use future annotation [Halstead 85] • (future e) indicates e may run concurrently with parent • Benefits • Notationally lightweight • Sequential algorithm still manifest • Implement to let concurrency be determined by the run-time system, based on system resources • Coordination between concurrent computations is transparent

8. Where to annotate? (define (split x) …) (define (merge x y) … (car x) …) (define (mergesort x) (let ((y,z) (split x)) (merge (mergesort y) (mergesort z)))) No - result is used immediately in following call

9. Where to annotate? (define (split x) …) (define (merge x y) … (car x) …) (define (mergesort x) (let ((y,z) (split x)) (merge (mergesort y) (mergesort z)))) Yes - recursive calls can operate in parallel

10. Multilisp Merge Sort (define (split x) …) (define (merge x y) … (car x) …) (define (mergesort x) (let ((y,z) (split x)) (merge (future (mergesort y) (future (mergesort z)))))

11. Basic Implementation Approach • (future e) • fork a new thread T to evaluate e • return a proxy p to the parent • called a future or promise • T stores result of e into p • Run-time system extracts result from p when accessed by the parent • Called a touch or claim

12. Implementing Touches Could be a future… (define (merge x y) … (car x) …) • Futurized implementation of (car x) (if (pair? (touch x)) (get first elem of x) (error)) • Where (touch x) is (if (future? x) (get x) x) Blocks until result has been computed

13. Optimization I • Forking a thread per future could be expensive and without advantage • Particularly if not many CPUs • Idea: only use as many threads as there are processors [Mohr et al 91] • At a future call, use idle thread, if any • Otherwise, continue using current thread • Save continuation on a separate queue • When a thread would block, save the current continuation and grab one from the queue

14. Optimization II • Once a future computation completes, its result is immutable • Proxy and further touches redundant • Thus • Use garbage collector to throw away the proxy and replace with the result [Halstead 85] • Avoid touching at all if static analysis can prove it’s unnecessary [Flanagan & Felleissen 95]

15. What about side effects? (let ((x 1) (_ (set! x 2)) x) (let ((x 1) (_ (future (set! x 2)) x) • Sequential version: 2 • Parallel version: either 1 or 2

16. Safety and Concurrency • Most Multilisp code is functional • No worry about inconsistencies • Non-functional code • Encapsulate abstractions that are mutable • Synchronize all accesses • Like “fully synchronized” Vector class in Java • What if the programmer makes a mistake? • Will look at this later in the talk

17. Futures in Java • Java is not Lisp/Scheme • Static typing • Side-effects are far more prevalent • Approach • Static analysis and transformation [Pratikakis et al 2004] • Detect safety problems at run-time [Welc et al 2005]

18. Example: HTTP handler procRequest(Socket sock) { Buffer in = readBuf(sock); Request req = translate(in); Buffer out = process(req); writeBuf(sock,out); } Request translate(Buffer in) { Request result; … in.foo() … return result; } …

19. Sample execution (original) procRequest(Socket sock) Buffer in = readBuf(sock) Request req Buffer out Socket

20. Read the buffer procRequest(Socket sock) Buffer in = readBuf(sock) Request req Buffer out Socket readBuf(sock) result = … return result;

21. Read the buffer procRequest(Socket sock) Buffer in = readBuf(sock) Request req Buffer out Socket readBuf(sock) result = … return result; String

22. Return it procRequest(Socket sock) Buffer in = readBuf(sock) Request req Buffer out Socket readBuf(sock) result = … return result; String

23. Return it procRequest(Socket sock) Buffer in = Request req Buffer out Socket String

24. Next call … procRequest(Socket sock) Buffer in = Request req = Buffer out translate(in) Request result; … in.foo() … return result; String

25. Suppose we had future procRequest(Socket sock) { Buffer in = future readBuf(sock); Request req = future translate(in); Buffer out = future process(req); writeBuf(sock,out); }

26. Sample execution (async) Socket procRequest(Socket sock) Buffer in = future readBuf(sock); Request req = Buffer out

27. Read the buffer in new thread Socket procRequest(Socket sock) Buffer in = Request req = Buffer out spawn thread readBuf(sock) result = … return result;

28. Placeholder to caller Socket procRequest(Socket sock) Buffer in = Request req = Buffer out Future readBuf(sock) result = … return result;

29. Calculate result in child Socket procRequest(Socket sock) Buffer in = Request req = Buffer out Future readBuf(sock) result = … return result; String

30. Store in placeholder Socket procRequest(Socket sock) Buffer in = Request req = Buffer out Future readBuf(sock) result = … return result; String

31. Child finished Socket procRequest(Socket sock) Buffer in = Request req = Buffer out Future String

32. Next call procRequest(Socket sock) Buffer in = Request req Buffer out Future spawn thread translate(Buffer in) Request result; … in.foo() … return result; String

33. Problems procRequest(Socket sock) Buffer in = Request req Buffer out Future spawn thread translate(Buffer in) Request result; … in.foo() … return result; String Callee and caller type not correct (should be Future or Object)

34. Problems procRequest(Socket sock) Buffer in = Request req Buffer out Future spawn thread translate(Buffer in) Request result; … in.foo() … return result; String Callee operations on argument assume a Buffer, not a Future

35. Problems procRequest(Socket sock) Buffer in = Request req Buffer out Future spawn thread translate(Buffer in) Request result; … map.add(in,result) … return result; String Callee operations might violate transparency

36. java.util.concurrent • Concurrency library in Java 1.5 public interface Future<T> { T get(); … } public class FutureTask<T> implements Future<T> { … }

37. java.util.concurrent • Could convert our HTTP program by hand to use this library, but • Would take a lot of code rewriting • Adjust the types, insert code to spawn the thread, to extract the underlying object from the future when needed, catch any exceptions that could be thrown … • Makes it hard to change policies later • What if I later want only one of the methods to be async? • Might result in inadvertent transparency violation

38. Proxy Design Pattern • The proxy and object share an interface • Addresses typing and code problems, but • Still might have to change the program to introduce an interface type, rather than the concrete type • Interfaces only name methods • Thus field accesses disallowed • Does not solve the transparency problems • Still can use ==, instanceof, etc. to distinguish between the object and its proxy

39. Solution:Proxy Programming Framework[Pratikakis et al 2004] • User indicates • where proxies are introduced, e.g. by future annotations on method calls. • what to do when a proxy’s underlying object is required, e.g. when calling a method or extracting a field from a proxy • An automatic program transformation inserts necessary code • For proxy introduction and coercion, avoiding transparency violations

40. Benefits • No code changes needed by hand • Policies can be changed easily • Prevents violations of transparency • Has applications beyond futures • Tracking of security-sensitive data • Not-null types • Stack allocation of objects

41. Summary of Approach • Formalization of analysis and transformation • Formally proven correct • Prototype implementation • Built on the SOOT Java bytecode analysis toolkit • Experimental evaluation, considering • Analysis running time • Quality of generated code

42. Three-Stage Transformation • Inference • Generate constraint graph describing how proxies could flow through the program. • Constraint solving • Solve the constraints, identifying where coercions are needed. • Transformation • Rewrite any classes requiring coercions, type changes, etc.

43. Inference • Each type has qualifierproxy or nonproxy • Like final, but never appears in source programs • proxy indicates the value may be a proxy • nonproxy indicates it is definitely not a proxy • nonproxy< proxy • Qualifier inference is used to assign qualifiers to types in the program, based on • Where proxies are introduced • Where non-proxies are required • How values flow between these locations

44. Inference • Whenever a nonproxy is required, e.g. to call a method, the analysis notes that the value may need to be coerced • E.g., get the underlying object from a future • Coercions are flow-sensitive • Once we check at runtime that a value is a non-proxy, we can assume it is from thereon • Like touch optimization in Multilisp • Can discard placeholder and avoid later touches

45. Constraint Solving • Standard • Based on graph reachability • If a possible proxy indeed flows to a location requiring a nonproxy, there will be a path between the two in the graph. • Requires a coercion as proxy≤ nonproxy

46. Transformation • For each class that • Requires a coercion • Introduces a proxy • … rewrite the class as necessary to insert code to implement them • Code provided by the user • Must avoid transparency violations • Forward calls to .equals(), .hashcode(), etc.

47. Before Analysis: procRequest procRequest(Socket sock) { Buffer in = future readBuf(sock); Request req = future translate(in); Buffer out = future process(req); writeBuf(sock,out); }

48. Inference constraints procRequest(Socket sock) { Buffer in = proxy readBuf(sock); Request req = proxy translate(in); Buffer out = proxy process(req); writeBuf(sock,out); } To method body for translate

49. After transformation procRequest(Socket sock) { Object in = new Proxy { private Object result; public void run() { result = readBuf(sock); } public synchronized Object get() { … return result; } public bool equals(Object o) { return get().equals(o); } }(); TPE.run((Runnable)in); Object req = new Proxy{ …translate(in)… Object out = new Proxy { …process(req)… writeBuf(sock,out); }

50. Before Analysis: translate Request translate(Buffer in) { Request result; … in.foo() … return result; }