Effective Static Deadlock Detection

1. Effective Static Deadlock Detection MayurNaik, Chang-Seo Park, KoushikSen and David Gay ICSE’09 Presented by Alex Kogan 27.04.2014

2. Outline • Introduction • Suggested Solution • Evaluation • Related work • Conclusions • Discussion

3. Introduction

4. What is a Deadlock? • An unintended condition in a shared-memory, multi-threaded program in which: • a set of threads block forever • because each thread in the set waits to acquire a lock being held by another thread in the set

5. Example // thread t1 sync (l1) { sync (l2) { … } } l1 t1 t2 l2 // thread t2 sync (l2) { sync (l1) { … } }

6. Motivation • Today’s concurrent programs are rife with deadlocks • 6,500 (~3%) of bugs reported in Sun’s bug database are deadlocks • Difficult to detect • Triggered non deterministically on specific thread schedules • Fail-stop behavior not guaranteed • Fixing other concurrency bugs like races can introduce new deadlocks

7. Previous Work • Dynamic approaches • Unsound • Inapplicable to open programs • Rely on input data • Static approaches(Soundness) • Type systems • Model checking • Dataflow analysis

8. Challenges l1 l1 l4 • Deadlock freedom is a complex property • can t1,t2 denote different threads? • can l1,l4 denote same lock? • can t1 acquire locks l1l2? • some more … t1 t1 t2 t2 l2 l2 l3

9. Example: jdk1.4 java.util.logging class LogManager { static LogManager manager =LogManager(); 155: Hashtable loggers = new Hashtable(); 280: boolean addLogger(Logger l) { String name = l.getName(); if (!loggers.put(name, l)) return false; // ensure l’s parents are instantiated for (...) { String pname = ...; 314: Logger.getLogger(pname); } return true; } 420: Logger getLogger(…) { return (Logger) loggers.get(name); } } class Logger { 226: static Logger getLogger(…) { LogManager lm = LogManager.manager; 228: Logger l = lm.getLogger(name); if (l == null) { l = new Logger(...); 231: lm.addLogger(l); } return l; } } l4 l1 l2 class Harness { static void main(String[] args) { 11: Thread() { void () { 13: Logger.getLogger(...); }}.start(); 16: Thread() { void () { 18: LogManager.manager.addLogger(...); }}.start(); } t1 l3 t2

10. Solution Outline • List set of conditions for a deadlock • Check each condition separately • Utilize existing static analyses for each condition • Report candidates that satisfy all conditions • Only find deadlocks involving 2 locks(threads)

11. Solution Java deadlock detection algorithm

12. Take II:Effective Static Deadlock Detection MayurNaik, Chang-Seo Park, KoushikSen and David Gay ICSE’09 Presented by Alex Kogan 11.05.2014

13. Context • Context in the seminar • Concurrency present challenges • We have seen mostly C analyses • OO?

14. Object-Sensitivity • Invented by A. Milanovaet al. [ISSTA 2002] • Produces points-to and call graph

15. Related Work • Anderson’s Algorithm • Flow & context insensitive • Base for speed & accuracy comparison • Flow Sensitivity • (k) CFA – Control Flow Analysis • Context sensitive

16. Idea • OO paradigm presents special needs • Objects play a major role • Call graph defined by inheritance • Fast (practical) • Adjustable

17. Object Sensitivity • Comparable to CFA • Uses the receiver object instead of the caller • Saves up to k receiving (this) objects • Uses them to calculate the context of each pointer

18. thisA.m.f=q o1 o1 f a o1 o2 thisA.m o1 qA.m o1 thisA.m.f=q o3 this.f=q ; o3 thisA.m o3 qA.m o3 aa o3 o4 f Example: Object-Sensitive Analysis class Y extends X {} class A { X f; void m(X q) { this.f=q ;}} A a = new A() ; a.m(new X()) ; A aa = new A() ; aa.m(new Y()) ;

19. f a o1 qA.m o1 thisA.m o1 thisA.m o2 qA.m o2 aa o2 o3 f Example: Object-Sensitive Analysis Loss Of Information for k=1 class A { X f; void m(A q) { this.f=q ;}} A[] a= new A[2]; a[0] = new A() ; a[1] = new A() ; for(inti=0; i<2; i++) a[i].m(new A()) ; o3 f qA.m o3 o4 f a[0].f.m(new A()) ;

20. f a o1 qA.m o1 thisA.m o1 thisA.m o2 qA.m o2 aa o2 o3 f Example: Object-Sensitive Analysis Success For k=2 class A { X f; void m(A q) { this.f=q ;}} A[] a= new A[2]; a[0] = new A() ; a[1] = new A() ; for(inti=0; i<2; i++) a[i].m(new A()) ; o3 f o3,o1 qA.m o4 a[0].f.m(new A()) ;

21. k-Object-Sensitivity Computation • Starting point • consider main method and class initializers reachable. • Add points-to for every new object (at local var v) • objects created in context c = (o = [h1,…,hk], m) • add ((o,m), v, [this, h1, …, hk-1]) • Add to call graph for every method call • for a instance method call v.n(...) add ([h1,…,hk], n’) • for context in which v is reachable • if n’= mstartthen also deem reachable n’’ = mrun • for a static method call n(..) add ([], n)

22. k-Object-Sensitivity Cont. • Produce the following relations • cg (C x C) • pairs of (,) x (, ) • method (in context ) is reachable from (with context ) • pt(C x V x O) • C – context of the call to create • V – pointer to objects(local variable) • O – abstract object • finite sequence of object allocation sites • - object allocation site • The rest are “this” objects of methods where o was allocated

23. thisA.m.f=q o1 o1 thisA.m.f=q o3 this.f=q ; o3 Building Call Graph & Points To class Y extends X {} class A { X f; void m(X q) { this.f=q ;} A a = new A() ; a.m(new X()) ; A aa = new A() ; aa.m(new Y()) ; Call Graph Points-To (([], mmain), v1, [h1]) (([], mmain), ([h1], m)) (([h1], m), v2, [h2]) (([], mmain), ([h3], m)) (([], mmain), v3, [h3]) (([h3], m), v2, [h4])

24. Example: jdk1.4 java.util.logging class LogManager { static LogManager manager =LogManager(); 155: Hashtable loggers = new Hashtable(); 280: boolean addLogger(Logger l) { String name = l.getName(); if (!loggers.put(name, l)) return false; // ensure l’s parents are instantiated for (...) { String pname = ...; 314: Logger.getLogger(pname); } return true; } 420: Logger getLogger(…) { return (Logger) loggers.get(name); } } class Logger { 226: static Logger getLogger(…) { LogManager lm = LogManager.manager; 228: Logger l = lm.getLogger(name); if (l == null) { l = new Logger(...); 231: lm.addLogger(l); } return l; } } l4 l1 l2 class Harness { static void main(String[] args) { 11: Thread() { void () { 13: Logger.getLogger(...); }}.start(); 16: Thread() { void () { 18: LogManager.manager.addLogger(...); }}.start(); } t1 l3 t2

25. Example: Step 1 • pt= {(([], mmain), v1, [h1]),(([], mmain), v2, [h2])} • cg = {(([], mmain), ([h1], mstart)),(([], mmain), ([h2], mstart))} • Deem reachable • ([h1], m4) • ([h2], m5) • m4 ,m5 are the run methods of the threads

26. Example: Step 2* • What is added to the call graph? • (([h1], m4) x ([], m1)) – Logger.getLogger() • (([h2], m5) x ([h3], m2)) – LogManager.lm.addLogger() • How about point-to? • Nothing new to be added in this step • Next step will add all local variables of functions getLogger() and addLogger()

27. Object Sensitivity: Analysis Time (k=2)

28. A Word About Precision:Modified Objects Per Statement OBJECT SENSITIVE ANDERSEN jb jess sablecc raytrace Average

29. For The Main Event Deadlock Detection

30. A Deadlock l1 t1 t2 l2

31. Deadlock Computation (Java) • Lock aliasing: • mayAlias(l1,l2) ∃o : (l1,sync(l1),o) ∈ pt ∧ (l2,sync(l2),o) ∈ pt • Lock-set aliasing • mayAlias(L1,L2) ∃l1 ∈ L1,l2 ∈ L2 : mayAlias(l1,l2) • Reachability:

32. Deadlock Computation (Java) • Threads = • Locks = • FinalDeadlocks = }

33. Call Graph vs Threads & Locks mmain … h2 mstart h2 mstart h2 mrun h1 mrun … … h3 madd [] mget …

34. Example: jdk1.4 java.util.logging class LogManager { static LogManager manager =LogManager(); 155: Hashtable loggers = new Hashtable(); 280: boolean addLogger(Logger l) { String name = l.getName(); if (!loggers.put(name, l)) return false; // ensure l’s parents are instantiated for (...) { String pname = ...; 314: Logger.getLogger(pname); } return true; } 420: Logger getLogger(…) { return (Logger) loggers.get(name); } } class Logger { 226: static Logger getLogger(…) { LogManager lm = LogManager.manager; 228: Logger l = lm.getLogger(name); if (l == null) { l = new Logger(...); 231: lm.addLogger(l); } return l; } } l4 l1 l2 class Harness { static void main(String[] args) { 11: Thread() { void () { 13: Logger.getLogger(...); }}.start(); 16: Thread() { void () { 18: LogManager.manager.addLogger(...); }}.start(); } t1 l3 t2

35. Deadlock computation example • Threads = • Locks =

36. Deadlock Computation - Tuples

37. Sound conditions for deadlock original tuples Reachable tuples Aliasing tuples Escaping tuples Parallel tuples All Deadlocks

38. Unsound conditions for deadlock Parallel tuples Reentrant tuples All True deadlocks Guarded tuples

39. Condition 1: Reachable l1 l1 l4 l4 • In some execution: • can a thread abstracted by t1 reach l1 • and after acquiring lock at l1, proceed to reach l2 while holding that lock? • and similarly for t2, l3, l4 • Solution: Use call-graph analysis t1 t1 t2 t2 l2 l2 l3 l3

40. Reachable Cont. • Running example: • easy to see both pass

41. Condition 2: Aliasing l1 l1 l4 • In some execution: • can a lock acquired at l1 be the same as a lock acquired at l4? • and similarly for l2, l3 • Solution: Use may-alias analysis t1 t2 l2 l2 l3

42. Aliasing Cont. • Running example: • Pass because is the context of both, which satisfies the may-alias condition

43. Condition 3: Escaping l1 l1 l4 l4 • In some execution: • can a lock acquired at l1 be thread-shared? • and similarly for each of l2, l3, l4 • Solution: Use thread-escape analysis t1 t2 l2 l2 l3 l3

44. Escaping Analysis Gist • A method for determining the scope of pointers • Thread-escape • has a pointer from a “thread spawning” site. • may be referenced by another thread-shared object • static objects • Produce relation • (c, v) • argument v of object c can be access by more than 1 thread • their thread escape is fairly imprecise

45. Escaping Cont. • Running Example: • LogManager.manager & Logger.class are static • So they escape everywhere and both tuples pass

46. Example: jdk1.4 java.util.logging class LogManager { static LogManager manager =LogManager(); 155: Hashtable loggers = new Hashtable(); 280: boolean addLogger(Logger l) { String name = l.getName(); if (!loggers.put(name, l)) return false; // ensure l’s parents are instantiated for (...) { String pname = ...; 314: Logger.getLogger(pname); } return true; } 420: Logger getLogger(…) { return (Logger) loggers.get(name); } } class Logger { 226: static Logger getLogger(…) { LogManager lm = LogManager.manager; 228: Logger l = lm.getLogger(name); if (l == null) { l = new Logger(...); 231: lm.addLogger(l); } return l; } } l4 l1 l2 class Harness { static void main(String[] args) { 11: Thread() { void () { 13: Logger.getLogger(...); }}.start(); 16: Thread() { void () { 18: LogManager.manager.addLogger(...); }}.start(); } t1 l3 t2

47. Condition 4: Parallel l1 l1 l4 l4 • In some execution: • can different threads abstracted by t1 and t2 • simultaneously reach l2 and l4? • Solution: Use may-happen-in-parallel analysis ≠ t1 t1 t2 t2 l2 l2 l3 l3

48. May-happen-in-parallel Gist • Filters: • Exact same threads • may-alias threads • Threads in child/parent relationship • by annotations • Produces relation • may be running in parallel when reaches method in context

49. May-happen-in-parallel Cont. • Running example: • Nothing prevents and from running in parallel • both tuples pass • What will it remove?

50. Condition 5: Non-reentrant l1 l1 l4 l4 • Property: In some execution: • can a thread abstracted by t1 acquire a non-reentrant lock at l1 • and, while holding that lock, proceed to acquire a non-reentrant lock at l2? • and similarly for t2, l3, l4 • Solution: Use call-graph + may-alias analysis • Unsound: Negation of over-approximation ≠ ≠ t1 t1 t2 t2 l2 l2 l3 l3