1 / 53

Ranjit Jhala Rupak Majumdar

Interprocedural Analysis . of. Asynchronous Programs. Ranjit Jhala Rupak Majumdar. Conclusions. Boost your pet Dataflow Analysis to work on Asynchronous Programs. … lets begin at the beginning. main(){ ... async reqs(); ... }. reqs(){ if(r == NULL){

syshe
Download Presentation

Ranjit Jhala Rupak Majumdar

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. InterproceduralAnalysis of AsynchronousPrograms Ranjit Jhala Rupak Majumdar

  2. Conclusions Boost your pet Dataflow Analysis to work on Asynchronous Programs … lets begin at the beginning

  3. main(){ ... async reqs(); ... } reqs(){ if(r == NULL){ async reqs(); return; } rc = malloc(…); if (rc == NULL){ return NO_MEM; } async client(rc,r->id); r = r->next; reqs(); } client(rc) client(*c,id){ ... c->id = id; ... return; } v10 Asynchronous Programs client reqs main global request_list *r; v4 [r==0] [r!=0] v5 v11 v0 v12 rc=malloc() v6 [rc!=0] [rc==0] v1 v13 reqs v7 c->id=id v2 v14 v8 r=r->next v3 v15 v9 reqs reqs

  4. main(){ ... async reqs(); ... } reqs(){ if(r == NULL){ async reqs(); return; } rc = malloc(…); if (rc == NULL){ return NO_MEM; } async client(rc,r->id); r = r->next; reqs(); } client(rc) client(*c,id){ ... c->id = id; ... return; } v10 Asynchronous Programs client reqs main global request_list *r; v4 [r==0] [r!=0] v5 v11 v0 v12 rc=malloc() v6 [rc!=0] [rc==0] v1 v13 reqs v7 c->id=id v2 v14 v8 r=r->next v3 v15 v9 reqs reqs

  5. client(rc) v10 Asynchronous Programs client reqs main v4 v5 v11 v0 v12 v6 Dispatch Location V3 Calls all other functions v1 v13 reqs v7 v2 v14 v8 reqs client v3 v15 v9 reqs reqs

  6. client(rc) v10 Asynchronous Program Execution client reqs main v4 [r==0] [r!=0] PC v5 v11 v0 v12 rc=malloc() v6 [rc!=0] [rc==0] v1 v13 reqs v7 c->id=id v2 v14 v8 reqs client r=r->next v3 v15 v9 reqs reqs

  7. client(rc) v10 Asynchronous Program Execution client reqs main • Async calls stored in set v4 [r==0] [r!=0] v5 v11 v0 v12 rc=malloc() v6 PC [rc!=0] [rc==0] v1 v13 reqs v7 c->id=id v2 v14 Pending Calls reqs v8 reqs client r=r->next v3 v15 v9 reqs reqs

  8. client(rc) v10 Asynchronous Program Execution client reqs main • Async calls stored in set • Execute at dispatch loop PC v4 [r==0] [r!=0] v5 v11 v0 v12 rc=malloc() v6 [rc!=0] [rc==0] v1 v13 reqs v7 c->id=id PC v2 v14 Pending Calls reqs v8 reqs client r=r->next v3 v15 v9 reqs reqs

  9. client(rc) v10 Asynchronous Program Execution client reqs main • Async calls stored in set • Execute at dispatch loop PC v4 [r==0] [r!=0] v5 v11 v0 v12 rc=malloc() v6 [rc!=0] [rc==0] v1 v13 reqs v7 c->id=id v2 v14 Pending Calls v8 reqs client r=r->next v3 v15 v9 reqs reqs

  10. client(rc) v10 Asynchronous Program Execution client reqs main • Async calls stored in set • Execute at dispatch loop • Sync calls exec at call site v4 [r==0] [r!=0] v5 v11 v0 v12 rc=malloc() v6 [rc!=0] [rc==0] v1 v13 PC reqs v7 c->id=id v2 v14 Pending Calls client(…) v8 reqs client r=r->next v3 v15 v9 reqs reqs

  11. client(rc) v10 Asynchronous Program Execution client reqs main • Async calls stored in set • Execute at dispatch loop • Sync calls exec at call site PC v4 [r==0] [r!=0] v5 v11 v0 v12 rc=malloc() v6 [rc!=0] [rc==0] v1 v13 reqs v7 c->id=id v2 v14 Pending Calls client(…) client(…) v8 reqs client r=r->next v3 v15 v9 reqs reqs

  12. client(rc) v10 Asynchronous Program Execution client reqs main • Async calls stored in set • Execute at dispatch loop • Sync calls exec at call site PC v4 [r==0] [r!=0] v5 v11 v0 v12 rc=malloc() v6 [rc!=0] [rc==0] v1 v13 reqs v7 c->id=id v2 v14 Pending Calls client(…) client(…) reqs v8 reqs client r=r->next v3 v15 v9 reqs reqs

  13. client(rc) v10 Asynchronous Program Execution client reqs main • Async calls stored in set • Execute at dispatch loop • Order is non-deterministic • Sync calls exec at call site v4 [r==0] [r!=0] PC v5 v11 v0 v12 rc=malloc() v6 [rc!=0] [rc==0] v1 v13 reqs v7 c->id=id v2 v14 Pending Calls client(…) client(…) reqs v8 reqs client r=r->next PC v3 v15 v9 reqs reqs

  14. client(rc) v10 Asynchronous Program Execution client reqs main • Async calls stored in set • Execute at dispatch loop • Order is non-deterministic • Sync calls exec at call site v4 [r==0] [r!=0] PC v5 v11 v0 v12 rc=malloc() v6 [rc!=0] [rc==0] v1 v13 reqs v7 c->id=id v2 v14 Pending Calls client(…) reqs v8 reqs client r=r->next PC v3 v15 v9 reqs reqs

  15. client(rc) v10 Asynchronous Program Execution client reqs main PC • Async calls stored in set • Execute at dispatch loop • Order is non-deterministic • Sync calls exec at call site v4 [r==0] [r!=0] v5 v11 v0 v12 rc=malloc() v6 [rc!=0] [rc==0] v1 v13 reqs v7 c->id=id v2 v14 Pending Calls reqs v8 reqs client r=r->next PC v3 v15 v9 reqs reqs

  16. Asynchronous Programs Why? Latency hiding and Parallelism Domains: • Distributed Systems • Web Servers • Embedded Systems • Discrete-event simulation Async calls stored in set • Execute at dispatch loop Sync calls execute at call site Languages and Libraries: • Java + Atomic Methods • LibAsync, LibEvent, … • NesC

  17. client(rc) v10 Q: How to Analyze Async Programs ? client reqs main Prove dereference of c is safe i.e. c not null at v13 v4 [r==0] [r!=0] v5 v11 v0 v12 rc=malloc() v6 v1 v13 [rc!=0] [rc==0] Dataflow Facts reqs v7 c->id=id (Must) Non-null (May) Null v2 v14 v8 r rc c r r client reqs r=r->next rc rc v3 v15 v9 c c reqs reqs

  18. client(rc) v10 Verification via Dataflow Analysis client reqs main Prove Flow fact holds at v13 v4 [r==0] [r!=0] c v5 v11 v0 v12 rc=malloc() v6 v1 v13 [rc!=0] [rc==0] Dataflow Facts reqs v7 c->id=id (Must) Non-null (May) Null v2 v14 v8 r rc c r r client reqs r=r->next rc rc v3 v15 v9 c c reqs reqs

  19. client(rc) v10 Dataflow Analysis client reqs main 1st Attempt Treat asynchronous calls as synchronous r rc v4 [r==0] [r!=0] r rc v5 v11 v0 v12 r r c rc=malloc() r rc v6 [Sharir-Pnueli 80] [Reps-Horwitz-Sagiv 95] r v1 v13 [rc!=0] [rc==0] r c r rc reqs v7 c->id=id v2 v14 v8 client reqs r=r->next Verification “works” … but unsoundly deduces global r is non-null! v3 v15 v9 reqs reqs

  20. client(rc) v10 Dataflow Analysis client reqs main 1st Attempt Treat asynchronous calls as synchronous v4 [r==0] [r!=0] v5 v11 v0 v12 rc=malloc() v6 Unsound Global rmay change between call, dispatch v1 v13 [rc!=0] [rc==0] r c reqs v7 c->id=id v2 v14 v8 client reqs Idea Separately track local and global facts r=r->next v3 v15 v9 reqs reqs

  21. client(rc) v10 Dataflow Analysis client reqs main 2nd Attempt Only execute async calls from dispatch location v4 [r==0] [r!=0] v5 v11 v0 v12 rc=malloc() v6 v1 v13 [rc!=0] [rc==0] reqs v7 c->id=id v2 v14 v8 client reqs r=r->next v3 v15 v9 reqs reqs

  22. client(rc) v10 Dataflow Analysis client reqs main 2nd Attempt Only execute async calls from dispatch location v4 [r==0] [r!=0] v5 v11 v0 v12 rc=malloc() v6 Imprecise Initial value of formals ? All values (>) too coarse v1 v13 [rc!=0] [rc==0] reqs v7 c->id=id v2 v14 v8 client reqs Idea Track pending calls with formals at call-site r=r->next v3 v15 v9 reqs reqs

  23. Encoding Pending Calls as Flow Facts Idea: Counters - For each kind of async call £ input fact: Count number of pending calls of kind - Expanded DFA facts: Dataflow facts £ Counters reqs 1 client, 5 client, 0 Idea Track pending calls with formals at call-site c c

  24. Key: Combining two Analyses Expanded DFA facts: Dataflow facts £ Counters r not null, rc maybe null, and pending calls: 1 to reqs, 5 to client (arg non-null) 0 to client (arg null) reqs 1 client, 5 client, 0 c r rc c

  25. Key: Combining two Analyses Expanded DFA facts: Dataflow facts £ Counters Counters: Restrict analysis to valid inter-procedural paths i.e. feasible sequences of async calls / dispatches Dataflow facts: Perform desired analysis over restricted paths

  26. client(rc) v10 Dataflow Analysis client reqs main 3rd Attempt Count # pending calls of each kind v4 [r==0] [r!=0] v5 v11 v0 v12 rc=malloc() v6 v1 v13 [rc!=0] [rc==0] reqs v7 c->id=id v2 v14 reqs 1 client, 5 client, 0 v8 client reqs c r rc r=r->next v3 v15 c v9 reqs reqs

  27. client(rc) v10 Dataflow Analysis client reqs main 3rd Attempt Count # pending calls of each kind v4 [r==0] [r!=0] v5 v11 v0 v12 rc=malloc() v6 Non-Terminating #Pending calls unbounded due to recursion, loops v1 v13 [rc!=0] [rc==0] reqs v7 c->id=id v2 v14 v8 client reqs Idea Approximate via Abstract counters r=r->next v3 v15 v9 reqs reqs

  28. reqs 1 client, 5 client, 0 c c Dealing with Unbounded Async Calls Over-Approximations : k1-Abstract Counters - For each async call £ input fact: Abstractly count number of pending calls of each kind - Values > k, abstracted to infinity 1 - Finite counter values = {0,1,…,k,1} - FiniteDFA facts: Dataflow facts £ k1-Abs counters - Analysis Terminates E.g. k =1 1

  29. Recall: Combining two Analyses Which interprocedural paths do k1 – Abstractions consider ? Expanded DFA facts: Dataflow facts £ Counters Counters: Restrict analysis to valid inter-procedural paths i.e. feasible sequences of async calls / dispatches Dataflow facts: Perform desired analysis over restricted paths

  30. reqs 1 client, 1 client, 0 PC c c client client reqs reqs reqs reqs reqs client Example: (k=1)1 Abstraction 1 2 3 4 5 6 7 8

  31. reqs 1 client, 1 client, 0 PC c c client client reqs reqs reqs reqs reqs client client Example: (k=1)1 Abstraction 1 2 3 4 5 6 7 8

  32. reqs 1 client, 1 client, 0 PC c c client client reqs reqs reqs reqs reqs client client client Example: (k=1)1 Abstraction 1 2 3 4 5 Valid 6 7 8 9 10

  33. reqs 1 client, 1 client, 0 PC c c client client reqs reqs reqs reqs reqs client client client Example: (k=1)1 Abstraction 1 Over-Approx: k1-Abstraction - Considers all valid paths - Plus, some invalid paths - DFAon superset of valid paths 2 3 4 5 Valid Invalid 6 7 No matching async call 8 9 10

  34. Dealing with Unbounded Async Calls Over-Approx: k1-Abstraction - Considers all valid paths - Plus, some invalid paths - DFAon superset of valid paths Idea How bad is over-approximation ? Find out using under-approximation! Over-approximate/Sound - Works for example … but imprecise in general - How to do exact DFA over set of valid paths ?

  35. reqs 1 client, 5 client, 0 c c Computing Under-Approximate Solutions Under-Approximations: k-Abstract Counters - For each async call £ input fact: Abstractly count number of pending calls of each kind - Values > k, abstracted to k - Effect: All calls after k are “dropped” - Finite counter values = {0,1,…,k} - Finite dataflow facts £ k-Abs counters, ) termination E.g. k =1 1

  36. Key: Combining two Analyses Which interprocedural paths do k – Abstractions consider ? Expanded DFA facts: Dataflow facts £ Counters Counters: Restrict analysis to valid inter-procedural paths i.e. feasible sequences of async calls / dispatches Dataflow facts: Perform desired analysis over restricted paths

  37. reqs 0 client, 1 client, 0 PC c c client client reqs reqs reqs Example: (k=1)Abstraction 1 2 3 4 5 Already (k=1) pending calls with given input fact Call at step 5 is “dropped” Only one pending call !

  38. reqs 0 client, 1 client, 0 PC c c client client reqs reqs reqs Example: (k=1)Abstraction 1 2 3 4 5 Already (k=1) pending calls with given input fact Call at step 5 is “dropped” Only one pending call !

  39. reqs 1 client, 1 client, 0 PC c c client client reqs reqs reqs reqs reqs client Example: (k=1) Abstraction 1 2 3 4 5 6 7 8 Only one call in (k=1)-Abstract pending set None remain after this dispatch

  40. reqs 1 client, 0 client, 0 PC c c client client reqs reqs reqs reqs reqs client client Example: (k=1) Abstraction 1 2 3 4 5 6 7 8 Only one call in (k=1)-Abstract pending set None remain after this dispatch Exists matching async call But no more calls in (k=1)-abstract pending set !

  41. client client reqs reqs reqs reqs reqs client client Example: (k=1) Abstraction 1 2 Valid but ignored by k-abstraction 3 4 5 6 7 8 9 ? PC Exists matching async call But no more calls in (k=1)-abstract pending set !

  42. client client client client Example: (k=1) Abstraction 1 2 Valid but ignored by k-abstraction 3 Under-Approx: k-Abstraction - Ignoresall invalid paths - andsome valid paths - DFA on subset of valid paths - Under-Approx. DFA solution 4 5 6 7 8 9

  43. K1-Abstract DFA Over-Approx Paths K-Abstract DFA Under-Approx Paths What we have: For all K … Both Computable Via Standard DFA [Sharir-Pnueli 80] [Reps-Horwitz-Sagiv 95] Require Exact DFA on Valid Paths

  44. K1-Abstract DFA Over-Approx Paths K-Abstract DFA Under-Approx Paths Increase K to Increase Precision K++ Require Exact DFA on Valid Paths K++ But how to compute exact DFA Solution ?

  45. K1-Abstract DFA Over-Approx Paths K-Abstract DFA Under-Approx Paths Theorem: There exists magic K … Require Exact DFA on Valid Paths K1-Abstract DFA Over-Approx K-Abstract DFA Under-Approx Approximations Converge! To Exact DFA on Valid Paths

  46. Algorithm AsyncDFA(){ k := 0 repeat over := DFA(k1-Counter); under := DFA(k-Counter); k := k+1; until (over = under); return over; } Require Exact DFA on Valid Paths DFA = Interprocedural Analysis via Summaries [Sharir-Pnueli 80, Reps-Horwitz-Sagiv 95]

  47. Proof “Obvious ? Finitely many solutions + monotonicity implies computable fixpoint …” Alas, over- and under- approximations could converge to different fixpoints …

  48. Proof Buzzwords • Counters are Well Quasi Ordered • Pre* exists • Initial configurations reaching a location • Constructable via complex backward algorithm • Petri Nets: [Esparza-Finkel-Mayr 98] • Async Programs: [Sen-Vishwanathan 06] • Magic k exists due to existence of Pre* • Simple forward algorithm • [PLDI 04, Geeraerts-Raskin-van Begin 04]

  49. Application: Safety Verification Ground Dataflow Facts = Predicate Abstraction Implemented on BLAST framework - Lazy Interprocedural DFA [POPL 02] • Predicates automatically found via Counterexample Formula Interpolation [POPL 04] • Reduced Product of Predicate Abstraction, Counter lattice [FSE 05]

  50. Preliminary Experiments • C LibEvent Programs • Load Balancer • Network Simulator • Properties • Buffer Overflow • Null Pointer Dereference • Protocol State • Several proved, bugs found

More Related