1 / 45

Symbolic Execution A quest for nails

Symbolic Execution A quest for nails. Willem Visser Stellenbosch University. Overview. Yesterday Classic symbolic execution Enhanced to use concrete results Today String domain Infinite loops Tomorrow Probabilities . How do we get there ?. How did we get here ?.

oren
Download Presentation

Symbolic Execution A quest for nails

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Symbolic ExecutionA quest for nails Willem VisserStellenbosch University

  2. Overview • Yesterday • Classic symbolic execution • Enhanced to use concrete results • Today • String domain • Infinite loops • Tomorrow • Probabilities

  3. How do we get there? How did we get here?

  4. void test(int x, int y) { if (x > 0) { if (y == hash(x)) S0; else S1; if (x > 3 && y > 10) S3; else S4; } } How do we obtain Statement Coverage?

  5. void test(int x, int y) { if (x > 0) { if (y == hash(x)) else S1; if (x > 3 && y > 10) S3; else S4; } } How do we obtain Statement Coverage?

  6. void test(int x, int y) { if (x > 0) { if (y == hash(x)) S0; else S1; if (x > 3 && y > 10) S3; else S4; } } int hash(x) { if (0<=x<=10) return x*10; else return 0; } Random Inputs might work if you are moderately lucky But there is a better way! Where you don’t need to win the Lottery

  7. Symbolic Execution test(X,Y) void test(int x, int y) { if (x > 0) { if (y == hash(x)) S0; else S1; if (x > 3 && y > 10) S3; else S4; } } int hash(x) { if (0<=x<=10) return x*10; else return 0; } [ X > 0 ] [ X > 0 ] hash (X) [ X>10 & … ] … [ 0<X<=10 & Y=X*10 ] S0 [ 0<X<=10 & Y!=X*10 ] S1 [ 3<X<=10 & 10<Y!=X*10] S3 [ 3<X<=10 & 10<Y=X*10] S3 [ 0<X<=10 & Y=X*10 & ! (X>3 & Y>10) ] S4 [ 0<X<=10 & Y!=X*10 & ! (X>3 & Y>10) ] S4 [ X > 0 ] hash (X) [ 0<X<=10 ] ret X*10 [ X>10] ret 0

  8. Symbolic Execution test(X,Y) void test(int x, int y) { if (x > 0) { if (y == hash(x)) S0; else S1; if (x > 3 && y > 10) S3; else S4; } } int hash(x) { if (0<=x<=10) return x*10; else return 0; } [ X > 0 ] [ X > 0 ] hash (X) Solve [ X>10 & … ] … X=1,Y=10 [ 0<X<=10 & Y=X*10 ] S0 Solve [ 0<X<=10 & Y!=X*10 ] S1 X=1,Y=0 Solve X=4,Y=11 [ 3<X<=10 & 10<Y=X*10] S3 [ 3<X<=10 & 10<Y!=X*10] S3 Solve X=1,Y=10 [ 0<X<=10 & Y=X*10 & ! (X>3 & Y>10) ] S4 [ 0<X<=10 & Y!=X*10 & ! (X>3 & Y>10) ] S4 [ X > 0 ] hash (X) [ 0<X<=10 ] ret X*10 [ X>10] ret 0

  9. One of the basic blocks in the Binomial Heap implementation required a minimum sequence of 13 API calls to be covered private void merge(BinomialHeapNode binHeap) { BinomialHeapNode temp1 = Nodes, temp2 = binHeap; while ((temp1 != null) && (temp2 != null)) { if (temp1.degree == temp2.degree) { BinomialHeapNode tmp = temp2; temp2 = temp2.sibling; tmp.sibling = temp1.sibling; temp1.sibling = tmp; temp1 = tmp.sibling; } else { if (temp1.degree < temp2.degree) { if ((temp1.sibling == null) || (temp1.sibling.degree > temp2.degree)) { // HERE! … X4(1) >= X8(1) && X10(2) > X8(1) && X10(2) <= X11(2) && X11(2) > 0 && X10(2) > 0 && X8(1) <= X9(1) && X9(1) > 0 && X8(1) > 0 && X4(1) <= X2(1) && X6(2) > X4(1) && X6(2) <= X7(2) && X7(2) > 0 && X6(2) > 0 && X4(1) <= X5(1) && X5(1) > 0 && X4(1) > 0 && X2(1) <= X0(1) && X2(1) <= X3(1) && X3(1) > 0 && X2(1) > 0 && X0(1) <= X1(1) && X1(1) > 0 && X0(1) > 0 insert(X0);insert(X1);insert(X2);insert(X3);insert(X4); insert(X5);insert(X6);insert(X7);insert(X8);insert(X9); insert(X10);insert(X11);extractMin();

  10. Symbolic Execution is not the best thing since it has a few serious namely : • It is inherently white-box • Only as good as the decision procedures

  11. void test(int x, int y) { if (x > 0) { if (y == hash(x)) S0; else S1; if (x > 3 && y > 10) S3; else S4; } } native int hash(x); OR int hash(x) { return x*x % 1023 } Code is not available so no SymExe is possible Assuming we only have a linear integer arithmetic DP we cannot handle the non-linearity here

  12. Concolic Execution or Directed Automated Random Testing (DART)Godefroid, Klarlund and Sen 2005 Novel combination of concrete and symbolic execution to overcome the two weaknesses of classic symbolic execution Executes program concretely, but collects the path condition, negates constraints on the PCafter a run and executes again with the newlyfound solutions.

  13. void test(int x, int y) { if (x > 0) { if (y == hash(x)) S0; else S1; if (x > 3 && y > 10) S3; else S4; } } native int hash(x) { if (0<=x<=10) return x*10; else return 0; } [ X>0 & Y!=10 & X>3] test(1,0) test(4,0) [ X > 0 ] [ X > 0 ] [ X > 0 & Y != 40 ] [ X > 0 & Y != 10 ] [ X>0 & Y!=40 & X>3 & Y<=10] [ X>0 & Y!=10 & X<=3] Concolic Execution

  14. void test(int x, int y) { if (x > 0) { if (y == hash(x)) S0; else S1; if (x > 3 && y > 10) S3; else S4; } } native int hash(x) { if (0<=x<=10) return x*10; else return 0; } [ X>0 & Y!=40 & X>3 & Y>10] [ X>0 & Y=40 & X>3 & Y>10] test(4,11) test(4,40) [ X > 0 ] [ X > 0 ] [ X > 0 & Y = 40 ] [ X > 0 & Y != 40 ] [ X>0 & Y!=40 & X>3 & Y>10] [ X>0 & Y=40 & X>3 & Y>10] Concolic Execution

  15. void test(int x, int y) { if (x > 0) { if (y == 40) S0; else S1; if (x > 3 && y > 10) S3; else S4; } } native int hash(x) { if (0<=x<=10) return x*10; else return 0; } [ X>0 & Y=40 & X>3 & Y>10] test(4,40) [ X > 0 ] [ X > 0 & Y = 40 ] [ X>0 & Y=40 & X>3 & Y>10] Concolic Execution

  16. void test(int x, int y) { if (x > 0) { if (y == hash(x)) S0; else S1; if (x > 3 && y > 10) S3; else S4; } } native int hash(x) { if (0<=x<=10) return x*10; else return 0; } [ X>0 & Y=40 & X<=3 & Y>10] test(1,40) [ X > 0 ] Divergence! Aimed to get S0;S4 But reached S1;S4 [ X > 0 & Y != 10 ] [ X>0 & Y!=10 & X<=3 & Y>10] ASSERT not via S0 Concolic Execution

  17. Symbolic Execution with Mixed Concrete-Symbolic SolvingPasareanu, Rungta, Visser 2011 Symbolic Execution that falls back onto concrete values when it doesn’t have access to the code or thedecision procedures don’t work. SymCrete = Symbolic + Concrete vs Concolic = Concrete + Symbolic

  18. Symbolic Execution test(X,Y) void test(int x, int y) { if (x > 0) { if (y == hash(x)) S0; else S1; if (x > 3 && y > 10) S3; else S4; } } native int hash(x) { if (0<=x<=10) return x*10; else return 0; } [ X > 0 ] [ X > 0 ] hash (X) [ X>10 & … ] … [ 0<X<=10 & Y=X*10 ] S0 [ 0<X<=10 & Y!=X*10 ] S1 [ 3<X<=10 & 10<Y=X*10] S3 [ 3<X<=10 & 10<Y!=X*10] S3 [ 0<X<=10 & Y=X*10 & ! (X>3 & Y>10) ] S4 [ 0<X<=10 & Y!=X*10 & ! (X>3 & Y>10) ] S4 [ X > 0 ] hash (X) [ 0<X<=10 ] ret X*10 [ X>10] ret 0

  19. Symbolic Execution test(X,Y) void test(int x, int y) { if (x > 0) { if (y == hash(x)) S0; else S1; if (x > 3 && y > 10) S3; else S4; } } native int hash(x) { if (0<=x<=10) return x*10; else return 0; } [ X > 0 ] [ X > 0 ] hash (X) • SymCrete 3 Steps • Split PC into two parts: • Part you can solve • Part you cannot solve • Solve the easy part and evaluatethe hard part with the solutions • Replace the hard part with theevaluated results and check SAT

  20. SymCrete Execution test(X,Y) void test(int x, int y) { if (x > 0) { if (y == hash(x)) S0; else S1; if (x > 3 && y > 10) S3; else S4; } } native int hash(x) { if (0<=x<=10) return x*10; else return 0; } [ X > 0 ] [ X > 0 ] hash (X) [ X>0 & Y=hash(X) ] S0 [ X>0 & Y!=hash(X) ] S1 easy hard 1 X>0 Y=hash(X) 2 X=1 Y=hash(1)=10 X>0 & Y!=10 is SAT X>0 & Y=10 is SAT 3

  21. SymCrete Execution test(X,Y) void test(int x, int y) { if (x > 0) { if (y == hash(x)) S0; else S1; if (x > 3 && y > 10) S3; else S4; } } native int hash(x) { if (0<=x<=10) return x*10; else return 0; } [ X > 0 ] [ X > 0 ] hash (X) [ X>0 & Y=hash(X) ] S0 [ X>3 & Y=hash(X) & Y>10 ] S3 X>3 & Y>10 Y=hash(X) [ 3>=X>0 & Y=hash(X)] S4 X=4 & Y=11 Y=hash(4) 3>=X>0 Y=hash(X) [X>3 & Y=40 & Y>10 is SAT X=1 Y=hash(1) [3>=X>0 & Y=10 is SAT

  22. SymCrete Execution test(X,Y) void test(int x, int y) { if (x > 0) { if (y == hash(x)) S0; else S1; if (x > 3 && y > 10) S3; else S4; } } native int hash(x) { if (0<=x<=10) return x*10; else return 0; } [ X > 0 ] [ X > 0 ] hash (X) x=1,y=10 [ X>0 & Y=hash(X) ] S0 x=1,y=0 [ X>0 & Y!=hash(X) ] S1 [ X>3 & Y=hash(X) & Y>10 ] S3 [ X>3 & Y!=hash(X) & Y>10 ] S3 x=4,y=40 x=4,y=11 [ 3>=X>0 & Y=hash(X)] S4 [ 3>=X>0 & Y!=hash(X)] S4 x=1,y=10 x=1,y=0

  23. The Risk of Unsoundness test (int x, int y) { if (x>=0 && x>y && y == x*x) S0; else S1; } Not Reachable [ X>=0 & X > Y & Y = X*X ] S0 Must add constraints on the solutions from Step 2 in Step 3 X>=0 & X>Y Y = X*X X=0, Y=-1 Y=0*0=0 X>=0 & X>Y & Y=0 & X=0 NOT SAT X>=0 & X>Y & Y=0 Is SAT which impliesS0 is Reachable Concolic will diverge instead

  24. 3 More Enhancements Incremental Solving User Annotations Random Solving

  25. Problem for Concolic [ X>0 & Y!=10 & Y>10] void test(int x, int y) { if (x > 0) { if (y == hash(x)) S0; else S1; if (y > 10) S3; else S4; } } native int hash(x) { if (0<=x<=10) return x*10; else return 0; } test(1,11) test(1,0) [ X > 0 ] [ X > 0 ] [ X > 0 & Y != 10 ] [ X > 0 & Y != 10 ] [ X>0 & Y!=10 & Y>10] [ X>0 & Y!=10 & Y<=10] After Negation Concolic is Stuck [ X>0 & Y=10 & Y>10]

  26. SymCrete Execution test(X,Y) void test(int x, int y) { if (x > 0) { if (y == hash(x)) S0; else S1; if (y > 10) S3; else S4; } } native int hash(x) { if (0<=x<=10) return x*10; else return 0; } [ X > 0 ] [ X > 0 ] hash (X) [ X>0 & Y=hash(X) ] S0 [ X>0 & Y=hash(X) & Y>10 ] S3 X>0 & Y>10 Y=hash(X) X=1 Y=hash(1) =10 X>0 & Y>10 & Y=10 & X=1 UNSAT Get another solution! X=2 Y=hash(2) =20 X>0 & Y>10 & Y=20 & X=2 is SAT

  27. SymCrete Execution test(X,Y) @Partition({“x>3”,”x<=3”}) void test(int x, int y) { if (x > 0) { if (y == hash(x)) S0; else S1; if (y > 10) S3; else S4; } } native int hash(x) { if (0<=x<=10) return x*10; else return 0; } [ X > 0 ] [ X > 0 ] hash (X) [ X>0 & Y=hash(X) ] S0 [ X>0 & Y=hash(X) & Y>10 ] S3 X>0 & Y>10 Y=hash(X) X=1 Y=hash(1) =10 X>0 & Y>10 & Y=10 & X=1 UNSAT Add user partitions one at a time X>0 & Y>10 & X > 3 Y=hash(X) X=4 Y=hash(4) =40 X>3 & Y>10 & Y=40 & X=4 is SAT

  28. Random Solving Pick solutions randomly from the solution space Current implementation only picks randomlyif the solution space is completely unconstrained - Not all solvers support the general feature -

  29. Implementation Symcrete Custom Listeners on SPF Symbolic PathFinderSPF Symbolic Execution extension for JPF called jpf-symbc Model Checker for JavaOpen Source http://babelfish.arc.nasa.gov/trac/jpf JavaPathFinder

  30. public String preserveTags(String body) {…}

  31. Infinite loops are the worst kind of error, since it is input driven and therefore can reappear frequently, in fact infinitely often!

  32. Symbolic String Analysis • (Almost) All Java String operations covered • Mixed Integer and String constraints • Automata and SMT (bitvector) back-ends • Part of Symbolic PathFinder • M.Sc. by Gideon Redelinghuys • Collaborators • Jaco Geldenhuys (Stellenbosch)

  33. Infinite Loop? while (x > 0) (x,y) = (x+y+2,-x); Try (1,-2)

  34. We only consider affine transformations on loop variablesand simple loop conditions such as x>0 and x>=0

  35. Infinite Loop? x,y are inputs while (x >= 0) { x := x – y; }

  36. Ranking functions x,y are inputs while (x >= 0) { assert(‘x > x); x := x – y; }

  37. Use ranking functions for non-termination!

  38. Ranking functions x,y are inputs while (x >= 0) { assert(‘x > x); x := x – y; } ‘x <= x {c /\ wp(s,‘x <= x)} s {c /\ wp(s,‘x <= x)} ‘x <= x ‘x <= x …

  39. Inductive? {x >= 0 /\ wp(x:=x-y,‘x <= x)} x := x - y {x >= 0/\ wp(x:=x-y,‘x <= x)} x,y are inputs while (x >= 0) { assert(‘x > x); x := x – y; } wp(x:=x-y,’x<=x) = {x <= x-y} {x >= 0 /\ y <= 0} x := x - y {x >= 0 /\ y <= 0}

  40. So how about just… while (c) { s; } {c /\ wp(s,!rr)} s {c /\ wp(s,!rr)}

  41. x,y are inputs while (x >= 0) { assert(‘x > x); x := x + y; y := 1 – y; }

  42. {x >= 0 /\ wp(x:=x+y;y:=1-y,‘x <= x)} x := x – y; y := 1 – y; {x >= 0/\ wp(x:=x+y;y:=1-y,‘x <= x)} x,y are inputs while (x >= 0) { assert(‘x > x); x := x + y; y := 1 – y; } wp(x:=x+y;y:=1-y,’x<=x) = {x <= x+(1-y)} {x >= 0 /\ y <= 1} x:=x+y;y:=1-y; {x >= 0 /\ y <= 1}

  43. ‘x <= x ‘x <= x ‘x <= x N ‘x <= x … ‘x <= x … while (c) { s; } {c /\ wp(sn,!rr)} sn {c /\ wp(sn,!rr)}

  44. We conjecture that if there is an infinite loop then there exist n such that for all x for which the following is true you will loop infinitely while (x0 > 0) { f(x) = Ax+b; } x0 > 0 /\ f1(x) > 0 /\ … /\ f2n-1(x) > 0 /\ x0 ≤ fn(x) => fn(x) ≤ f2n(x) Can we derive n from the number of variables in x? For 1 variable n = 2 For 2 variables n >= 6

  45. Implementation AffineLoopListener Custom Listener on SPF Tries n = 0..6 Symbolic PathFinderSPF Symbolic Execution extension for JPF called jpf-symbc Model Checker for JavaOpen Source http://babelfish.arc.nasa.gov/trac/jpf JavaPathFinder

More Related