1 / 16

Generating Tests from Counterexamples

Generating Tests from Counterexamples. Jinseong Jeon ARCS, KAIST. Actual Anxiety. What makes him panic?. What makes C.E. happen makes him panic!. Counterexample!. Greater Goals. How can it happen?.  a test vector. Any other cases?.  a test suite. Counterexample!. Test Driver

harrietb
Download Presentation

Generating Tests from Counterexamples

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Generating Testsfrom Counterexamples Jinseong Jeon ARCS, KAIST

  2. Actual Anxiety What makes him panic? What makes C.E. happen makes him panic! Counterexample! CS750b, KAIST

  3. Greater Goals • How can it happen? a test vector • Any other cases? a test suite Counterexample! CS750b, KAIST

  4. Test Driver Generator Program Test Driver Testing Target Pred. Test Suite Generator Test Suite The Greatest Goal Automated Debugger! CS750b, KAIST

  5. Test Driver Generator Program Test Driver Testing Target Pred. Test Suite Generator Test Suite Contents • How to generate a test vector? • How to generate a test suite? • How to generate a test driver? CS750b, KAIST

  6. from Trace to Test (1/2) [ Program ] [ Trace ] [ Trace formula ] Example() { if (y == x) y++; if (z <= x) y++; a = y – z; if (a < x) LOC: } assume (y = x) y = y + 1 assume !(z <= x) a = y – z assume (a < x) <y,0> = <x,0> <y,1> = <y,0> + 1 : <z,0> · <x,0> <a,2> = <y,1> - <z,0> <a,2> < <x,0> p ,( pc  LOC ) CS750b, KAIST

  7. integer linear programming (ILP) solver from Trace to Test (2/2) [ Trace formula ] [ Assignment ] [ Test vector ] <y,0> = <x,0> <y,1> = <y,0> + 1 : <z,0> · <x,0> <a,2> = <y,1> - <z,0> <a,2> < <x,0> <x,0>  0 <y,0>  0 <y,1>  1 <z,0>  2 <a,2>  -1 <x,0>  0 <y,0>  0 <z,0>  2 CS750b, KAIST

  8. feasible region Linear Programming • object function • maximize c1x1 + c2x2 • problem constraints • a11x1 + a12x2· b1 • a12x1 + a22x2· b2 • Algorithms • Simplex, Branch and Bound, etc. CS750b, KAIST

  9. An ILP Application • Buffer Overrun Detection using Liner Programming and Static Analysis int main() { char header[2048], buf[1024], *cc1, *cc2, *ptr; int i; FILE *fp; ... ptr = fgets(header, 2048, fp); cc1 = copy_buffer(header); for (i = 0; i < 10; i++) { ptr = fgets(buf, 1024, fp); cc2 = copy_buffer(buf); } } header!alloc!max · 2048 header!alloc!min ¸ 0 ... header!used!max · 2048 header!used!min ¸ 1 cc1!used!max ¸ header!used!max cc1!used!min · header!used!min ... i’!max ¸ i!max + 1 I’!min · i!min + 1 ... CS750b, KAIST

  10. Test Suite Gen. worklistà all locations in decreasing order of d.f.numbering Reach. Treeà a single node, the root test suiteÃ; while (worklist;) { qà pop(worklist) p-traceà MC(Reach. Tree, p, q) ifReach. Tree is complete then worklistÃ; else test suiteÃtest suite[ { test_vector(p-trace) } } returntest suite CS750b, KAIST

  11. Heuristics • A test vector can cover several locations.  we can remove those locations from the worklist. • MC’s unfolding (visiting) order  uncovered first, covered last • Time-out option CS750b, KAIST

  12. Lib. call User input Test Driver Gen. Test suite Test Driver Original code <1,1,1> Test-feeding func. <1,0,1> <0,1,2> <0,1,1> <0,0,1> <0,0,0> CS750b, KAIST

  13. A Security Example (1/2) int saved_uid, saved_euid; work_and_drop_priv() { L5: FILE *fp = fopen(FILENAME,”w”); L6: if (!fp) { L7: return; } L8: // work L9: seteuid(saved_uid); } int get_root_privileges() { L1: if (saved_euid == 0) { L2: return -1; } L3: seteuid(0); L4: return 0; } int main(int argc, char *argv[]) { L10: saved_uid = getuid(); L11: saved_euid = geteuid(); L12: seteuid(saved_uid); L13: // work under normal mode L14: if (get_root_privileges() == 0 ) { L15: work_and_drop_priv(); } L16: execv(argv[1], argv+1); } CS750b, KAIST

  14. A Security Example (2/2) L10: saved_uid = getuid(); L11: saved_euid = geteuid(); L12: seteuid(saved_uid); L14: tmp = get_root_privileges(); L1: if (saved_euid != 0) /* fails */ L3: seteuid(saved_euid); L4: return 0; L14: if (tmp == 0) /* succeeds */ L15: work_and_drop_priv(); L5: fp = fopen(FILENAME, “w”); L6: if (!fp) /* succeeds */ L7: return; L16: /* uid = 0 */ [ A trace generated by BLAST ] CS750b, KAIST

  15. Experiments • kbfiltr, floppy, cdaudio, parport, parclass •  Microsoft Windows device drivers • ping an implementation of the ping utility • ftpd a Linux port of the ftp daemon CS750b, KAIST

  16. Conclusions • pros • generate a test suite using model-checker • generate an automated debugger • dead code detection, safety verification • cons • only integer variables • what is really affected by BLAST? CS750b, KAIST

More Related