1 / 66

Strauss: A Specification Miner

Strauss: A Specification Miner. Glenn Ammons Department of Computer Sciences University of Wisconsin-Madison. How should programs behave?. Strauss mines temporal specifications for example, always call free after malloc Why? To debug To verify To test To modify To understand.

ladonna
Download Presentation

Strauss: A Specification Miner

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Strauss:A Specification Miner Glenn Ammons Department of Computer Sciences University of Wisconsin-Madison

  2. How should programs behave? • Strauss mines temporal specifications • for example, always call free after malloc • Why? • To debug • To verify • To test • To modify • To understand

  3. How should programs behave? • Strauss mines temporal specifications • for example, always call free after malloc • Why? • To debug • To verify • To test • To modify • To understand • Specs constrain programs • like structured programming, types, etc.

  4. For all calls Y := accept(sock = X): start X := socket() read(sock = Y) write(sock = Y) Y := accept(sock = X) close(sock = X) close(sock = Y) end Strauss output • Spec. says what programs should do: • What order of calls? • What values in calls?

  5. Traces ... socket(domain = 2, type = 1, proto = 0, return = 7)) ... ... socket(domain = 2, type = 1, proto = 0, return = 7)) ... ... 7 := socket(domain = 2, type = 1, proto = 0) ... Strauss inputs Interface (e.g., sockets) Library Client programs Strauss

  6. State transition model (STM) def socket() close(def use sock) def accept(use sock) read(use sock) write(use sock) Traces ... socket(domain = 2, type = 1, proto = 0, return = 7)) ... ... socket(domain = 2, type = 1, proto = 0, return = 7)) ... ... 7 := socket(domain = 2, type = 1, proto = 0) ... Strauss inputs Interface (e.g., sockets) Library Client programs Strauss

  7. State transition model (STM) def socket() close(def use sock) def accept(use sock) read(use sock) write(use sock) Traces ... socket(domain = 2, type = 1, proto = 0, return = 7)) ... ... socket(domain = 2, type = 1, proto = 0, return = 7)) ... ... 7 := socket(domain = 2, type = 1, proto = 0) ... Seed pattern accept Strauss inputs Interface (e.g., sockets) Library Client programs Strauss

  8. My contributions • A dynamic approach to mining • analyze communication, not source code • A tool for mining specifications • practical: scalable and partially automatic • temporal specs may refer to multiple data values • requires little info about code • Heuristics to tell good code from bad code • Birds of a feather flock together • Found specifications and bugs • two goals: summarize and predict

  9. Summary of results • Mined 8 specs. for X11 • example: XInternAtom should only be called during initialization, not in the event loop. • Found 61 bugs • In widely distributed programs • wish (Tk), xpdf, xpilot, ... • Included serious races and performance bugs • By verifying dynamically (on traces)

  10. Related work: obtaining specs. • By hand • From programmers • Types • Specification languages and annotations • From analysts, testers • Abstraction tools (Bandera [Dwyer et al.]) • With specification mining

  11. Comparing spec. miners • Strauss • Specs: temporal, multiple objects, arbitrary NFAs • Mining: from traces, local, no code necessary • Goal: specs. for interfaces • Other work • Concurrent work • FSMs for Java objects [Whaley, Martin, and Lam] • Simple templates [Metacompilation] • Loop invariants [ESC/Java] • Earlier work • Arithmetic properties [Daikon] • Cliches [Programmer’s Apprentice] • Communicating processes [Verisoft]

  12. Trace programs (as PDGs) socket(...) socket(...) socket(...) accept(...) accept(...) accept(...) write(...) write(...) write(...) read(...) read(...) read(...) start . read(so = Y) close(...) close(...) close(...) X := socket() . Y := accept(so = X) write(so = Y) close(so = X) close(so = Y) end Specification Overview of Strauss Traces and STM Seed pattern extract scenarios apply STM Scenarios Classify and learn

  13. State transition model (STM) def socket() close(def use sock) def accept(use sock) read(use sock) write(use sock) Trace + STM = Trace program 1 7 := socket(domain = 2, type = 1, proto = 0) 2 0x100 := malloc(size = 23) 3 8 := accept(sock = 7, addr = 0x40, addr_len = 0x50) 4 23 := write(sock = 8, buf = 0x100, len = 23) 5 0 := close(sock = 8) ... Trace program Vars: v7, v8 1 v7 := socket() 2 skip 3 v8 := accept(sock = v7) 4 write(sock = v8) 5 v8 := close(sock = v8) ...

  14. State transition model (STM) def socket() close(def use sock) def accept(use sock) read(use sock) write(use sock) Trace + STM = Trace program 1 7 := socket(domain = 2, type = 1, proto = 0) 2 0x100 := malloc(size = 23) 3 8 := accept(sock = 7, addr = 0x40, addr_len = 0x50) 4 23 := write(sock = 8, buf = 0x100, len = 23) 5 0 := close(sock = 8) ... Trace program Vars: v7, v8 1 v7 := socket() 2 skip 3 v8 := accept(sock = v7) 4 write(sock = v8) 5 v8 := close(sock = v8) ...

  15. State transition model (STM) def socket() close(def use sock) def accept(use sock) read(use sock) write(use sock) Trace + STM = Trace program 1 7 := socket(domain = 2, type = 1, proto = 0) 2 0x100 := malloc(size = 23) 3 8 := accept(sock = 7, addr = 0x40, addr_len = 0x50) 4 23 := write(sock = 8, buf = 0x100, len = 23) 5 0 := close(sock = 8) ... Trace program Vars: v7, v8 1 v7 := socket() 2 skip 3 v8 := accept(sock = v7) 4 write(sock = v8) 5 v8 := close(sock = v8) ...

  16. State transition model (STM) def malloc() free(def p) read(use buf) write(use buf) A different STM 1 7 := socket(domain = 2, type = 1, proto = 0) 2 0x100 := malloc(size = 23) 3 8 := accept(sock = 7, addr = 0x40, addr_len = 0x50) 4 23 := write(sock = 8, buf = 0x100, len = 23) 5 0 := close(sock = 8) ... Trace program Var: v0x100 1 skip 2 v0x100 := malloc() 3 skip 4 write(buf = v0x100) 5 skip ...

  17. State transition model (STM) def malloc() free(def p) read(use buf) write(use buf) A different STM 1 7 := socket(domain = 2, type = 1, proto = 0) 2 0x100 := malloc(size = 23) 3 8 := accept(sock = 7, addr = 0x40, addr_len = 0x50) 4 23 := write(sock = 8, buf = 0x100, len = 23) 5 0 := close(sock = 8) ... Trace program Var: v0x100 1 skip 2 v0x100 := malloc() 3 skip 4 write(buf = v0x100) 5 skip ...

  18. Trace programs (with PDGs) socket(...) socket(...) socket(...) accept(...) accept(...) accept(...) write(...) write(...) write(...) read(...) read(...) read(...) start . read(so = Y) close(...) close(...) close(...) X := socket() . Y := accept(so = X) write(so = Y) close(so = X) close(so = Y) end Specification Overview of Strauss Traces and STM Seed pattern extract scenarios apply STM Scenarios Classify and learn

  19. Extracting scenarios from a TPDG Pick a seed Slice forwards Slice backwards Chop each pair

  20. Extracting scenarios from a TPDG Pick a seed Slice forwards Slice backwards Chop each pair

  21. Extracting scenarios from a TPDG Pick a seed Slice forwards Slice backwards Chop each pair

  22. Extracting scenarios from a TPDG Pick a seed Slice forwards Slice backwards Chop each pair

  23. Extracting scenarios from a TPDG Pick a seed Slice forwards Slice backwards Chop each pair

  24. Extracting scenarios: example Vars: v7, v8 1 v7 := socket 2 v8 := accept(sock = v7) 3 write(sock = v8) 4 v8 := close(sock = v8) 5 v7 := close(sock = v7)

  25. Extracting scenarios: example Vars: v7, v8 1 v7 := socket 2 v8 := accept(sock = v7) 3 write(sock = v8) 4 v8 := close(sock = v8) 5 v7 := close(sock = v7) Pick a seed

  26. Extracting scenarios: example Vars: v7, v8 1 v7 := socket 2 v8 := accept(sock = v7) 3 write(sock = v8) 4 v8 := close(sock = v8) 5 v7 := close(sock = v7) Pick a seed Slice forwards

  27. Extracting scenarios: example Vars: v7, v8 1 v7 := socket 2 v8 := accept(sock = v7) 3 write(sock = v8) 4 v8 := close(sock = v8) 5 v7 := close(sock = v7) Pick a seed Slice forwards Slice backwards

  28. Extracting scenarios: example Vars: v7, v8 1 v7 := socket 2 v8 := accept(sock = v7) 3 write(sock = v8) 4 v8 := close(sock = v8) 5 v7 := close(sock = v7) Pick a seed Slice forwards Slice backwards Chop each pair

  29. Trace programs (with PDGs) socket(...) socket(...) socket(...) accept(...) accept(...) accept(...) write(...) write(...) write(...) read(...) read(...) read(...) start . read(so = Y) close(...) close(...) close(...) X := socket() . Y := accept(so = X) write(so = Y) close(so = X) close(so = Y) end Specification Overview of Strauss Traces and STM Seed pattern extract scenarios apply STM Scenarios Classify and learn

  30. Strings A C E G B A C E G B A C E G B Overview of learning Scenarios (dep. graphs) standardize Learn NFA NFA

  31. Standardizing scenarios v7 := socket() v8 := accept(sock = v7) write(sock = v8) read(sock = v8) v8 := close(sock = v8) v7 := close(sock = v7) v2 := socket() v3 := accept(sock = v2) v2 := close(sock = v2) read(sock = v3) write(sock = v3) v3 := close(sock = v3)

  32. Standardizing scenarios v7 := socket() v8 := accept(sock = v7) v7 := close(sock = v7) read(sock = v8) write(sock = v8) v8 := close(sock = v8) v2 := socket() v3 := accept(sock = v2) v2 := close(sock = v2) read(sock = v3) write(sock = v3) v3 := close(sock = v3)

  33. Standardizing scenarios X := socket() Y := accept(sock = X) X := close(sock = X) read(sock = Y) write(sock = Y) Y := close(sock = Y) X := socket() Y := accept(sock = X) X:= close(sock = X) read(sock = Y) write(sock = Y) Y := close(sock = Y)

  34. Standard scenario = string X := socket() Y := accept(sock = X) X := close(sock = X) read(sock = Y) write(sock = Y) Y := close(sock = Y) X := socket() Y := accept(sock = X) X:= close(sock = X) read(sock = Y) write(sock = Y) Y := close(sock = Y) A B C D E F

  35. Strings A C E G B A C E G B A C E G B Overview of learning Scenarios (dep. graphs) standardize Learn NFA NFA

  36. Raman’s PFSA algorithm • Build a weighted retrieval tree • Merge similar states

  37. Raman’s PFSA algorithm • Build a weighted retrieval tree • Merge similar states socket 100 accept 95 5 write read 5 70 25 write read write 5 70 25 write write read 25 70 25 5 end write

  38. Raman’s PFSA algorithm • Build a weighted retrieval tree • Merge similar states socket 100 accept 95 5 write read 5 70 25 write read write 5 70 25 write write read 25 70 25 5 end write

  39. Raman’s PFSA algorithm • Build a weighted retrieval tree • Merge similar states socket 100 accept 95 5 write read 5 70 25 write read write 5 70 25 25 write read 100 end

  40. Raman’s PFSA algorithm • Build a weighted retrieval tree • Merge similar states socket 100 accept 95 5 5 read write 70 25 read 75 25 25 write read 100 end

  41. Raman’s PFSA algorithm • Build a weighted retrieval tree • Merge similar states socket 100 accept 95 25 5 5 read write 70 75 25 25 write read 100 end

  42. Strings A C E G B A C E G B A C E G B socket(...) socket(...) socket(...) accept(...) accept(...) accept(...) write(...) write(...) write(...) read(...) read(...) read(...) close(...) close(...) close(...) Overview of learning Scenarios (dep. graphs) standardize Learn NFA NFA

  43. Strings Classified strings A C E G B A C E G B A C E G B socket(...) socket(...) socket(...) A C E G B A C E G B accept(...) accept(...) accept(...) A C E G B write(...) write(...) write(...) read(...) read(...) read(...) close(...) close(...) close(...) Overview of learning Scenarios (dep. graphs) standardize Classify Learn NFA NFA Learn NFA Specification

  44. b1 g0 G0 G0 b2 b0 g1 G0 G0 G0 Classifying scenarios

  45. b1 b1 g0 g0 G0 G0 G0 G0 b2 b2 b0 b0 g1 g1 G0 G0 G0 G0 G0 G0 Classifying scenarios

  46. b1 b1 g0 G0 G0 b2 b2 b0 b0 g1 G0 G0 G0 g0 g0 G0 G0 G0 G0 g1 g1 G0 G0 G0 G0 G0 G0 Classifying scenarios

  47. Concept analysis: mammals

  48. Concept analysis: mammals cats gibbons dogs dolphins humans whales {}

  49. Concept analysis: mammals cats gibbons dogs hairy

  50. Concept analysis: mammals cats dogs 4-legged hairy

More Related