Software Design Workshop: Object Model Analysis with Alloy

1. 6894 · workshop in software designlecture 11 · november 18, 1998 · object model analysis

2. what an Alloy model contains • elements • declarations • components & their types • multiplicity constraints • basic temporal constraints (sticky, fixed) • invariants • more elaborate on individual components • constraints that apply across components • definitions • constraints that define new components • strictly unnecessary, but make model cleaner • operations • describe changes in configuration • assertions • redundant, like runtime assertions in code • remember to poll class re: weds

3. example • model Family { • domain {Person} • state { • Married, Parent : Person • sticky partition Male, Female : Person • spouse : Married ! -> Married ! • children : sticky Parent # -> Person # • (siblings) : Person -> Person • } • inv { • all p | p.spouse.spouse = p • no p | p.spouse in p.siblings • } • def siblings { • all p | p.siblings = p.~children.children - p • } • op Marry (p1 : Person, p2 : Person) { • p1.spouse := p2 && p2.spouse := p1 } Parent # spouse children ! Person Married # ! Female Male

4. example, ctd • assert { • one {p | no p.~children} -> some a | all p | p != a -> a in p.~children* • } • }

5. what should a checker do? • goals • make modelling more compelling • ie, more like programming • generate samples of states and transitions • find errors early • in particular • invariants • check consistency: ie, at least one configuration exists • definitions • check consistency • check determinism: can’t have two values of defined component • operations • check consistency: ie, at least one transition exists • check preservation: all invariants preserved • assertions • check validity

6. all boils down to solving • every analysis reduces to • finding an assignment of values to components • that makes a formula true • examples • invariant INV consistent? • if INV has an assignment • definition DEF of component d deterministic? • if there’s no assignment for • (some d | DEF && d = d1) && (some d | DEF && d = d2) && d1 != d2 • invariant INV preserved by operation OP? • if there’s no assignment for • INV && OP && not INV’ • assertion A true? • if there’s no assignment to • not A

7. a sample problem • the solving problem • given an assignment, find a solution • example • formula • a, b : Person ! • spouse : Person ? -> Person ? • siblings : Person -> Person • b in (a.spouse.siblings & a.siblings.spouse) • a solution • Person = {Daniel, Tim, Claudia, Emily} • spouse = {Daniel -> Claudia, Claudia -> Daniel, Tim -> Emily, Emily -> Tim} • siblings = {Daniel -> Tim, Tim -> Daniel, Claudia -> Emily, Emily -> Claudia} • a = Daniel • b = Emily

8. snags • dilemma • Alloy is undecidable • no algorithm exists that can determine • whether an arbitrary formula has a solution • so a complete, automatic tool is impossible • in fact • even simpler languages are undecidable • Tarski’s relational calculus • formulas ::= (e1 = e2) • exprs ::= r | Id | e1 ; e2 | e1 + e2 | -e • but let’s be pragmatic • what counts is • finding solutions quickly, when they exist, most of the time • can search systematically for solutions if they’re small • fundamental question • how big a solution do you need to consider?

9. 4 smallestrevealingscope catch miss cumulative invalid claims 90% small scope hypothesis • an empirical hypothesis • a high proportion of the invalid claims that occurin practice in models can be refutedby counterexamples in small scopes

10. some revealing scopes • bug in mobile IP • 2 hosts, 1 mobile host, 1 message • flaw in Word • 3 styles, 2 formatting rules • flaw in proof of FAA handoff algorithm • 2 controllers, 3 planes • DoD high level simulation architecture • 3 clients • futurebus cache protocol (Clarke et al) • 1 bus segment, 1 cache line, 3 processors

11. how Fox works ALLOY • compilation • translate Alloy into kernel • expand shorthands • infer types • normalizing • put in disjunctive normal form (DNF) • (formula11 && formula12 && formula13 && …) || • (formula21 && formula22 && formula23 && …) || • … • skolemize • (some x | F) translated to F • existential variables become components • boolify • pick a scope: number of elements in each domain • (user gets choice) • for each clause of the DNF, generate a boolean formula TRANSLATE KERNEL TRANSLATE SCOPE BOOL SOLVE STATE,TRANS

12. example • true or false? • if my sibling’s spouse is my spouse’s sibling, • then my sibling’s spouse’s sibling is her spouse’s sibling’s spouse • in Alloy • me.sibling.spouse = me.spouse.sibling -> • me.sibling.spouse.sibling = me.sibling.spouse.spouse.sibling.spouse • to check • suppose there are 3 persons • let sibling be the matrix sib_ij, where sib_ij is true if the jth person is a sibling of the ith • let spouse be the matrix spo_ij, where spo_ij is true if the jth person is a sibling of the ith • let me be represented by a boolean vector me_i, where me_i is true if i am the ith person • then claim is true (for 3 persons) unless there is a solution to • me_1 or me_2 or me_3 • not (me_1 and me_2) … • (me_1 and sib_12 and spo_23 and me_1 and spo_12 and sib_23) or …

13. boolifying, no variables • assume • no quantifiers • only components appear in expressions, no free variables • components • s : D make a vector of fresh boolean vars that is scope(D) long • r : D1 -> D2 make a matrix of fresh boolean vars that is scope(D1) x scope(D2) • expressions • e1 + e2 make a vector whose ith element is the OR of the ith elts of e1 and e2 • e1 & e2 make a vector whose ith element is the AND of the ith elts of e1 and e2 • e.r make a vector whose jth element is the OR over all i’s of (e_i and r_ij) • formulas • f && g just conjoin • f || g disjoin

14. boolifying with variables • assume one variable, for now • variables • v : D represent as a function from 1.. scope (D) to fresh boolean vars • expressions • e1 = e2 suppose e1, e2 are represented by functions E1, E2 • then e1 = e2 is represented by E such that E(i) = (E1(i) = E2(i)) • formulas • f && g suppose f, g are represented by functions F, G • then f && g is represented by H such that H(i) = F(i) && G(i) • all v | f suppose f is represented by function F • then (all v | f) is represented by (F(1) and F(2) and …) • more than one variable • exprs and formulas become functions of several variables • for quantifier, have to collapse on appropriate variable

15. solving the boolean formula • OTS solvers • Davis Putnam • developed in 1960, many versions available • still one of the best (deterministic) methods • WalkSAT • non-deterministic algorithm, uses hill-climbing • present formula in conjunctive normal form (CNF) • in each step, tries to increase # clauses that are true • example: Mobile IP analysis • scope #bool vars #bool clauses • 3 438 2436 4 760 6548 5 1170 16536 • all took < 1s to analyze

16. non-deterministic solvers • WalkSAT [Kautz & Selman, 1994] • a solver for hard SAT problems • hill climbing, on number of satisfied clauses • incomplete (so may miss a bug, but seems rare) • amazingly fast • basic algorithm • for i = 1 to MAX_TRIES T = randomly generated assignment for j = 1 to MAX_FLIPS if T satisfies formula return T v = variable whose flipping gives largest increase in #clauses satisfied T = T with v flippedreturn “no assignment found” • refinement: at each step, • with prob p, follow standard scheme • with prob (1 - p), pick a variable from an unsatisfied clause and flip it

17. research challenges • language development • operations: inferring frame conditions • extensions: specialized types, sequences • checking technology • interestingness toggle: for r: X -> X, obtain r like {x0 -> {x0, x1, x2}, x1 -> {x0}} • scaling: devise new ways to combat formula explosion • solving: tailor solver to problem? • miscellaneous • connections to code • generate RTAs, monitoring code, from invariants • check invariants statically • self-updating software • what constraints should downloaded object satisfy? • protocol aspects • how are Alloy models and ELHs related? • build and analyze a single model?

18. sample output • counterexample generated by Fox • PERSON: PERSON = {P0,P1,P2} • me: PERSON! = P2 • sibling: PERSON -> PERSON = {P1 -> {P2}, P2 -> {P1}} • spouse: PERSON -> PERSON = {P1 -> {P2}, P2 -> {P0,P1}} • so add • all p | p.spouse.spouse = p • Fox then generates • PERSON: PERSON = {P0,P1,P2} • me: PERSON! = P2 • sibling: PERSON -> PERSON = {P1 -> {P1}, P2 -> {P0,P1}} • spouse: PERSON -> PERSON = {P0 -> {P1}, P1 -> {P0}, P2 -> {P2}} • so add • no p | p in p.spouse • and no counterexample generated…