Inferring Synchronization under Limited Observability

Inferring Synchronization under Limited Observability (work in progress) Martin Vechev, EranYahav, Greta Yorsh IBM T.J. Watson Research Center

Concurrency is Hard. • Assist the programmer in systematically exploring alternative choices of synchronization • correctness • synchronization cost

Inferring Synchronization • Input • program P • specification S • Output a program P’ • P’ satisfies S • P’ obtained from P by adding synchronization • Challenge: eliminate invalid interleavings while preserving as many valid ones as possible

Dimensions for Comparing Solutions • Permissiveness • P1 is more permissive than P2 when  P2    P1  • Synchronization Cost • P1 has lower cost than P2 when the running time of synchronization code in P1 is smaller than that of P2

Observability • Connection between permissiveness and synchronization cost • User input: upper bound on synchronization cost • Limits the observations about program state that can be made by the synchronization code cost incomparable solutions max perm permissiveness max perm for bounded cost

Synchronization under Limited Observability • Input • program P • specification S • cost function and cost bound C • Output a program P’ • P’ satisfies S • P’ obtained from P by adding synchronization restricted to C Is it always possible to find P’ s.t.  P    P’  ? NO!

Maximally Permissive Program • P’ is maximally permissive with respect to C • P’ satisfies S • P’ obtained from P by adding synchronization restricted to C • for every P’’ obtained from P by adding synchronization restricted to C if  P’    P’’  then P’’ does not satisfy S

Our Goal • Input • program P • specification S • observability C • Output a concurrent program P’ • P’ satisfies S • P’ obtained from P by adding synchronization restricted to C • P’ is maximally permissive with respect to C • synchronization code in P’ must not block indefinitely

Synchronization Mechanisms • Semaphores • Monitors • Conditional critical region (CCR) • Fine grained (e.g., CAS) • Locks • ....

Conditional Critical Regions • Syntax of CCR • Declarative • Synchronization code • can observe the program state • does not modify program state • How to infer guards for CCRs ? guard  stmt

Limited Observability • Bounded cost of synchronization means restricted language of guards LG = { guard | cost(guard) < bound } • Limits observations about program state thatcan be made by the guards

Example Languages of Guards • EQ(V) • boolean combination of equalities between variable from V and integer constant • (x != 1 || z != 0) • EvenOdd(V) • boolean combinations of predicates even and odd applied to program expressions over V • e(x) || o(y)

Example • Program op1 { 1: x = z + 1 } op2 { 2: y = x + 1 } op3 { 3: z = y + 1 } main { int x = 0, y = 0, z = 0; op1 || op2 || op3 } • Specification !(y = 2 && z = 1) • Full observability EQ({ x,y,z })

Example 1,2,30,0,0 z=y+1 y=x+1 x=z+1 e,2,31,0,0 1,e,30,1,0 1,2,e0,0,1 y=x+1 z=y+1 z=y+1 y=x+1 x=z+1 x=z+1 e,e,31,2,0 e,2,e1,0,1 e,e,31,1,0 1,e,e0,1,2 e,2,e2,0,1 1,e,e0,1,1 y=x+1 y=x+1 z=y+1 x=z+1 x=z+1 z=y+1 e,e,e1,2,3 e,e,e1,2,1 e,e,e1,1,2 e,e,e3,1,2 e,e,e,2,3,1 e,e,e2,1,1

Example • Program op1 { 1: x = z + 1 } op2 { 2: y = x + 1 } op3 { 3: (x!=1 || y!=0 || z!=0)z = y + 1 } main { int x = 0, y = 0, z = 0; op1 || op2 || op3 } • Specification !(y = 2 && z = 1) • Full observability EQ( { x,y,z } )

Example • Program op1 { 1: x = z + 1 } op2 { 2: y = x + 1 } op3 { 3: z = y + 1 } main { int x = 0, y = 0, z = 0; op1 || op2 || op3 } • Specification !(y = 2 && z = 1) • Limited observability EQ( { x, z} )

Example • Program op1 { 1: (x != 0 || z != 0)  x = z + 1 } op2 { 2: y = x + 1 } op3 { 3: (x != 1 || z != 0) z = y + 1 } main { int x = 0, y = 0, z = 0; op1 || op2 || op3 } • Specification !(y = 2 && z = 1) • Limited observability EQ( { x, z} )

Example 1,2,30,0,0 x!=1 || z!=0  x!=0 || z!=0 z=y+1 y=x+1 x=z+1 e,2,31,0,0 1,e,30,1,0 1,2,e0,0,1 x!=1 || z!=0  x!=1 || z!=0  y=x+1 x!=0|| z!=0 z=y+1 z=y+1 x!=0 || z!=0 y=x+1 x=z+1 x=z+1 e,e,31,2,0 e,2,e1,0,1 e,e,31,1,0 1,e,e0,1,2 e,2,e2,0,1 1,e,e0,1,1 x!=0 || z!=0 x!=1 || z!=0  x!=1 || z!=0  x!=0 || z!=0 y=x+1 y=x+1 z=y+1 x=z+1 x=z+1 z=y+1 e,e,e1,2,3 e,e,e1,2,1 e,e,e1,1,2 e,e,e3,1,2 e,e,e,2,3,1 e,e,e2,1,1

Our Approach • Construct transition system of P and S • Remove a (minimal) set of transitions such that the result satisfies S • Implement resulting transition system as program by strengthening guards of CCRs in P

Removing Transitions • Which transitions to remove? • bad-transitions transitions on a path to doomed state • cut-transitions transitions from non-doomed to doomed state • In what order to remove transitions?

Algorithm GREEDY(P : Program) : Program { R = ∅ while (true) { ts = < States , Transitions \ R, Init > if valid(ts) return implement(P,R) B = cut-transitions(ts) if B = ∅ abort “cannot find valid synchronization” select a transition t ∈ B R = R ∪ equiv(t) } }

Example • Program op1 { 1: x = z + 1 } op2 { 2: y = x + 1 } op3 { 3: z = y + 1 } main { int x = 0, y = 0, z = 0; op1 || op2 || op3 } • Specification !(y = 2 && z = 1) • Limited observability EQ( { x, z} )

Example 1,2,30,0,0 z=y+1 y=x+1 x=z+1 e,2,31,0,0 1,e,30,1,0 1,2,e0,0,1 y=x+1 z=y+1 z=y+1 y=x+1 x=z+1 x=z+1 e,e,31,2,0 e,2,e1,0,1 e,e,31,1,0 1,e,e0,1,2 e,2,e2,0,1 1,e,e0,1,1 y=x+1 y=x+1 z=y+1 x=z+1 x=z+1 z=y+1 e,e,e1,2,3 e,e,e1,2,1 e,e,e1,1,2 e,e,e3,1,2 e,e,e,2,3,1 e,e,e2,1,1

Side Effects • Transitions associated with the same CCR are controlled by the same guard • Strengthening guard associated with transition t1 has side-effect • if no guard can distinguish between source(t1) and sourc(t2) due to limited observability • transition system without t1 but with t2 is not implementable • Side effect may create new doomed states!

Step 0 1,2,30,0,0 z=y+1 y=x+1 x=z+1 e,2,31,0,0 1,e,30,1,0 1,2,e0,0,1 y=x+1 z=y+1 z=y+1 y=x+1 x=z+1 x=z+1 e,e,31,2,0 e,2,e1,0,1 e,e,31,1,0 1,e,e0,1,2 e,2,e2,0,1 1,e,e0,1,1 y=x+1 y=x+1 z=y+1 x=z+1 x=z+1 z=y+1 e,e,e1,2,3 e,e,e1,2,1 e,e,e1,1,2 e,e,e3,1,2 e,e,e,2,3,1 e,e,e2,1,1

Step 1 1,2,30,0,0 x!=1 || z!=0  z=y+1 y=x+1 x=z+1 e,2,31,0,0 1,e,30,1,0 1,2,e0,0,1 x!=1 || z!=0  x!=1 || z!=0  y=x+1 z=y+1 z=y+1 y=x+1 x=z+1 x=z+1 e,e,31,2,0 e,2,e1,0,1 e,e,31,1,0 1,e,e0,1,2 e,2,e2,0,1 1,e,e0,1,1 x!=1 || z!=0  x!=1 || z!=0  y=x+1 y=x+1 z=y+1 x=z+1 x=z+1 z=y+1 e,e,e1,2,3 e,e,e1,2,1 e,e,e1,1,2 e,e,e3,1,2 e,e,e,2,3,1 e,e,e2,1,1

Step 2 1,2,30,0,0 x!=1 || z!=0  x!=0 || z!=0 z=y+1 y=x+1 x=z+1 e,2,31,0,0 1,e,30,1,0 1,2,e0,0,1 x!=1 || z!=0  x!=1 || z!=0  y=x+1 x!=0|| z!=0 z=y+1 z=y+1 x!=0 || z!=0 y=x+1 x=z+1 x=z+1 e,e,31,2,0 e,2,e1,0,1 e,e,31,1,0 1,e,e0,1,2 e,2,e2,0,1 1,e,e0,1,1 x!=0 || z!=0 x!=1 || z!=0  x!=1 || z!=0  x!=0 || z!=0 y=x+1 y=x+1 z=y+1 x=z+1 x=z+1 z=y+1 e,e,e1,2,3 e,e,e1,2,1 e,e,e1,1,2 e,e,e3,1,2 e,e,e,2,3,1 e,e,e2,1,1

Algorithm GREEDY(P : Program) : Program { R = ∅ while (true) { ts = < States , Transitions \ R, Init > if valid(ts) return implement(P,R) B = cut-transitions(ts) if B = ∅ abort “cannot find valid synchronization” select a transition t ∈ B R = R ∪ equiv(t) } }

Algorithms • Greedy algorithm • removes cut-transitions • result satisfies spec (or abort) • if there are no side-effects then the result is maximally permissive • can be extended to be maximally permissive • polynomial • Exhaustive algorithm • removes bad-transitions • result satisfies spec (or abort) • (all) maximally permissive • exponential

Initial Evaluation • Prototype • greedy algorithm • transition system constructed using SPIN • Examples • Dining philosophers • Asynchronous counters • Race correction

Infinite Transition System • Finite state abstraction • Same algorithm

Example • Program op1 { 1: x = x + 1 2: y = y + 1 3: goto 1 } op2 { 4: x = x - 1 5: y = y - 1 6: goto 4 } main { x = 0, y =0; op1 || op2 } • Specification pc2 == 6  even(x+y) • Limited observability EvenOdd(x,y)

Example • Program op2 { 4: x = x - 1 5: odd(x+y) y = y - 1 6: goto 4 } op1 { 1: odd(x+y)  x = x + 1 2: y = y + 1 3: goto 1 } Result satisfies the spec but might block • Specification pc2 == 6  even(x+y) • Limited observability EvenOdd(x,y)

Inferring Guards under Abstraction • Conservatively eliminate potentially stuck states • cannot guarantee maximally permissive • Refine when state becomes potentially stuck • terminates if there is a finite bisimulation quotient • Specialized abstractions for stuckness • related to abstractions for termination

Summary • Greedy and exhaustive algorithms for inferring guards of CCRs • maximally permissive programs • limited observability • side effects • implementability • observational equivalence • characterizing observable states • minimize synchronization cost

Related Work • Recovery and predication mechanisms • STM, Isolator, Tolerace • Synthesis from temporal specification • controller synthesis • Program repair as a game • memoryless maximal winning strategy

Ongoing and Future Work • Greedy algorithm based on domination • Conditions for maximal permissiveness • Minimize synchronization cost for given observability • Complexity of guard inference (NP-hard,coNP-hard,2) • Abstraction for stuck states • Temporal safety and liveness properties • Infer other synchronization mechanisms • meta-data, atomic sections, non-blocking

Inferring Synchronization under Limited Observability