Loading in 5 sec....

CSPs: Adding Structure to SATPowerPoint Presentation

CSPs: Adding Structure to SAT

- 361 Views
- Uploaded on
- Presentation posted in: Sports / GamesEducation / CareerFashion / BeautyGraphics / DesignNews / Politics

CSPs: Adding Structure to SAT

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

CSPs: Adding Structure to SAT

George Katsirelos Fahiem Bacchus

University of Toronto

- Finite domain Constraint Satisfaction Problems (CSPs).
- Formally equivalent to SAT
- Important practical differences.
- Different algorithmic techniques have been developed in the two areas.
- Understanding these can help cross fertilize both fields.

BackgroundThe SAT and CSP Formalisms

- SAT = hV, Ci
- V = {V1, V2, …, Vn} is a set of Boolean variables
- C = {c1, c2, …, ck} a set of clauses.

- CSP = hV, D, Ci
- V = {V1, V2, …, Vn} is a set of multi-valued variables
- D = {D1, D2, …, Dn} is a set of value domains, with Di being the domain of values for variable Vi
- C = {C1, C2, …, Ck} is a set of constraints.

- In both CSP and SAT the aim is to find an assignment of values for all of the variables:
- In SAT these values must satisfy the clauses
- In CSPs these values must satisfy the constraints.

- A constraint C(X1,X2, …, Xk) over the variables X1, …, Xk is a Boolean function
- It maps assignments to these variables to {0,1} C(X1,X2, …, Xk) : DX1££ DXk {0,1}

- If a tuple of assignments maps to 1, then these assignments satisfy the constraint, otherwise these assignments the falsify the constraint.

- We can specify the constraint with a table
- C(X,Y,Z) with DX = DY = DZ = {1, 2, 3}

- Thus we can represent the constraint as a set of satisfying assignment tuples

- Or as a set of falsifying assignment tuples

- Extensionalrepresentations specify the constraint as an explicit list of satisfying assignments (or falsifying assignments).
- Extensional representations were used in the 2005 CSP solver competition. But are almost never used in practice.
- The extensional representation becomes very large, growing exponentially with the number of variables the constraint is over.

- Constraint could also be represented intensionally as an algorithm for computing the Boolean function.

=

X · Y · Z

- Intensional representations are typical in practice.
- To specify a CSP problem in a CSP solver one supplies subroutines to implementing the constraints of the problem.
- Commercial CSP solvers supply a large library of predefined common constraints.
- You then simply specify the variables of the CSP, their domains, and the constraints that are over them.

- Further insight into the relation between SAT and CSPs is provided by looking at how we can translate between the formalisms.

- Translating in this direction is trivial
- Each SAT variable becomes a CSP variable, with {0,1} as its domain of values.
- Each clause is equivalent to a Boolean function from the variables it is over
- (x, y, -z) A function mapping (x=0,y=0,z=1) to 0, all other assignments of x,y,z to 1.

- The other direction requires two steps
- Converting the multi-valued variables into a set of Boolean assignment variables.
- Converting the constraints into clauses over the assignment variables.

- Let X be a CSP variable with Dx = {d1, …, dm}
- We create m Boolean assignment variables x1, x2, …, xm these have the the interpretation
xi is true iff X=di.

- The CSP variable X must have a value and it must have a unique value.
- Hence the Boolean assignment variables x1, x2, …, xm associated with a particular CSP variable are mutually exclusive and exhaustive.
- This is captured by adding the clauses
- (x1, x2, …, xm) X must have a value
- (-xi, -xj) for all (i j) X has a unique value

- Now we convert the constraints to clauses.
- Each falsifying assignment tuple in the constraint’s extensional representation is equivalent to a clause.
- So a constraint becomes a set of clauses, one for each falsifying assignment.

- Each falsifying tuple is a set of assignment variables that cannot be simultaneously true.
- E.g.. –(x1^ y2^ z1)
- Pushing the negation in we get a clause(-x1_ -y2_ -z1)

- There are various optimizations that can be applied to this basic translation.
- Specific constraints admit more compact encodings.

Modeling with CSPs

- CSPs offer
- Multi valued variables: more natural for modeling real problems.
- Constraints over groups of variables that permit a more natural encoding of the constraints of the problem.

- Industrial applications are much easier to formalize using CSPs, and the range of application of CSP technology in industry far exceeds that of SAT.

- Place N queens on an NxN chess board so that no queen can attack any other queen.

- Place N queens on an NxN chess board so that queen can attack any other queen.

- N, Queen variables, one for each column

- Place N queens on an NxN chess board so that queen can attack any other queen.

- N values for each variable:
- The row we place that column’s queen on.

- Constraints
- AllDiff(Q1, …, QN) each Queen has a unique value (can’t be in the same row)
- Cij(Qi,Qj): |Qi – Qj| |i-j| (for each i j)
- can’t be on same diagonal

- A SAT encoding of N-Queens more complex to specify.
- SAT encodings almost impossible to generate by hand.

- Modeling using the richer language of CSPs, translate to SAT (automatically), solve using standard SAT solver.
- Understanding the pros and cons of this approach gives us further insight into the algorithmic differences between CSP and SAT solvers.

Solving CSPs

- SAT and CSP backtracking solvers differ in the three main parts of backtracking
- Propagation as we descend the search tree
- Learning as we ascend from failed subtrees
- Heuristics for guiding the branching decisions

- The clause learning in SAT solvers can be exploited.
- The mutually exclusive and exhaustive clauses for the multi-valued variables are not fully exploited.
- Branching heuristics insensitive to CSP structure.
- Unit propagation weaker than propagation methods employed in CSP solvers.

- These clauses impose a useful structure on the assignment variables.

- (x1, x2, …, xm) X must have a value
- (-xi, -xj) for all (i j) X has a unique value

- In general, the disjunction of any subset of positive literals is equivalent to the conjunction of the complimentary set of negative literals. E.g., if m=4
- x1_ x2´ –x3^ –x4
- x3´ -x1^ -x2^ -x4

- This structure could be exploited in various ways. For example,
- Two negative assignment literals clause is redundant
- (y1, y2, -x1, -x2, -z3) subsumed by (-x1,-x2)

- Negative assignment literal remove all positive literals from same variable.
- (y1, y2, -y3, x1, -x2, -z3)
- Resolve with (-y1, -y3) and (-y2, -y3) to obtain subsuming clause (-y3, -x2, -z3).

- Sets of clauses can be replaced by a single clause.
- Dx = Dy = {1, 2, 3, 4}(R, -x1, -y1) (R, -x1, -y2) (R, -x2, -y1) (R, -x2, -y2)
(R, -x1, -y1) (R, -x1, -y2) (R, -x2, -y1) (R, -x2, -y2)Equivalent to single clause(R, x3, x4, y3 , y4).

- Dx = Dy = {1, 2, 3, 4}(R, -x1, -y1) (R, -x1, -y2) (R, -x2, -y1) (R, -x2, -y2)

- (R, x3, x4, y3 , y4) ´ (R, (-x2^ -x1), (-y2^ -y1))
- Multiply this out and you get 8 clauses.

- Under unit propagation contradictions arise when –x is inferred in a context where x is already true
- This causes some clause to be falsified (conflict clause).

- With multi-valued variables we always have
- xi´ -x1^ –xi-1^ –xi+1^ -xm

- Hence conflicts arise only from refuting all values from some CSP variable’s domain
- -x1^^ -xm

- In CSP solvers the number of unrefuted values of a variable is always considered in the branching heuristic.
- In a SAT solver we shouldn’t choose to branch on xi without considering the status of other associated assignment variables.

Ansótegui1 et al 2003.

- Unit Prop in a SAT solver on the clauses generated by a constraint is equivalent to Forward Checking in CSPs.
- Forward Checking. Wait until all but one variable of the constraint is instantiated, and then prune incompatible values from the domain of the sole remaining uninstantiated variable.

- (-x1_ -y2_ -z1)
- (-x1_ –y3_ –z1)
- …

- Each clause contains one negated assignment literal from each CSP variable in the constraint.
- To make the clause unit one has to make all but of these assignment variables true:
- Equivalent to assigning the corresponding CSP variable
- x1´ X=1, y2´ Y=2

- Then unit propagation will falsify all assignments to the remaining unassigned CSP variable that would violate the constraint
- (-x1_ –y3_ –z1), (-x1_ –y3_ –z2)X=1 ^ Y=3 Z 1 & Z 2

- However, in practice, FC does not perform particularly well.
- A superior form of propagation is GAC.

- Given a constraint C(X1,X2, …, Xk)
- di2 DXi is supported(in C) if there exists a set of assignments{X1 = d1, …, Xi = di ,…, Xk = dk} that satisfies C:C(X1=d1, …, Xk = dk) = 1.
- This set is called a support for di.

- Supports for X = 1 (x1)
- Supports for x3If 3 is removed from the domain of Z, i.e., -z3 becomes true, x3 will loose its only support.

- The constraint C(X1,X2, …, Xk) is said to be GAC if for all of its variables Xi every value in their domain is supported (in C).
- We can make C(X1,X2, …, Xk) GAC by removing all unsupported values from the domains of its variables.

- GAC propagation is the dynamic process of making all of the constraints GAC.
- If d 2 DX is pruned from the domain of X while making C1 GAC.
- Then the other constraints over X must have GAC reestablished.
- This might prune values of other variables, and their constraints in turn must be made GAC once again.

- During search we make all constraints GAC at the root.
- The assignment X=1 means X 2, X 3, …
- Thus constraints over X have to have GAC reestablished by GAC propagation.
- Reestablishing GAC at every node is called Maintaining GAC.

- By using extra variables in the SAT encoding we can establish GAC with Unit Propagation.
- But Unit Propagation on the standard encoding is less powerful than GAC.

Bessière et al. 2003

- The power of GAC can be characterized using the notion of prime implicates.
- If T is a set of clauses, then the clause c is a prime implicate if
- c is non-tautological
- T ² c
- T ⊭ c’ for any c’ that is a subset of c

- If T is a set of clauses, then the clause c is a prime implicate if

- Let TC be the set of clauses of the constraint C, along with the mutually exclusive and exhaustive clauses for each of the C’s variables.
- Now we replace TC by the prime implicates of TC, PIc
Theorem: Unit prop over PIc achieves precisely GAC propagation on the constraint C.

- Note that GAC is localto the constraint. Communication between constraints occurs only through unit implicants (pruned values).
- So GAC is complete local inference for units.

- Achieving GAC over a generic constraint C(X1,X2, …, Xk) requires time exponential in K.
- However, there is however a huge body of knowledge in the CSP literature on how to achieve GAC on particular constraints in time polynomial in K.
- These methods (called propagators) exploit the special structure of the constraint.

- The fundamental problems
- is the size of the encoding
- The vast body of knowledge about propagators for GAC cannot be exploited.

- Exploiting propagators GAC has much in common with exploiting specialized theories in SMT.

CSP Solvers

- Constraints can be represented intensionally and propagators can be exploited.
- The multi-valued variable structure, and information about the constraints can be exploited for branching decisions.
- Learning in CSP solvers is much weaker than clause learning in Sat solvers, and it doesn’t integrate well with GAC propagation.

- We can improving learning in CSPs and achieve a better integration with GAC.
- We can also integrate GAC propagators with learning using ideas that are essentially identical to those used in SMT
- These ideas were developed independently.

- In CSPs learning from failed subtrees has a long tradition. Learning is typically called “nogood” recording.

- A NoGood in CSPs is a set of assignments that cannot be extended to a solution.
- (X=3, Y=2, Z=1)
- Translating this to SAT we get -(x3^ y2^ z1) ´ (-x3, -y2, -z1)

- Nogoods are negative clauses
- (clauses containing only negative literals).

- Restricting learning to NoGoods (negative clauses) restricts the solver’s “resolution power” to Negative Resolution.
- Negative resolution: every resolution step involves a clause negative clause.

- Negative resolution not as powerful as general resolution
- CSP solvers sometimes suffer a super-polynomial slowdown over SAT solvers running on the SAT encoding.

Mitchell 2003Katsirelos PhD thesis

- As a result of this restriction to learning negative clauses learning is hardly ever used CSP solvers in practice.
- Learning negative clauses is also produces particularly ineffective clauses from GAC.

- GAC prunes domain values. It forces negated assignment literals.
- Like SMT all we need to do is to label those literals with clauses.

X = 1 CHOICEX 2 (X 1, X 2) (variable can only have one value)Y 1 (Y 1, X=2) (Non-negative clause reason from GAC)Z 1 (Z 1, X=2) A = 2 CHOICEA 1 …A 3 …X 1 (X 1, A 2) (conflict clause fromconstraint over X, and A)

- We can resolve backwards from a conflict along the implication trail from a conflict to learn various types of new clauses
- I.e., we can apply standard SAT clause learning techniques.

- With Unit Prop each literal is implied by a specific clause that became unit, so the clause for labeling an implied literal is obvious.
- With GAC a value is pruned (an assignment variable is made false) as the result of many different clauses of the constraint.
- In particular, a value is pruned by GAC when it looses all of its supports.
- Each support (which is a tuple of assignments to the variables of the Constraint) is lost when one of its assignments is made false.

Supports for X = 1

- E.g., (X=1 ^ Y=1 ^ Z=1). This support can be lost if 1 is pruned from the domain of Y or from Z. (Y1, or Z1)

- If GAC on this constraint prunes X=1, a reason for this pruning is a set of currently true non-assignments that hits all of X=1’s supports.

Supports for X = 1

E.g.,Y1, Z2, Z3 covers all of X=1’s supports in this constraint:

Y1Æ Z2 Æ Z3!X1´(Y=1, Z=2, Z=3, X1)

- We put this implication on the trailX1 (X1, Y=1, Z=2, Z=3)
- Note that
- this is a non-negative clause
- We can compute this clause on the fly from an extensional representation of the constraint.
- There is no need to precompute and store all such possible “pruning” clauses.

- How do we obtain clausal reasons from GAC propagators?
- We can no longer find a hitting set for the supports of the value, there is no explicit representation of these supports.

- AllDiff(X1, …, Xn) is satisfied only by tuples of assignments to the Xi that are all different, i.e.,
- ij ! XiXj

- A way of enforcing GAC on AllDiff in poly-time was the probably the first propagator developed in the CSP literature. Regin [1994].
- The method utilizes maximum matchings in bipartite graphs.
- Since then dozens of propagators have been developed.

- DPLL must take exponential time on the pigeon hole problem PHP: this problem is hard for general resolution.
- PHP can be encoded as a single AllDiff(P1, …, Pn) each with domain of values {1, …, n-1}.
- This constraint has no satisfying tuples so every value will be pruned by GAC.
- A CSP solver can solve this problem in polynomial time. GAC propagation at the root, no search.

- How do we obtain a clausal reason for a value pruned by AllDiff?
- For Alldiff a value is pruned from a variable domain only when that value is consumedby some other variables. That is, the value must be used by some other variable.

DX = {1, 2, 3,4}, DY = {1, 2, 3,4}, DZ = {1, 2, 3}, DW = {1, 4, 5}

DX = {1, 2, 3,4}, DY = {1, 2, 3,4}, DZ = {1, 2, 3}, DW = {1, 4, 5}

Prune 4 from domain of X and Y

Initially all values are supported

DX = {1, 2, 3,4}, DY = {1, 2, 3,4}, DZ = {1, 2, 3}, DW = {1, 4, 5}

DX = {1, 2, 3,4}, DY = {1, 2, 3,4}, DZ = {1, 2, 3}, DW = {1, 4, 5}

1, 2, 3 are consumed by X, Y, Z. (Hall interval)

W cannot be assigned 1 since that value is consumed.

DX = {1, 2, 3,4}, DY = {1, 2, 3,4}, DZ = {1, 2, 3}, DW = {1, 4, 5}

- X4 Æ Y4 ! W1
- Clause reason for W1
(W1,X=4,Y=4)

W cannot be assigned 1 since that value is consumed.

- If allDiff Xd the reason is
- First find the set of variables that still have d in their domain.
- These variables must be consuming d.
- And the reason is the set of pruned values in their domains.

- Katsirelos has found ways to compute clausal reasons for a variety of known propagators.
- He has implemented this in a CSP solver called EFC, which contains
- a number of built in intensional constraints
- GAC propagators for them.
- Clausal reasons supplied by these propagators.

- This solver can be thought of as a multi-valued variable SMT solver, where T includes a set of constraints known in the CSP literature.
- Like SMT solvers it can display very impressive performance.

- More work still needs to be done
- There are many more propagators that we don’t yet know how to get clausal reasons from.
- Heuristics remain poorly understood, and still need improvement.

- From Katsirelos & Bacchus, “Generalized NoGoods in CSPs”

- CSP solvers use propagation Queues
- Sequences the propagators, as some propagators are more expensive.

- Flexible ways of specifying the level of propagation for each constraint.
- Only forward checking is preformed on some, GAC on others, we delay GAC until all but 3 variables of the constraint are instantiated etc.

- Bounds propagation.
- Order the domain values and instead of pruning all unsupported values, we maintain upper and lower bounds on the possible domain values.
- Possible to do more efficiently than GAC in many cases.
- This would correspond to generating only a subset of the implied literals.
- Once we get to a solution all literals are set anyway.

- Multi-valued variables are very useful. But not fully exploited in SAT solvers.
- Huge body of known constraints and algorithms for propagating them.

- CSPs add structure to SAT.
- This structure can be exploited to make modeling easier.
- Can be used to identify groups of clauses over which
- higher levels of propagation can be profitable
- Specialized non-resolution based algorithms can be exploited.
- Connecting these extra kinds of reasoning with clause learning adds considerable extra power.