Foundations of Constraint Processing CSCE421/821, Spring 2011 cse.unl/~ cse421 - PowerPoint PPT Presentation

slide1 n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Foundations of Constraint Processing CSCE421/821, Spring 2011 cse.unl/~ cse421 PowerPoint Presentation
Download Presentation
Foundations of Constraint Processing CSCE421/821, Spring 2011 cse.unl/~ cse421

play fullscreen
1 / 82
Foundations of Constraint Processing CSCE421/821, Spring 2011 cse.unl/~ cse421
107 Views
Download Presentation
leann
Download Presentation

Foundations of Constraint Processing CSCE421/821, Spring 2011 cse.unl/~ cse421

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. Basic Consistency Methods Foundations of Constraint Processing CSCE421/821, Spring 2011 www.cse.unl.edu/~cse421 All questions: cse421@cse.unl.edu Berthe Y. Choueiry (Shu-we-ri) Avery Hall, Room 360 choueiry@cse.unl.edu Tel: +1(402)472-5444

  2. Lecture Sources Required reading • Algorithms for Constraint Satisfaction Problems, Mackworth and Freuder AIJ'85 • Sections 3.1, 3.2, 3.3. Chapter 3. Constraint Processing. Dechter Recommended • Sections 3.4—3.10. Chapter 3. Constraint Processing. Dechter • Networks of Constraints: Fundamental Properties and Application to Picture Processing, Montanari, Information Sciences 74 • Consistency in Networks of Relations, Mackworth AIJ'77 • Constraint Propagation with Interval Labels, Davis, AIJ'87 • Path Consistency on Triangulated Constraint Graphs, Bliek & Sam-Haroud IJCAI'99

  3. Outline • Motivation and background • Node consistency and its complexity • Arc consistency and its complexity • Criteria for performance comparison and CSP parameters. • Path consistency and its complexity • Global consistency properties • Minimality • Decomposability • When PC guarantees global consistency

  4. Consistency checking Motivation • CSP are solved with search (conditioning) • Search performance is affected by: - problem size - amount of backtracking • Backtracking may yield.. thrashing Thrashing Exploring non-promising sub-trees and rediscovering the same inconsistencies over and over again  Malady of backtrack search

  5. V2 V1 {1, 2, 3} {1, 2, 3} V3<V1 V3<V2 V3 { 0, 1, 2} V3>0 V3<V4 V3<V5 V4 V5 {1, 2} { 1, 2} V5<V4 (Some) causes of thrashing • Search order: • V1, V2, V3, V4, V5 • What happens in search if we: • do not check CV3 (node inconsistency) • do not check constraint CV3,V5 (arc inconsistency) • do not check constraints CV3,V4, CV3,V5, and CV4,V5 (path inconsistency)

  6. Consistency checking Goal: eliminate inconsistent combinations Algorithms(for binary constraints): • Node consistency • Arc consistency (AC-1, AC-2, AC-3, ..., AC-7, AC-3.1, etc.) • Path consistency (PC-1, PC-2, DPC, PPC, etc.) • Constraints of arbitrary arity: • Waltz algorithm ancestor of AC's • Generalized arc-consistency (Dechter, Section 3.5.1 ) • Relational m-consistency (Dechter, Section 8.1)

  7. Overview of Recommended Reading • Davis, AIJ'77 • Focuses on the Waltz algorithm • Studies its performance and quiescence for given: • Constraint types (bounded diff, algebraic, etc.) • Domains types (continuous or finite) • Mackworth, AIJ'77 • presents NC, AC-1, AC-2, PC-1, PC-2 • Mackworth and Freuder, AIJ'85 • studies their complexity  Mackworth and Freuder concentrate on finite domains  More people work on finite domains than on continuous ones  Continuous domains can be quite tough

  8. Constraint Propagation with Interval Labels Ernest Davis • `Old' paper (1987): terminology slightly different • Interval labels: continuous domains • Section 8: sign labels (discrete) • Concerned with Waltz algorithm (e.g., quiescence, completeness) • Constraint types vs. domain types (Table 3) • Addresses applications of reasoning about • Physical Systems (circuit, SPAM) • time relations (TMM) •  Advice: read Section 3, more if you wish

  9. Waltz algorithm for label inference • REFINE(C(Vi, Vk, Vm, Vn), Vi) finds a new label for Vi consistent with C. • REVISE(C (Vi, Vk, Vm, Vn)) refines the domains of Vi, Vk, Vm, Vn by iterating over these variables until quiescence (i.e., no domains is further refined) • Waltz Algorithm • revises all constraints by iterating over each constraint connected to a variable whose domain has been revised • Is the ancestor of arc-consistency

  10. WARNING → Completeness: (i.e., for solving the CSP)Running the Waltz Algorithm does not solve the problem. A=2  B=3 is still not a solution! →Quiescence: The Waltz algorithm may go into infinite loops even if problem is solvable x  [0, 100] x = y y  [0, 100] x = 2y →Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms of • constraint types • domain types

  11. Importance of this paper  Establishes that constraints of bounded differences (temporal reasoning) can be efficiently solved by the Waltz algorithm (O(n3), n number of variables)  Early paper that attracts attention on the difficulty of label propagation in interval labels (continuous domains). This work has been continued by Faltings (AIJ 92) who studied the early-quiescence problem of the Waltz algorithm.

  12. Basic consistency algorithms Examining finite, binary CSPs and their complexity Notation: Given variables i, j, k with values x, y, z, the predicate Pijk(x, y, z) is true iff the 3-tuple x, y, z  Cijk Node consistency: checking Pi(x) Arc consistency: checking Pij(x,y) Path consistency: bit-matrix manipulation

  13. time space Worst-case asymptotic complexity Worst-case complexity as a function of the input parameters Upper bound: f(n) is O(g(n)) means that f(n)c.g(n) f grows as g or slower Lower bound: f(n) is  (h(n)) means that f(n)  c.h(n) f grows as g or faster Input parameters for a CSP: n = number of variables a = (max) size of a domain dk = degree of Vk ( n-1) e = number of edges (or constraints)  [(n-1), n(n-1)/2]

  14. Outline • Motivation and background • Node consistency and its complexity • Arc consistency and its complexity • Criteria for performance comparison and CSP parameters. • Path consistency and its complexity • Global consistency properties • Minimality • Decomposability • When PC guarantees global consistency

  15. Node consistency (NC) Procedure NC(i): Di Di { x | Pi(x) } Begin for i  1 until n do NC(i) end

  16. Complexity of NC Procedure of NC(i) DiDi { x | Pi(x) } Begin for i  1 until n do NC(i) end • For each variable, we check a values • We have n variables, we do n.a checks • NC is O(a.n)

  17. Outline • Motivation and background • Node consistency and its complexity • Arc consistency and its complexity • Criteria for performance comparison and CSP parameters. • Path consistency and its complexity • Global consistency properties • Minimality • Decomposability • When PC guarantees global consistency

  18. Vj Vi Vi Vj 1 1 1 2 2 2 2 3 3 3 Arc-consistency Adapted from Dechter Definition: Given a constraint graph G, • A variable Vi is arc-consistent relative to Vj iff for every value aDVi, there exists a value bDVj | (a, b)CVi,Vj. • The constraint CVi,Vj is arc-consistent iff • Vi is arc-consistent relative to Vj and • Vj is arc-consistent relative to Vi. • A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc-consistent

  19. Procedure Revise Revises the domains of a variable i Procedure Revise(i,j): Begin DELETE  false for each x  Dido if there is no y  Dj such that Pij(x, y) then begin delete x from Di DELETE  true end return DELETE end • Revise is directional • What is the complexity of Revise? {a2}

  20. Revise: example R. Dechter Apply the Revise procedure to the following example

  21. Vj Vi Vi Vj 1 1 1 2 2 2 2 3 3 3 Effect of Revise Adapted from Dechter Question: Given • two variables Vi and Vj • their domains DVi and DVj, and • the constraint CVi,Vj, write the effect of the Revise procedure as a sequence of operations in relational algebra Hint: • Think about the domain DVi as a unary constraint CVi and • consider the composition of this unary constraint and the binary one.. • Solution: DVi  DViVi (CVi,VjDVi) • This is actually equivalent to DViVi (CVi,VjDVi)

  22. Arc Consistency (AC-1) Procedure AC-1: 1 begin 2 for i 1 untilndo NC(i) 3 Q  {(i, j) | (i,j)  arcs(G), i  j } 4 repeat 5 begin 6 CHANGE  false 7 for each (i, j)  Q do CHANGE  ( REVISE(i, j) or CHANGE ) 8 end 9 until ¬ CHANGE 10 end • AC-1 does not update Q, the queue of arcs • No algorithm can have time complexity below O(ea2) • Alert: Algorithms do not, but should, test for empty domains

  23. Warning • What I don’t like about the algorithms as specified in most papers is that they do not check for domain wipe-out. • In your code, always check for domain wipe out and terminate/interrupt the algorithm when that occurs • In practice may save lots of #CC and cycles

  24. V1 V1 V1 V1 V3 V2 V3 V3 V2 V3 V2 V2 V1 V1 V1 V3 V2 V3 V3 V2 V2 Arc consistency • AC may discover the solution Example borrowed from Dechter

  25. Arc consistency 2. AC may discover inconsistency Example borrowed from Dechter

  26. x [0, 10] x [0, 7] x [4, 7] x  y-3 x  y-3 x  y-3 y [0, 10] y [3, 10] y [7, 10] NC & AC Example courtesy of M. Fromherz Unary constraint x>3 is imposed AC propagates bounds again AC propagates bound

  27. Complexity of AC-1 Procedure AC-1: 1 begin 2 for i  1 untilndo NC(i) 3 Q  {(i, j) | (i,j)  arcs(G), i  j 4 repeat 5 begin 6 CHANGE  false 7 for each (i, j)  Q do CHANGE  (REVISE(i, j) or CHANGE) 8 end 9 until ¬ CHANGE 10 end Note: Q is not modified and |Q| = 2e • 4  9 repeats at most n·a times • Each iteration has |Q| = 2e calls to REVISE • Revise requires at most a2 checks of Pij  AC-1 is O(a3· n · e)

  28. AC versions • AC-1 does not update Q, the queue of arcs • AC-2 iterates over arcs connected to at least one node whose domain has been modified. Nodes are ordered. • AC-3 same as AC-2, nodes are not ordered

  29. Arc consistency (AC-3) AC-3 iterates over arcs connected to at least one node whose domain has been modified Procedure AC-3: 1 begin 2 fori 1 untilndo NC(i) 3 Q { (i, j) | (i, j)  arcs(G), i j } 4 WhileQ is not empty do 5 begin 6 select and delete any arc (k, m) from Q 7 If Revise(k, m) thenQQ { (i, k) | (i, k)  arcs(G), i  k, i  m } 8 end 9 end

  30. Complexity of AC-3 • Procedure AC-3: • 1 begin • 2 fori 1 untilndo NC(i) • 3 Q {(i, j) | (i, j) arcs(G), i j } • 4 WhileQ is not empty do • 5 begin • select and delete any arc (k, m) from Q • 7 If Revise(k, m) thenQQ { (i, k) | (i, k)  arcs(G), i  k, i  m } • 8 end • 9 end • First |Q| = 2e, then it grows and shrinks 48 • Worst case: • 1 element is deleted from DVk per iteration • none of the arcs added is in Q • Line 7: a·(dk - 1) • Lines 4 - 8: • Revise: a2 checks of Pij •  AC-3 is O(a2(2e + a(2e-n))) • Connected graph (en – 1), AC-3 is O(a3e) • If AC-p, AC-3 is O(a2e)  AC-3 is (a2e) • Complete graph: O(a3n2)

  31. V2 V2 V1 V1 { 1, 2, 3, 4 } { 1, 2, 3, 4, 5 } { 1, 3, 5 } { 1, 2, 3, 4, 5 } { 1, 3, 5 } { 1, 2, 3, 5 } { 1, 2, 3, 4, 5 } { 1, 2, 3, 4, 5 } V4 V3 V4 V3 Example: Apply AC-3 Example: Apply AC-3 Thanks to Xu Lin DV1 = {1, 2, 3, 4, 5} DV2 = {1, 2, 3, 4, 5} DV3 = {1, 2, 3, 4, 5} DV4 = {1, 2, 3, 4, 5} CV2,V3 = {(2, 2), (4, 5), (2, 5), (3, 5), (2, 3), (5, 1), (1, 2), (5, 3), (2, 1), (1, 1)} CV1,V3 = {(5, 5), (2, 4), (3, 5), (3, 3), (5, 3), (4, 4), (5, 4), (3, 4), (1, 1), (3, 1)} CV2,V4 = {(1, 2), (3, 2), (3, 1), (4, 5), (2, 3), (4, 1), (1, 1), (4, 3), (2, 2), (1, 5)}

  32. Applying AC-3 Thanks to Xu Lin • Queue = {CV2, V4, CV4,V2, CV1,V3, CV2,V3, CV3, V1, CV3,V2} • Revise(V2,V4): DV2DV2 \ {5} = {1, 2, 3, 4} • Queue = {CV4,V2, CV1,V3, CV2,V3, CV3, V1, CV3, V2} • Revise(V4, V2): DV4 DV4 \ {4} = {1, 2, 3, 5} • Queue = {CV1,V3, CV2,V3, CV3, V1, CV3, V2} • Revise(V1, V3): DV1 {1, 2, 3, 4, 5} • Queue = {CV2,V3, CV3, V1, CV3, V2} • Revise(V2, V3): DV2 {1, 2, 3, 4}

  33. Applying AC-3 (cont.) Thanks to Xu Lin • Queue = {CV3, V1, CV3, V2} • Revise(V3, V1): DV3 DV3 \ {2} = {1, 3, 4, 5} • Queue = {CV2, V3, CV3, V2} • Revise(V2, V3): DV2 DV2 • Queue = {CV3, V2} • Revise(V3, V2): DV3 {1, 3, 5} • Queue = {CV1, V3} • Revise(V1, V3): DV1 {1, 3, 5} END

  34. Main Improvements Mohr & Henderson (AIJ 86): • AC-3 O(a3e) AC-4 O(a2e) • AC-4 is optimal • Trade repetition of consistency-check operations with heavy bookkeeping on which and how many values support a given value for a given variable • Data structures it uses: • m: values that are active, not filtered • s: lists all vvp's that support a given vvp • counter: given a vvp, provides the number of support provided by a given variable • How it proceeds: • Generates data structures • Prepares data structures • Iterates over constraints while updating support in data structures

  35. counter ( V , V ), 3 0 4 2 ( V , V ), 2 0 4 2 ( V , V ), 1 0 4 2 ( V , V ), 1 0 2 3 M M ( V , V ), 4 0 1 3 Step 1: Generate 3 data structures • mand s have as many rows as there are vvp’s in the problem • counterhasas many rows as there are tuples in the constraints

  36. Step 2: Prepare data structures Data structures: s and counter. • Checks for every constraint CVi,Vj all tuples Vi=ai, Vj=bj_) • When the tuple is allowed, then update: • s(Vj,bj) s(Vj,bj)  {(Vi, ai)} and • counter(Vj,bj)  (Vj,bj) + 1 Update counter ((V2, V3), 2) to value 1 Update counter ((V3, V2), 2) to value 1 Update s-htable (V2, 2) to value ((V3, 2)) Update s-htable (V3, 2) to value ((V2, 2)) Update counter ((V2, V3), 4) to value 1 Update counter ((V3, V2), 5) to value 1 Update s-htable (V2, 4) to value ((V3, 5)) Update s-htable (V3, 5) to value ((V2, 4))

  37. Constraints CV2,V3 and CV3,V2 Updating m Note that (V3, V2),4  0 thus we remove 4 from the domain of V3 and update (V3, 4)  nil in m Updating counter Since 4 is removed from DV3 then for every (Vk, l) | (V3, 4) s[(Vk, l)], we decrement counter[(Vk, V3), l] by 1

  38. Summary of arc-consistency algorithms • AC-4 is optimal (worst-case) [Mohr & Henderson, AIJ 86] • Warning: worst-case complexity is pessimistic. Better worst-case complexity: AC-4 Better average behavior: AC-3 [Wallace, IJCAI 93] • AC-5: special constraints [Van Hentenryck, Deville, Teng 92] functional, anti-functional, and monotonic constraints • AC-6, AC-7: general but rely heavily on data structures for bookkeeping [Bessière & Régin] • Now, back to AC-3: AC-2000, AC-2001≡AC-3.1, AC3.3, etc. • Non-binary constraints: • GAC (general) [Mohr & Masini 1988] • all-different (dedicated) [Régin, 94]

  39. Outline • Motivation and background • Node consistency and its complexity • Arc consistency and its complexity • Criteria for performance comparison & CSP parameters • Path consistency and its complexity • Global consistency properties • Minimality • Decomposability • When PC guarantees global consistency

  40. CSP parameters ‹n, a, t, d› • n is number of variables • a is maximum domain size • t is constraint tightness: • d is constraint density where e is the #constraints, emin=(n-1), and emax = n(n-1)/2 Lately, we use constraint ratiop = e/emax → Constraints in random problems often generated uniform → Use only connected graphs (throw the unconnected ones away)

  41. Criteria for performance comparison • Bounding time and space complexity (theoretical) • worst-case: always • average-case: never • best- case: sometimes • Counting #CC and #NV (theoretical, empirical) • Measuring CPU time (empirical)

  42. Performance comparisonCourtesy of Lin XU AC-3, AC-7, AC-4 on n=10,a=10,t,d=0.6 displaying #CC and CPU time AC-3 AC-4 AC-7

  43. Wisdom (?) Free adaptation from Bessière • When a single constraint check is very expensive to make, use AC-7. • When there is a lot of propagation, use AC-4 • When little propagation and constraint checks are cheap, use AC-3x

  44. AC2001 • When checking (Vi,a) against Vj • Keep track of the Vj value that supports (Vi,a) • Last((Vi,a),Vi) • Next time when checking again (Vi,a) against Vj, we start from Last((Vi,a),Vi) and don’t have to traverse again the domain of Vj again or list of tuples in CViVj • Big savings..

  45. Performance comparison Courtesy of Shant AC3, AC3.1, AC4 on on n=40,a=16,t,d=0.15 displaying #CC and CPU time

  46. AC-what? Instructor’s personal opinion • Used to recommend using AC-3.. • Now, recommend using AC2001 • Do the project on AC-* if you are curious.. [Régin 03]

  47. = = = AC is not enough Example borrowed from Dechter Arc-consistent? Satisfiable?  seek higher levels of consistency V V 1 1 b a a b V V V V 2 3 2 a b 3 b a a b a b

  48. Outline • Motivation and background • Node consistency and its complexity • Arc consistency and its complexity • Criteria for performance comparison and CSP parameters. • Path consistency and its complexity • Global consistency properties • Minimality • Decomposability • When PC guarantees global consistency

  49. V2 V1 Vm-1 Vm V0 for all y  DVm for all x  DV0 Consistency of a path A path (V0, V1, V2, …, Vm) of length m is consistent iff • for any value xDV0 and for any value yDVm that are consistent (i.e., PV0 Vm(x, y)) •  a sequence of values z1, z2, … , zm-1 in the domains of variables V1, V2, …, Vm-1, such that all constraints between them (along the path, not across it) are satisfied (i.e., PV0 V1(x, z1)  PV1 V2(z1, z2)  …  PVm-1 Vm(zm-1, zm) )

  50. V2 V1 Vm-1 Vm V0 for all y  DVm for all x  DV0 Note  The same variable can appear more than once in the path  Every time, it may have a different value  Constraints considered: PV0,Vm and those along the path  All other constraints are neglected