Foundations of Constraint Processing CSCE421/821, Fall 2004: - PowerPoint PPT Presentation

slide1 n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Foundations of Constraint Processing CSCE421/821, Fall 2004: PowerPoint Presentation
Download Presentation
Foundations of Constraint Processing CSCE421/821, Fall 2004:

play fullscreen
1 / 78
Foundations of Constraint Processing CSCE421/821, Fall 2004:
85 Views
Download Presentation
dory
Download Presentation

Foundations of Constraint Processing CSCE421/821, Fall 2004:

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. Consistency Algorithms Foundations of Constraint Processing CSCE421/821, Fall 2004: www.cse.unl.edu/~choueiry/F04-421-821/ Berthe Y. Choueiry (Shu-we-ri) Avery Hall, Room 123B choueiry@cse.unl.edu Tel: +1(402)472-5444 Consistency: Properties & Algorithms

  2. Lecture Sources Required reading • Algorithms for Constraint Satisfaction Problems, Mackworth and Freuder AIJ'85 • Sections 3.1, 3.2, 3.3. Chapter 3. Constraint Processing. Dechter Recommended • Sections 3.4—3.10. Chapter 3. Constraint Processing. Dechter • Networks of Constraints: Fundamental Properties and Application to Picture Processing, Montanari, Information Sciences 74 • Consistency in Networks of Relations, Mackworth AIJ'77 • Constraint Propagation with Interval Labels, Davis, AIJ'87 • Path Consistency on Triangulated Constraint Graphs, Bliek & Sam-Haroud IJCAI'99 Consistency: Properties & Algorithms

  3. Outline • Motivation and material • Node and arc consistency and their complexity • Criteria for performance comparison and CSP parameters. • Which AC algorithm to choose? Project results may tell! • Path consistency and its complexity • K-consistency, minimality and decomposability • Other related results: box consistency, higher levels of consistency, (i, j)-consistency, etc. Consistency: Properties & Algorithms

  4. Consistency checking Motivation • CSP are solved with search (conditioning) • Search performance is affected by: - problem size - amount of backtracking • Backtracking may yield Thrashing Exploring non-promising sub-trees and rediscovering the same inconsistencies over and over again  Malady of backtrack search Consistency: Properties & Algorithms

  5. V2 V1 {1, 2, 3} {1, 2, 3} V3<V1 V3<V2 V3 { 0, 1, 2} V3>0 V3<V4 V3<V5 V4 V5 {1, 2} { 1, 2} V5<V4 (Some) causes of thrashing Search order: V1, V2, V3, V4, V5 What happens in search if we: • do not check the constraint on V3 (node inconsistency) • do not check constraint between V3 and V5 (arc inconsistency) • do not check constraints between V3—V4, V3—V5, and V4—V5 (path inconsistency) Consistency: Properties & Algorithms

  6. Consistency checking Goal: eliminate inconsistent combinations Algorithms(for binary constraints): • Node consistency • Arc consistency (AC-1, AC-2, AC-3, ..., AC-7, AC-3.1, etc.) • Path consistency (PC-1, PC-2, DPC, PPC, etc.) • Constraints of arbitrary arity: • Waltz algorithm ancestor of AC's • Generalized arc-consistency (Dechter, Section 3.5.1 ) • Relational m-consistency (Dechter, Section 8.1) Consistency: Properties & Algorithms

  7. Overview of Recommended Reading • Davis, AIJ'77 • focuses on the Waltz algorithm • studies its performance and quiescence for given: • constraint types (order relations, bounded diff, algebraic, etc.) • domains types (continuous or finite) • Mackworth, AIJ'77 • presents NC, AC-1, AC-2, PC-1, PC-2 • Mackworth and Freuder, AIJ'85 • studies their complexity  Mackworth and Freuder concentrate on finite domains  More people work on finite domains than on continuous ones  Continuous domains can be quite tough Consistency: Properties & Algorithms

  8. Constraint Propagation with Interval Labels Ernest Davis • `Old' paper (1987): terminology slightly different • Interval labels: continuous domains • Section 8: sign labels (discrete) • Concerned with Waltz algorithm (e.g., quiescence, completeness) • Constraint types vs. domain types (Table 3) • Addresses applications from reasoning • about Physical Systems (circuit, SPAM) • about time relations (TMM) •  Advice: read Section 3, more if you wish Consistency: Properties & Algorithms

  9. Waltz algorithm for label inference • Refine(C(Vi, Vk, Vm, Vn), Vi) - finds a new label for Vi consistent with C. • Revise(C (Vi, Vk, Vm, Vn)) refines the domains of Vi, Vk, Vm, Vn by iterating over these variables until quiescence (no domains is further refined) • Waltz Algorithm revises all constraints by iterating over each constraint connected to a variable whose domain has been revised • Waltz Algorithm ancestor of arc-consistency Consistency: Properties & Algorithms

  10. WARNING → Completeness: Running the Waltz Algorithm does not solve the problem. A=2 and B=3 is still not a solution! →Quiescence: The Waltz algorithm may go into infinite loops even if problem is solvable x  [0, 100] x = y y  [0, 100] x = 2y →Davis provides a classification of problems in terms of completeness and quiescence of the Waltz algorithm f(constraint types, domain types), see Table 3 Consistency: Properties & Algorithms

  11. Importance of this paper  Establishes that constraints of bounded differences (temporal reasoning) can be efficiently solved by the Waltz algorithm (O(n3), n number of variables)  Early paper that attracts attention on the difficulty of label propagation in interval labels (continuous domains). This work has been continued by Faltings (AIJ 92) who studied the early-quiescence problem of the Waltz algorithm. Consistency: Properties & Algorithms

  12. Consistency checking algorithms Examining finite, binary CSPs and their complexity Notation: Given variables i, j, k with values x, y, z, the predicate Pijk(x, y, z) is true iff the 3-tuple x, y, z  Cijk Node consistency: checking Pi(x) Arc consistency: checking Pij(x,y) Path consistency: bit-matrix manipulation Consistency: Properties & Algorithms

  13. time space Worst-case asymptotic complexity Worst-case complexity as a function of the input parameters Upper bound: f(n) is O(g(n)) means that f(n)c.g(n) f grows as g or slower Lower bound:f(n) is  (h(n)) means that f(n)  c.h(n) f grows as g or faster Input parameters for a CSP: n = number of variables a = (max) size of a domain dk = degree of Vk ( n-1) e = number of edges (or constraints)  [(n-1), n(n-1)/2] Consistency: Properties & Algorithms

  14. Outline • Motivation and material • Node & arc consistency and their complexity • Criteria for performance comparison and CSP parameters. • Which AC algorithm to choose? Project results may tell! • Path consistency and its complexity • K-consistency, minimality and decomposability • Other related results: box consistency, higher levels of consistency, (i, j)-consistency, etc. Consistency: Properties & Algorithms

  15. Node consistency (NC) Procedure NC(i): Di Di { x | Pi(x) } Begin for i  1 until n do NC(i) end Consistency: Properties & Algorithms

  16. Complexity of NC Procedure of NC(i) DiDi { x | Pi(x) } Begin for i  1 until n do NC(i) end • For each variable, we check a values • We have n variables, we do n.a checks • NC is O(a.n) Consistency: Properties & Algorithms

  17. Vj Vi Vi Vj 1 1 1 2 2 2 2 3 3 3 Arc-consistency Adapted from Dechter Definition: Given a constraint graph G, • A variable Vi is arc-consistent relative to Vj iff for every value aDVi, there exists a value bDVj | (a, b)CVi,Vj. • The constraint CVi,Vj is arc-consistent iff • Vi is arc-consistent relative to Vj and • Vj is arc-consistent relative to Vi. • A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc-consistent Consistency: Properties & Algorithms

  18. Procedure Revise Revises the domains of a variable i Procedure Revise(i,j): Begin DELETE  false for each x  Dido if there is no y  Dj such that Pij(x, y) then begin delete x from Di DELETE  true end return DELETE end • Revise is directional • What is the complexity of Revise? {a2} Consistency: Properties & Algorithms

  19. Revise: example R. Dechter Apply the Revise procedure to the following example Consistency: Properties & Algorithms

  20. Vj Vi Vi Vj 1 1 1 2 2 2 2 3 3 3 Effect of Revise Adapted from Rina Dechter Question: Given • two variables Vi and Vj • their domains DVi and DVj, and • the constraint CVi,Vj, write the effect of the Revise procedure as a sequence of operations in relational algebra Hint: • Think about the domain DVi as a unary constraint CVi and • consider the composition of this unary constraint and the binary one.. • Solution: DVi  DViVi (CVi,VjDVi) • This is actually equivalent to DViVi (CVi,VjDVi) Consistency: Properties & Algorithms

  21. Arc consistency (AC-1) Procedure AC-1: 1 begin 2 for i 1 untilndo NC(i) 3 Q  {(i, j) | (i,j)  arcs(G), i  j } 4 repeat 5 begin 6 CHANGE  false 7 for each (i, j)  Q do CHANGE  ( REVISE(i, j) or CHANGE ) 8 end 9 until ¬ CHANGE 10 end • AC-1 does not update Q, the queue of arcs • No algorithm can have time complexity below O(ea2) Consistency: Properties & Algorithms

  22. V1 V1 V1 V1 V3 V2 V3 V3 V2 V3 V2 V2 V1 V1 V1 V3 V2 V3 V3 V2 V2 Arc consistency • AC may discover the solution Example borrowed from Dechter Consistency: Properties & Algorithms

  23. Arc consistency 2. AC may discover inconsistency Example borrowed from Dechter Consistency: Properties & Algorithms

  24. x [0, 10] x [0, 7] x [4, 7] x  y-3 x  y-3 x  y-3 y [0, 10] y [3, 10] y [7, 10] NC & AC Example courtesy of M. Fromherz In the temporal problem below • AC propagates bound • Unary constraint x>3 is imposed • AC propagates bounds again Consistency: Properties & Algorithms

  25. Complexity of AC-1 Procedure AC-1: 1 begin 2 for i  1 untilndo NC(i) 3 Q  {(i, j) | (i,j)  arcs(G), i  j 4 repeat 5 begin 6 CHANGE  false 7 for each (i, j)  Q do CHANGE  (REVISE(i, j) or CHANGE) 8 end 9 until ¬ CHANGE 10 end Note: Q is not modified and |Q| = 2e • 4  9 repeats at most n·a times • Each iteration has |Q| = 2e calls to REVISE • Revise requires at most a2 checks of Pij  AC-1 is O(a3· n · e) Consistency: Properties & Algorithms

  26. AC versions • AC-1 does not update Q, the queue of arcs • AC-2 iterates over arcs connected to at least one node whose domain has been modified. Nodes are ordered. • AC-3 same as AC-2, nodes are not ordered Consistency: Properties & Algorithms

  27. Arc consistency (AC-3) AC-3 iterates over arcs connected to at least one node whose domain has been modified Procedure AC-3: 1 begin 2 fori 1 untilndo NC(i) 3 Q { (i, j) | (i, j)  arcs(G), i j } 4 WhileQ is not empty do 5 begin 6 select and delete any arc (k, m) from Q 7 If Revise(k, m) thenQQ { (i, k) | (i, k)  arcs(G), i  k, i  m } 8 end 9 end Consistency: Properties & Algorithms

  28. Procedure AC-3: 1 begin 2 fori 1 untilndo NC(i) 3 Q {(i, j) | (i, j) arcs(G), i j } 4 WhileQ is not empty do 5 begin select and delete any arc (k, m) from Q 7 If Revise(k, m) thenQQ { (i, k) | (i, k)  arcs(G), i  k, i  m } 8 end 9 end First |Q| = 2e, then it grows and shrinks 48 Worst case: 1 element is deleted from DVk per iteration none of the arcs added is in Q Line 7: a·(dk - 1) Lines 4 - 8: Revise: a2 checks of Pij  AC-3 is O(a2(2e + a(2e-n))) Connected graph (en – 1), AC-3 is O(a3e) If AC-p, AC-3 is O(a2e)  AC-3 is (a2e) Complete graph: O(a3n2) Complexity of AC-3 Consistency: Properties & Algorithms

  29. V2 V2 V1 V1 { 1, 2, 3, 4 } { 1, 2, 3, 4, 5 } { 1, 3, 5 } { 1, 2, 3, 4, 5 } { 1, 3, 5 } { 1, 2, 3, 5 } { 1, 2, 3, 4, 5 } { 1, 2, 3, 4, 5 } V4 V3 V4 V3 Example: Apply AC-3 Example: Apply AC-3 Thanks to Xu Lin DV1 = {1, 2, 3, 4, 5} DV2 = {1, 2, 3, 4, 5} DV3 = {1, 2, 3, 4, 5} DV4 = {1, 2, 3, 4, 5} CV2,V3 = {(2, 2), (4, 5), (2, 5), (3, 5), (2, 3), (5, 1), (1, 2), (5, 3), (2, 1), (1, 1)} CV1,V3 = {(5, 5), (2, 4), (3, 5), (3, 3), (5, 3), (4, 4), (5, 4), (3, 4), (1, 1), (3, 1)} CV2,V4 = {(1, 2), (3, 2), (3, 1), (4, 5), (2, 3), (4, 1), (1, 1), (4, 3), (2, 2), (1, 5)} Consistency: Properties & Algorithms

  30. Applying AC-3 Thanks to Xu Lin • Queue = {CV2, V4, CV4,V2, CV1,V3, CV2,V3, CV3, V1, CV3,V2} • Revise(V2,V4): DV2DV2 \ {5} = {1, 2, 3, 4} • Queue = {CV4,V2, CV1,V3, CV2,V3, CV3, V1, CV3, V2} • Revise(V4, V2): DV4 DV4 \ {4} = {1, 2, 3, 5} • Queue = {CV1,V3, CV2,V3, CV3, V1, CV3, V2} • Revise(V1, V3): DV1 {1, 2, 3, 4, 5} • Queue = {CV2,V3, CV3, V1, CV3, V2} • Revise(V2, V3): DV2 {1, 2, 3, 4} Consistency: Properties & Algorithms

  31. Applying AC-3 (cont.) Thanks to Xu Lin • Queue = {CV3, V1, CV3, V2} • Revise(V3, V1): DV3 DV3 \ {2} = {1, 3, 4, 5} • Queue = {CV2, V3, CV3, V2} • Revise(V2, V3): DV2 DV2 • Queue = {CV3, V2} • Revise(V3, V2): DV3 {1, 3, 5} • Queue = {CV1, V3} • Revise(V1, V3): DV1 {1, 3, 5} END Consistency: Properties & Algorithms

  32. Main Improvements Mohr & Henderson (AIJ 86): • AC-3 O(a3e) AC-4 O(a2e) • AC-4 is optimal • Trade repetition of consistency-check operations with heavy bookkeeping on which and how many values support a given value for a given variable • Data structures it uses: • m: values that are active, not filtered • s: lists all vvp's that support a given vvp • counter: given a vvp, provides the number of support provided by a given variable • How it proceeds: • Generates data structures • Prepares data structures • Iterates over constraints while updating support in data structures Consistency: Properties & Algorithms

  33. counter ( V , V ), 3 0 4 2 ( V , V ), 2 0 4 2 ( V , V ), 1 0 4 2 ( V , V ), 1 0 2 3 M M ( V , V ), 4 0 1 3 Step 1: Generate 3 data structures • mand s have as many rows as there are vvp’s in the problem • counterhasas many rows as there are tuples in the constraints Consistency: Properties & Algorithms

  34. Step 2: Prepare data structures Data structures: s and counter. • Checks for every constraint CVi,Vj all tuples Vi=ai, Vj=bj_) • When the tuple is allowed, then update: • s(Vj,bj) s(Vj,bj)  {(Vi, ai)} and • counter(Vj,bj)  (Vj,bj) + 1 Update counter ((V2, V3), 2) to value 1 Update counter ((V3, V2), 2) to value 1 Update s-htable (V2, 2) to value ((V3, 2)) Update s-htable (V3, 2) to value ((V2, 2)) Update counter ((V2, V3), 4) to value 1 Update counter ((V3, V2), 5) to value 1 Update s-htable (V2, 4) to value ((V3, 5)) Update s-htable (V3, 5) to value ((V2, 4)) Consistency: Properties & Algorithms

  35. Constraints CV2,V3 and CV3,V2 Updating m Note that (V3, V2),4  0 thus we remove 4 from the domain of V3 and update (V3, 4)  nil in m Updating counter Since 4 is removed from DV3 then for every (Vk, l) | (V3, 4) s[(Vk, l)], we decrement counter[(Vk, V3), l] by 1 Consistency: Properties & Algorithms

  36. Summary of arc-consistency algorithms • AC-4 is optimal (worst-case) [Mohr & Henderson, AIJ 86] • Warning: worst-case complexity is pessimistic. Better worst-case complexity: AC-4 Better average behavior: AC-3 [Wallace, IJCAI 93] • AC-5: special constraints [Van Hentenryck, Deville, Teng 92] functional, anti-functional, and monotonic constraints • AC-6, AC-7: general but rely heavily on data structures for bookkeeping [Bessière & Régin] • Now, back to AC-3: AC-2000, AC-2001, AC-3.1, AC3.3 • Non-binary constraints: • GAC (general) [Mohr & Masini 1988], • all-different (dedicated) [Régin, 94] Consistency: Properties & Algorithms

  37. Outline • Motivation and material • Node and arc consistency and their complexity • Criteria for performance comparison & CSP parameters. • Which AC algorithm to choose? Project results may tell! • Path consistency and its complexity • K-consistency, minimality and decomposability • Other related results: box consistency, higher levels of consistency, (i, j)-consistency, etc. Consistency: Properties & Algorithms

  38. CSP parameters ‹n, a, t, d› • n is number of variables • a is maximum domain size • t is constraint tightness: • d is constraint density where e is the #constraints, emin=(n-1), and emax = n(n-1)/2 Lately, we use constraint ratiop = e/emax → Constraints in random problems often generated uniform → Use only connected graphs (throw the unconnected ones away) Consistency: Properties & Algorithms

  39. Criteria for performance comparison • Bounding time and space complexity (theoretical) • worst-case, • average-case, • best- case • Counting the number of constraint checks (theoretical, empirical) • Measuring CPU time (empirical) Consistency: Properties & Algorithms

  40. Performance comparisonCourtesy of Lin XU AC-4 AC-7 AC-3 Consistency: Properties & Algorithms

  41. Wisdom (?) Free adaptation from Bessière • When a single constraint check is very expensive to make, use AC-7. • When there is a lot of propagation, use AC-4 • When little propagation and constraint checks are cheap, use AC-3x Advice?Instructor's personal opinion Use AC-3.. Consistency: Properties & Algorithms

  42. Outline • Motivation and material • Node and arc consistency and their complexity • Criteria for performance comparison and CSP parameters. • Which AC algorithm to choose? Project results may tell! • Path consistency and its complexity • K-consistency, minimality and decomposability • Other related results: box consistency, higher levels of consistency, (i, j)-consistency, etc. Consistency: Properties & Algorithms

  43. = = = AC is not enough Example borrowed from Dechter Arc-consistent? Satisfiable?  seek higher levels of consistency V V 1 1 b a a b V V V V 2 3 2 a b 3 b a a b a b Consistency: Properties & Algorithms

  44. V2 V1 Vm-1 Vm V0 for all y  DVm for all x  DV0 Consistency of a path A path (V0, V1, V2, …, Vm) of length m is consistent iff • for any value xDV0 and for any value yDVm that are consistent (i.e., PV0 Vm(x, y)) •  a sequence of values z1, z2, … , zm-1 in the domains of variables V1, V2, …, Vm-1, such that all constraints between them (along the path, not across it) are satisfied (i.e., PV0 V1(x, z1)  PV1 V2(z1, z2)  …  PVm-1 Vm(zm-1, zm) ) Consistency: Properties & Algorithms

  45. V2 V1 Vm-1 Vm V0 for all y  DVm for all x  DV0 Note  The same variable can appear more than once in the path  Every time, it may have a different value  Constraints considered: PV0,Vm and those along the path  All other constraints are neglected Consistency: Properties & Algorithms

  46. V2 V3 V1 V4 {a, b, c} {a, b, c} {a, b, c} {a, b, c} All mutex constraints {a, b, c} {a, b, c} V5 V7 {a, b, c} V6 Example: consistency of a path  Check path length = 2, 3, 4, 5, 6, ....             Consistency: Properties & Algorithms

  47. Path consistency: definition  A path of length m is path consistent  A CSP is path consistent Property of a CSP Definition: A CSP is path consistent (PC) iff every path is consistent (i.e., any length of path) Question: should we enumerate every path of any length? Answer: No, only length 2, thanks to [Mackworth AIJ'77] Consistency: Properties & Algorithms

  48. Making a CSP Path Consistent (PC) Special case: Complete graph Theorem: In a complete graph, if every path of length 2 is consistent, the network is path consistent [Mackworth AIJ'77]  PC-1: two operations, composition and intersection  Proof by induction. Special case: Triangulated graph Theorem: In a triangulated graph, if every path of length 2 is consistent, the network is path consistent [Bliek & Sam-Haroud ‘99]  PPC (partially path consistent)  PC Consistency: Properties & Algorithms

  49. Tools for PC-1 Two operators • Constraint composition: ( • ) R13 = R12• R23 • Constraint intersection: (  ) R13 R13, old R13, induced Consistency: Properties & Algorithms

  50. Path consistency (PC-1) Achieved by composition and intersection (of binary relations expressed as matrices) over all paths of length two. Procedure PC-1: 1 Begin 2 Yn R 3 repeat 4 begin 5 Y0 Yn 6 For k 1 untilndo 7 For i 1 untilndo 8 For j 1 untilndo 9 Ylij Yl-1ij Yl-1ik• Yl-1kk• Yl-1kj 10 end 11 until Yn = Y0 12 Y  Yn 10 end Consistency: Properties & Algorithms