Loading in 2 Seconds...

CSCE 896-004, Fall 2009 cse.unl/~choueiry/F09-896-004/ Questions : cse421@cse.unl

Loading in 2 Seconds...

- 80 Views
- Uploaded on

Download Presentation
## PowerPoint Slideshow about 'CSCE 896-004, Fall 2009 cse.unl/~choueiry/F09-896-004/ Questions : cse421@cse.unl' - reed-sullivan

**An Image/Link below is provided (as is) to download presentation**

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

CSCE 896-004, Fall 2009

www.cse.unl.edu/~choueiry/F09-896-004/

Questions: cse421@cse.unl.edu

Broadcast: cse421-ml@cse.unl.edu

Berthe Y. Choueiry (Shu-we-ri)

AVH 360

choueiry@cse.unl.edu

Tel: +1(402)472-5444

Overview 1

Outline

- Motivating example, application areas
- CSP: Definition, representation
- Some simple modeling examples
- More on definition and formal characterization
- Basic solving techniques
- (Implementing backtrack search)
- Advanced solving techniques
- Issues & research directions

Overview 1

Motivating example

- Context: You are a senior in college
- Problem: You need to register in 4 courses for the Spring semester
- Possibilities: Many courses offered in Math, CSE, EE, CBA, etc.
- Constraints: restrict the choices you can make
- Unary: Courses have prerequisites you have/don't have Courses/instructors you like/dislike
- Binary: Courses are scheduled at the same time
- n-ary: In CE: 4 courses from 5 tracks such as at least 3 tracks are covered
- You have choices, but are restricted by constraints
- Make the right decisions!!

Overview 1

Motivating example (cont’d)

- Given
- A set of variables: 4 courses at UNL
- For each variable, a set of choices (values)
- A set of constraints that restrict the combinations of values the variables can take at the same time
- Questions
- Does a solution exist? (classical decision problem)
- How two or more solutions differ? How to change specific choices without perturbing the solution?
- If there is no solution, what are the sources of conflicts? Which constraints should be retracted?
- etc.

Overview 1

Practical applications

Adapted from E.C. Freuder

- Radio resource management (RRM)
- Databases (computing joins, view updates)
- Temporal and spatial reasoning
- Planning, scheduling, resource allocation
- Design and configuration
- Graphics, visualization, interfaces
- Hardware verification and software engineering
- HC Interaction and decision support
- Molecular biology
- Robotics, machine vision and computational linguistics
- Transportation
- Qualitative and diagnostic reasoning

Overview 1

Constraint Processing

- is about ...
- Solving a decision problem…
- … While allowing the user to state arbitrary constraints in an expressive way and
- Providing concise and high-level feedback about alternatives and conflicts
- Related areas:
- AI, OR, Algorithmic, DB, TCS, Prog. Languages, etc.

Overview 1

reinforce

Power of Constraints Processing- Flexibility & expressiveness of representations
- Interactivity
- users can constraints

Overview 1

Outline

- Motivating example, application areas
- CSP: Definition, representation
- Some simple modeling examples
- More on definition and formal characterization
- Basic solving techniques

Overview 1

Defining a problem

- General template of any computational problem
- Given:
- Example: a set of objects, their relations, etc.
- Query/Question:
- Example: Find x such that the condition y is satisfied
- How about the Constraint Satisfaction Problem?

Overview 1

Definition of a CSP

- GivenP = (V, D, C )
- V is a set of variables,
- D is a set of variable domains (domain values)
- C is a set of constraints,
- Query: can we find a value for each variable such that all constraints are satisfied?

Overview 1

Other queries…

- Find a solution
- Find all solutions
- Find the number of solutions
- Find a set of constraints that can be removed so that a solution exists
- Etc.

Overview 1

Representation I

- Given P = (V, D, C ), where
- Find a consistent assignment for variables

- Constraint Graph
- Variable node (vertex)
- Domain node label
- Constraint arc (edge) between nodes

Overview 1

V1

{1, 2, 3, 4}

{ 3, 6, 7 }

V2

v1+v3 < 9

v2 > v4

v2 < v3

{ 3, 4, 9 }

{ 3, 5, 7 }

V3

V4

Representation II

- Given P = (V, D, C ), where

- Example I:
- Define C ?

Overview 1

Outline

- Motivating example, application areas
- CSP: Definition, representation
- Some simple modeling examples
- More on definition and formal characterization
- Basic solving techniques

Overview 1

Example II: Temporal reasoning

- Give one solution: …….
- Satisfaction, yes/no: decision problem

Overview 1

Example III: Map coloring

Using 3 colors (R, G, & B), color the US map such that no two adjacent states have the same color

- Variables?
- Domains?
- Constraints?

Overview 1

Example III: Map coloring (cont’d)

Using 3 colors (R, G, & B), color the US map such that no two adjacent states have the same color

Overview 1

{ a, b } { a, c, d }

{ b, c, d }

Example IV: Resource AllocationWhat is the CSP formulation?

Overview 1

{ a, b } { a, c, d }

{ b, c, d }

Constraint Graph

T1

{ R1, R3 }

T4

T6

{ R1, R2, R3 }

{ R2, R4 }

{ R1, R3 }

{ R2, R4 }

{ R1, R2, R3 }

T7

T3

{ R1, R3 }

T5

Example IV: RA (cont’d)Interval Order

{ R1, R3 }

T1

{ R1, R3 }

T2

{ R1, R3, R4 }

{ R1, R3 }

T2

T4

T3

{ R1, R2, R3 }

T5

{ R2, R4 }

T6

{ R2, R4 }

T7

Overview 1

Example V: Puzzle

- Given:
- Four musicians: Fred, Ike, Mike, and Sal, play bass, drums, guitar and keyboard, not necessarily in that order.
- They have 4 successful songs, ‘Blue Sky,’ ‘Happy Song,’ ‘Gentle Rhythm,’ and ‘Nice Melody.’
- Ike and Mike are, in one order or the other, the composer of ‘Nice Melody’ and the keyboardist.
- etc ...
- Query: Who plays which instrument and who composed which song?

Overview 1

Example V: Puzzle (cont’d)

- Formulation 1:
- Variables: Bass, Drums, Guitar, Keyboard, Blue Sky, Happy Song
- Gentle Rhythm and Nice Melody.
- Domains: Fred, Ike, Mike, Sal
- Constraints: …
- Formulation 2:
- Variables: Fred's-instrument, Ike's-instrument, …,
- Fred's-song, Ikes's-song, Mike’s-song, …, etc.
- Domains:
- { bass, drums, guitar, keyboard }
- { Blue Sky, Happy Song, Gentle Rhythm, Nice Melody}
- Constraints: …

Overview 1

Example VI: Product Configuration

- Train, elevator, car, etc.
- Given:
- Components and their attributes (variables)
- Domain covered by each characteristic (values)
- Relations among the components (constraints)
- A set of required functionalities (more constraints)
- Find: a product configuration
- i.e., an acceptable combination of components
- that realizes the required functionalities

Overview 1

Example VII: Cryptarithmetic puzzles

- DX1 = DX2 = DX3 = {0,1}
- DF=DT=DU=DW=DR=DO=[0,9]
- O+O = R+10X1
- X1+W+W = U+10X2
- X2+ T+T = O + 10X3
- X3=F
- Alldiff({F,D,U,V,R,O})

Overview 1

Constraint Graph

V1

{1, 2, 3, 4}

{ 3, 6, 7 }

T1

V2

{ R1, R3 }

T4

T6

{ R1, R2, R3 }

v1+v3 < 9

v2 > v4

v2 < v3

{ R2, R4 }

{ R1, R3 }

{ 3, 4, 9 }

{ 3, 5, 7 }

V3

V4

{ R2, R4 }

{ R1, R2, R3 }

T7

T3

{ R1, R3 }

T5

Constraint types: examples- Example I: algebraic constraints
- Example II:

(algebraic) constraints

of bounded difference

- Example III & IV: coloring, mutual exclusion, difference constraints
- Example V & VI: elements of C must be made explicit

Overview 1

More examples

- Example VII:Databases
- Join operation in relational DB is a CSP
- View materialization is a CSP
- Example VIII: Interactive systems
- Data-flow constraints
- Spreadsheets
- Graphical layout systems and animation
- Graphical user interfaces
- Example IX: Molecular biology (bioinformatics)
- Threading, etc

Overview 1

Outline

- Motivating example, application areas
- CSP: Definition, representation
- Some simple modeling examples
- More on definition and formal characterization
- Basic solving techniques

Overview 1

a, b

V3

V2

a, c

b, c

Representation (again)Macrostructure G(P):

- constraint graph for

binary constraints

- constraint network for

non-binary constraints

Micro-structure (P):

Co-microstructure co-(P):

(V1, a ) (V1, b)

(V2, a ) (V2, c)

(V3, b ) (V3, c)

no goods

(V1, a ) (V1, b)

(V2, a ) (V2, c)

(V3, b ) (V3, c)

Overview 1

V1

{1, 2, 3, 4}

{ 3, 6, 7 }

V2

v1+v3 < 9

v2 > v4

v2 < v3

{ 3, 4, 9 }

{ 3, 5, 7 }

V3

V4

Constraint arity I

- Given P = (V, D, C ) , where

- How to represent the constraint V1 + V2 + V4 < 10?

Overview 1

- Given P = (V, D, C ) , where

Constraints: universal, unary, binary, ternary, … , global.

A Constraint Network:

Overview 1

Domain types

- P = (V, D, C ) where
- Domains:
- Restricted to {0,1}: Boolean CSPs
- Finite (discrete), enumeration works
- Continuous, sophisticated algebraic techniques are needed
- Consistency techniques on domain bounds

Overview 1

Constraint terminology

- Scope: The set of variables to which the constraint applies
- Arity: unary, binary, ternary, …, global
- Definition:
- Extension: all tuples are enumerated
- Intension: when it is not practical (or even possible) to list all tuples
- Define types/templates of common constraints to be used repeatedly: linear constraints, All-Diff (mutex), Atmost, TSP-constraint, cycle-constraint, etc.)
- Implementation:
- Predicate function
- Set of tuples (list or table)
- Binary matrix (bit-matrix)
- Constrained Decision Diagrams ([Cheng & Yap, AAAI 05])
- etc.

Overview 1

Complexity of CSP

Characterization:

- Decision problem
- In general, NP-complete

by reduction from 3SAT

Overview 1

Proving NP-completeness

- Show that 1 is in NP
- Given a problem 1 in NP, show that an known NP-complete problem 2 can be efficiently reduced to 1
- Select a known NP-complete problem 2 (e.g., SAT)
- Construct a transformationf from 2 to 1
- Prove that f is a polynomial transformation

(Check Chapter 3 of Garey & Johnson)

Overview 1

What is SAT?

Given a sentence:

- Sentence: conjunction of clauses
- Clause: disjunction of literals
- Literal: a term or its negation
- Term: Boolean variable

Question: Find an assignment of truth values to the Boolean variables such the sentence is satisfied.

Overview 1

CSP is NP-Complete

- Verifying that an assignment for all variables is a solution
- Provided constraints can be checked in polynomial time
- Reduction from 3SAT to CSP
- Many such reductions exist in the literature (perhaps 7 of them)

Overview 1

Problem reduction

Example: CSP into SAT (proves nothing, just an exercise)

Notation: variable-value pair = vvp

- vvp term
- V1 = {a, b, c, d} yields x1 = (V1, a), x2 = (V1, b), x3 = (V1, c), x4 = (V1, d),
- V2 = {a, b, c} yields x5 = (V2, a), x6 = (V2, b), x7 = (V2,c).
- The vvp’s of a variable disjunction of terms
- V1 = {a, b, c, d} yields
- (Optional) At most one VVP per variable

Overview 1

CSP into SAT (cont.)

Constraint:

- Way 1: Each inconsistent tuple one disjunctive clause
- For example: how many?
- Way 2:
- Consistent tuple conjunction of terms
- Each constraint disjunction of these conjunctions

transform into conjunctive normal form (CNF)

Question: find a truth assignment of the Boolean variables such that the sentence is satisfied

Overview 1

Outline

- Motivating example, application areas
- CSP: Definition, representation
- Some simple modeling examples
- More on definition and formal characterization
- Basic solving techniques
- Modeling and consistency checking
- Constructive, systematic search
- Iterative improvement, local search

Overview 1

Before starting search!

Consider:

- Importance of modeling/formulation:
- To control the size of the search space
- Preprocessing
- A.k.a. constraint filtering/propagation, consistency checking
- reduces size of search space

Overview 1

V1

V2

V3

V4

4 Rows

4 Column positions

V11 V12 V13 V14

16 Cells

V21 V22 V23 V24

V31 V32 V33 V34

V41 V42 V43 V44

Importance of modeling- N-queen: formulation 1
- Variables?
- Domains?
- Size of CSP?
- N-queens: formulation 2
- Variables?
- Domains?
- Size of CSP?

{0,1}

Overview 1

1- B: [ 5 .. 14 ]

14

C: [ 6 .. 15 ]

2- A: [ 2 .. 10 ]

2

C: [ 6 .. 14 ]

14

6

Constraint checking- Arc-consistency

B

A < B

A

[ 5.... 18]

B < C

[ 1.... 10 ]

2 < C - A < 5

3- B: [ 5 .. 13 ]

[ 4.... 15]

C

Overview 1

Constraint checking

- Arc-consistency: every combination of two adjacent variables
- 3-consistency, k-consistency (k n)
- Constraint filtering, constraint checking, etc..
- Eliminate non-acceptable tuples prior to search
- Warning: arc-consistency does not solve the problem

still is not a solution!

Overview 1

Systematic search

- Starting from a root node
- Consider all values for a variable V1
- For every value for V1, consider all values for V2
- etc..
- For n variables, each of domain size d
- Maximum depth? fixed!
- Maximum number of paths? size of search space, size of CSP

Overview 1

Systematic search (I)Back-checking

- Systematic search generates dnpossibilities
- Are all possibilities acceptable?
- Expand a partial solution only when it is consistent
- This yields early pruning of inconsistent paths

Overview 1

Systematic search (II)Chronological backtracking

What if only one solution is needed?

- Depth-first search & Chronological backtracking
- DFS: Soundness? Completeness?

Overview 1

Systematic search (III)Intelligent backtracking

What if the reason for failure was higher up in the tree?

Backtrack to source of conflict !!

- Backjumping, conflict-directed backjumping, etc.

Overview 1

Systematic search (IV)Ordering heuristics

- Which variable to expand first?
- Heuristics:
- most constrained variable first (reduce branching factor)
- most promising value first (find quickly first solution)

Overview 1

1

2

2

2

4

1

3

4

1

3

1

4

1

2

4

4

1

1

2

3

3

2

2

Q

Q

Q

Q

Q

Q

Q

1

3

Solution!

26 nodes visited.

Systematic search (V)Back-checking- Search tree with only backtrack search?

Root node

Q

Overview 1

1

1

3

4

3

4

2

Q

Q

Q

Q

Q

Domain Wipe Out V3

Q

Q

Domain Wipe Out V4

Q

Solution!

8 nodes visited.

Systematic search (VI)Forward checkingSearch Tree with domains filter by Forward Check

Root node

Overview 1

Improving BT search

- General purpose methods for
- Variable, value ordering
- Improving backtracking: intelligent backtracking avoids repeating failure
- Look-ahead techniques: propagate constraints as variables are instantiated

Overview 1

Search-tree branching

- K-way branching
- One branch for every value in the domain
- Binary branching
- One branch for the first value
- One branch for all the remaining values in the domain
- Used in commercial constraint solvers: ILOG, Eclipse
- Have different behaviors (e.g., WRT value ordering heuristics [Smith, IJCAI 95])

Overview 1

Summary of backtrack search

- Constructive, systematic, exhaustive
- In general sound and complete
- Back-checking: expands nodes consistent with past
- Backtracking: Chronological vs. intelligent
- Ordering heuristics:
- Static
- Dynamic variable
- Dynamic variable-value pairs
- Looking ahead:
- Partial look-ahead
- Forward checking (FC)
- Directional arc-consistency (DAC)
- Full (a.k.a. Maintaining Arc-consistency or MAC)

Overview 1

CSP: a decision problem (NP-complete)

- Modeling: abstraction and reformulation
- Preprocessing techniques:
- eliminate non-acceptable tuples prior to search
- Systematic search:
- potentially dn paths of fixed lengths
- chronological backtracking
- intelligent backtracking
- variable/value ordering heuristics
- Search ‘hybrids:’
- Mixing consistency checking with search: look-ahead strategies

Overview 1

Non-systematic search

- Iterative repair, local search: modifies a global but inconsistent solution to decrease the number of violated constraints
- Examples: Hill climbing, taboo search, simulated annealing, GSAT, WalkSAT, Genetic Algorithms, Swarm intelligence, etc.
- Features: Incomplete & not sound
- Advantage:anytime algorithm
- Shortcoming:cannot tell that no solution exists

Overview 1

Outline of CSP 101

- We have seen
- Motivating example, application areas
- CSP: Definition, representation
- Some simple modeling examples
- More on definition and formal characterization
- Basic solving techniques
- We will move to
- (Implementing backtrack search)
- Advanced solving techniques
- Issues & research directions

Overview 1

Download Presentation

Connecting to Server..