Hybrid of search and inference: time-space tradeoffs chapter 10

Hybrid of search and inference: time-space tradeoffschapter 10 ICS-275 Spring 2007

Chapter 10:Hybrids of Search and Inference • Interleaving conditioning and elimination • Cutset decomposition • AND/OR cutset decomposition • Super-bucket and super-clusters • Approximating conditioning in a hybrid ICS-275

Satisfiability: Inference vs search Search = O(exp(n)) Search = exp(dfs-height) ICS-275

DR versus DPLL: Complementary Properties Uniform random 3-CNFs (large induced width) (k,m)-tree 3-CNFs (bounded induced width) ICS-275

Outline; Road Map Tasks Methods ICS-275

The effect of Conditioning X1 X2 X3 X4 X5 ICS-275

The effect of Conditioning • Select a variable X1 X2 X3 X4 X5 ICS-275

The effect of Conditioning X1 X2 X3 X4 X5 …... X1  a X1  c X1  b X2 X2 X2 X3 X3 X3 …... X4 X5 X4 X5 X4 X5 ICS-275

The effect of Conditioning X1 X2 X3 Condition on X2, and BE or BE(w=2) now. X4 X5 …... X1  a X1  c X1  b X2 X2 X2 X3 X3 X3 …... X4 X5 X4 X5 X4 X5 ICS-275

The effect of Conditioning X1 X2 General principle: Condition until tractable X3 X4 X5 …... X1  a X1  c X1  b X2 X2 X2 X3 X3 X3 …... X4 X5 X4 X5 X4 X5 ICS-275

Interleaving Cond and Elim (VE+C) ICS-275

Interleaving Cond and Elim ICS-275

Interleaving Cond and Elim ... ... ICS-275

ICS-275

DCDR(b): empirical results ICS-275

Road Map • Interleaving conditioning and elimination • Cutset decomposition • W-cutset, cycle-cutset • AND/OR w-cutset • Super-bucket and super-clusters ICS-275

H H H G G G N N N H H G G D D D N N F F F M M M D D A A A A J J J F F M M O O O J J E E E O O E E C C C P P P B B B C C P P B B K K K L L L K K L L B H H H G G G N N N D D D C F F M M F M J J J O O O E E E C C P P P K K L L K L Cycle cutset Cycle cutset = {A,B,C} ICS-275

E M D Graph Coloring problem L C A B K H F G J A=yellow A=green E M E M D D L L C C B B K K H H F F G G J J Conditioning and cycle-cutset • Inference may require too much memory • Condition (guessing) on some of the variables ICS-275

E M D Graph Coloring problem L C A B K H F G J A=yellow A=green B=red B=blue B=green B=red B=blue B=yellow E E E E E E M M M M M M D D D D D D L L L L L L C C C C C C K K K K K K H H H H H H F F F F F F G G G G G G J J J J J J Conditioning • Inference may require too much memory • Condition on some of the variables ICS-275

The cycle-cutset scheme: condition until a tree ICS-275

Time-space tradeoff Theorem: The w-cutset scheme yields space complexity exp(w) and time complexity exp(w+c_w), where c_w is the size of the w-cutset. As w decreases, c_w increases. Thecycle-cutset decomposition is linear space and Has time complexity of ICS-275

Elim-cond(w) or w-cutset scheme • Idea: runs backtracking search on the w-cutset variables and bucket-elimination on the remaining variables. • Input: A constraint network R = (X,D,C), Y a w-cutset, d an ordering that starts with Y whose adjusted induced-width, along d, is bounded by w, Z = X-Y. • Output: A consistent assignment, if there is one. • 1. while {y}  next partial solution of Y found by backtracking, do • a) z  solution found by adaptive-consistency(R_y). • B) if z is not false, return solution (y,z). • 2. endwhile. • return: the problem has no solutions. ICS-275

Finding a w-cutset • Verifying a w-cutset can be done in polynomial time • A simple greedy: use a good induced-width ordering and starting at the top add to the w-cutset any variable with more than w parents. • Alternative: generate a tree-decomposition and select a w-cutset that reduce each cluster below w (Bidyuk and Dechter UAI2005). ICS-275

Treewidth equals cycle cutset G D G A D A B F C C E L K L K H M M J J N N treewidth = cycle cutset = 4 ICS-275

Treewidth smaller than cycle cutset G D G B D A B F F C C E E K H L J treewidth = 2 cycle cutset = 5 ICS-275

Time-Space complexity of of w-cutset • Space: O(exp(w)) • W-cutset: a set that when removed the induced-width is w. • c(w): size of w-cutset. • Time: O(exp(w+c(w))) on OR space ICS-275

Time vs space • Random Graphs (50 nodes, 200 edges, average degree 8, w*23) 60 Branch and bound 50 40 Bucket elimination W+c(w) 30 20 10 0 2 5 8 -1 11 14 17 20 23 26 29 32 w ICS-275

DCDR(b): empirical results(Rish and Dechter 2000) ICS-275

Empirical evaluation of w-cutset • Cycle-cutset for CSPs (dechter 1990) • Alternating w-cutset for cnfs (Rish and Dechter 2000) • Alternating w-cutset for Constraint optimization (Larrosa and Dechter, 2002) ICS-275

The idea of super-buckets Larger super-buckets (cliques) =>more time but less space • Complexity: • Time: exponential in clique (super-bucket) size • Space: exponential in separator size ICS-275

Example ICS-275

Super-Bucket Elimination, SBE(k) • Eliminate sets of variables such that: • individual eliminations are too costly in space (namely, each variable in the set has degree larger than k) • the join degree is lower than k ICS-275

Sep-based time-space tradeoff • Let T be a tree-decomposition of hypergraph H. Let s_0,s_1,...,s_n be the sizes of the separators in T, listed in strictly descending order. With each separator size s_i we associate a secondary tree decomposition T_i, generated by combining adjacent nodes whose separator sizes are strictly greater than s_i. • We denote by r_i the largest set of variables in any cluster of T_i. • Note that as s_i decreases, r_i increase. • Theorem: The complexity of CTE when applied to each T_i is O( n exp(r_i)) time, and O( n exp(s_i)) space. ICS-275

Complexity • Theorem: If R = (X,D,C) is a constraint network whose constraint graph has non-separable components of at most size r, then the super-bucket elimination algorithm, whose buckets are the non-separable components, is time exponential O(n exp(r)) and is linear in space. ICS-275

Hybrids of hybrids • hybrid(b_1,b_2): • First, a tree-decomposition having separators bounded by b_1 is created, followed by application of the CTE algorithm, but each clique is processed by elim-cond(b_2). If c^*_{b_2} is the size of the maximum b_2-cutset in each clique of the b_1-tree-decomposition, the algorithm is space exponential in b_1 but time exponential in c^*_{b_2}. • Special cases: • hybrid(b_1,1): Applies cycle-cutset in each clique. • b_1 = b_2. For b=1, hybrid(1,1) is the non-separable components utilizing the cycle-cutset in each component. • The space complexity of this algorithm is linear but its time complexity can be much better than the cycle-cutsets cheme or the non-separable component approach alone. ICS-275

Case study: combinatorial circuits: benchmark used for fault diagnosis and testing community Problem: Given a circuit and its unexpected output, identify faulty components. The problem can be modeled as a constraint optimization problem and solved by bucket elimination. ICS-275

Case study: C432 Join-tree of c432 Seperator size is 23 A circuit’s primal graph For every gate we connect inputs and outputs ICS-275

Join-tree of C3540 (1719 vars)max sep size 89 ICS-275

Hybrid of search and inference: time-space tradeoffs chapter 10