1 / 48

Life must be lived forward, but can only be understood backward.--Kierkegaard

Life must be lived forward, but can only be understood backward.--Kierkegaard. Lp relaxation. PGs can be used as a basis for finding plans directly. If there exists a k-length plan, it will be a subgraph of the k-length planning graph.

emilen
Download Presentation

Life must be lived forward, but can only be understood backward.--Kierkegaard

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Life must be lived forward, but can only be understood backward.--Kierkegaard Lp relaxation

  2. PGs can be used as a basis for finding plans directly If there exists a k-length plan, it will be a subgraph of the k-length planning graph. (see the highlighted subgraph of the PG for our example problem)

  3. Finding the subgraphs that correspond to valid solutions.. --Can use specialized graph travesal techniques --start from the end, put the vertices corresponding to goals in. --if they are mutex, no solution --else, put at least one of the supports of those goals in --Make sure that the supports are not mutex --If they are mutex, backtrack and choose other set of supports. {No backtracking if we have no mutexes; basis for “relaxed plans”} --At the next level subgoal on the preconds of the support actions we chose. --The recursion ends at init level --Consider extracting the plan from the PG directly -- This search can also be cast as a CSP Variables: literals in proposition lists Values: actions supporting them Constraints: Mutex and Activation The idea behind Graphplan

  4. A5 P1 P1 P1 A6 A1 G1 X I1 X P2 P2 A7 G2 I2 A2 P3 P3 A8 G3 P4 P4 I3 A9 G4 A3 X P5 P5 A10 P6 P6 P6 A4 A11 Backward search in Graphplan Animated

  5. The Story Behind Memos… • Memos essentially tell us that a particular set S of conditions cannot be achieved at a particular level k in the PG. • We may as well remember this information—so in case we wind up subgoaling on any set S’ of conditions, where S’ is a superset of S, at that level, you can immediately declare failure • “Nogood” learning—Storage/matching cost vs. benefit of reduced search.. Generally in our favor • But, just because a set S={C1….C100} cannot be achieved together doesn’t necessarily mean that the reason for the failure has got to do with ALL those 100 conditions. Some of them may be innocent bystanders. • Suppose we can “explain” the failure as being caused by the set U which is a subset of S (say U={C45,C97})—then U is more powerful in pruning later failures • Idea called “Explanation based Learning” • Improves Graphplan performance significantly…. [Rao, IJCAI-99; JAIR 2000]

  6. Graphplan Trivia • One can see the process of extracting the plan as verifying that at least one execution thread is devoid of n-ary mutexes • As long as we mark statically interfering actions, Graphplan backward search will be sound • The rest of the mutex propagation is only to improve efficiency

  7. N-ary mutexes.. • In general, when we talk about n-ary mutexes, we refer to minimal n-ary mutexes • An n-ary mutex is considered “minimal” if no subset of it is also a mutex • Graphplan memos are non-minimal n-ary mutexes while EBL memos are more likely to be minimal n-ary mutexes • A (minimal) n-ary mutex at level i is considered hard n-ary mutex if it depends on minimal m-ary mutexes at levels [1.. i-1] where m > n. If not, it is considered easy • The binary mutex propagation of Graphplan is guaranteed to detect only easy 1-ary and 2-ary mutexes (and non-minimal higher order mutexes). • EBL based memos can be seen as unearthing hard mutexes (of all orders)

  8. Graphplan is claimed to give “step-optimal” plan (where each step may have multiple actions) Only if we assume that actions must not interfere even on irrelevant variables (consider A1 and A2 give g1 and g2; A1 adds p and A2 deletes p. Do you consider A1 and A2 to be parallelly executable?) Can we make it give “length-optimal” plan (length=#actions)? Can we make it give “cost-optimal” plan? (cost = Sum of action costs; with actions having differing costs) Graphplan and Optimality

  9. Alternative Combinatorial Substrates • The problem of finding a valid plan from the planning graph can be encoded on any combinatorial substrate • Alternatives: • CSP [GP-CSP] • SAT [Blackbox; SATPLAN] • IP [Vossen et. Al]

  10. Compilation to SAT [Kautz & Selman] Goals: In(A),In(B) SAT is CSP with Boolean Variables Init: At-R-E-0 & At-A-E-0 & At-B-E-0 Goal: In-A-1 & In-B-1 Graph: “cond at k => one of the supporting actions at k-1” In-A-1 => Load-A-1 In-B-1 => Load-B-1 At-R-M-1 => Fly-R-1 At-R-E-1 => P-At-R-E-1 Load-A-1 => At-R-E-0 & At-A-E-0 “Actions => preconds” Load-B-1 => At-R-E-0 & At-B-E-0 P-At-R-E-1 => At-R-E-0h ~In-A-1 V ~ At-R-M-1 ~In-B-1 V ~At-R-M-1“Mutexes”

  11. Goals: In(A),In(B) Compilation to CSP [Do & Kambhampati, 2000] CSP: Given a set of discrete variables, the domains of the variables, and constraints on the specific values a set of variables can take in combination, FIND an assignment of values to all the variables which respects all constraints Variables: Propositions (In-A-1, In-B-1, ..At-R-E-0 …) Domains: Actions supporting that proposition in the plan In-A-1 : { Load-A-1, #} At-R-E-1: {P-At-R-E-1, #} Constraints: Mutual exclusion ~[ ( In-A-1 = Load-A-1) & (At-R-M-1 = Fly-R-1)] ; etc.. Activation In-A-1 != # & In-B-1 != # (Goals must have action assignments) In-A-1 = Load-A-1 => At-R-E-0 != # , At-A-E-0 != # (subgoal activation constraints) [Corresponds to a regression-based proof]

  12. Blackbox • Blackbox solves planning problems by converting them into SAT. • Very fast • Tried different solvers • Local search (GSAT) • Systematic search with EBL (RelSAT) • In 2000, GP-CSP could beat Blackbox • But in 2001, a newer “SUPER-DUPER” SAT solver called CHAFF was developed, that significantly speeds up Graphplan • By folks in CAD community (EE folks—see getting other people to do our work) • Currently, CSP people are trying to copy over the ideas from CHAFF to CSP.

  13. CSP Encodings can be more compact: GP-CSP Do & Kambhampati, 2000

  14. Compilation to Integer Linear Programming ILP: Given a set of real valued variables, a linear objective function on the variables, a set of linear inequalities on the variables, and a set of integrality restrictions on the variables, Find the values of the feasible variables for which the objective function attains the maximum value -- 0/1 integer programming corresponds closely to SAT problem • Motivations • Ability to handle numeric quantities, and do optimization • Heuristic value of the LP relaxation of ILP problems • Conversion • Convert a SAT/CSP encoding to ILP inequalities • E.g. X v ~Y v Z => x + (1 - y) + z >= 1 • Explicitly set up tighter ILP inequalities (Cutting constraints) • If X,Y,Z are pairwise mutex, we can write x+y+z <= 1 (instead of x+y <=1 ; y+z <=1 ; z +x <= 1) [ Walser & Kautz; Vossen et. al; Bockmayr & Dimopolous]

  15. Empirical performance of IP Encodings • As seen in the previous slide, IP encodings are not competitive with SAT/CSP encodings for classical planning • Not surprising because we are neither exercising the “optimizing” properties of IP, nor are we exercising its ability to encode numerical variables • IP may well do better if we are interested in finding “cost-optimal” plans of a given makespan (length) • No one has confirmed this as yet… • IP encodings may well be more useful when we have numerical quantities such as actions consuming metric resources (preconditions and effects involve numbers) • See the work by Kautz and Walser(AAAI-99) or LPSat (Wolfman/Weld—IJCAI 99).

  16. Relative Tradeoffs Offered bythe various compilation substrates • CSP encodings support implicit representations • More compact encodings [Do & Kambhampati, 2000] • Easier integration with Scheduling techniques • ILP encodings support numeric quantities • Seamless integration of numeric resource constraints [Walser & Kautz, 1999] • Not competitive with CSP/SAT for problems without numeric constraints • SAT encodings support axioms in propositional logic form • May be more natural to add (for whom ;-) • Very active research on finding faster solvers.. • Davis Putnam ( Satz; RelSat; Chaff) • Gsat

  17. Need to adapt techniques already known in CSP/SAT etc Can exploit approaches for compacting the plan Can make the search incremental across iterations Can exploit the latest advances in SAT/CSP solvers Compilation stage can be time consuming, leads to memory blow-up Makes it harder to exploit search from previous iterations Makes it easier to add declarative control knowledge Direct vs. compiled solution extraction Compiled DIRECT

  18. LPG: Local Search on Planning Graphs • The winner of the 2003 planning competition was “LPG” • LPG does local search on the planning graphs directly (rather than, say, convert to SAT and use GSAT/WalkSAT). • Select a seed subgraph of a k-length graph • If it is solution; you are done. • If not “Repair” it • Repairs include adding/deleting action/precondition nodes • + expanding the planning graph • Repair techniques tend to be more tuned to planning graphs, and are thus more effective in solving planning problems. [A short paper is in AIPS 2002. http://prometeo.ing.unibs.it/lpg]

  19. But why go from Planning Graph to CSP/SAT/ILP?Why not directly make those encodings?Planning and Model Checking

  20. Basically GP minimum Mutex propagation (which will add more derived constraints)

  21. Mutex propagation as CSP pre-processing • Suppose we start with a PG that only marks every pair of “interfering” actions as mutex • Any pair of non-noop actions are interfering • Any pair of actions are interfering if one gives P and other gives or requires ~P • No propagation is done • Converting this PG and CSP and solving it will still give a valid solution (if there is one) • So what is mutex propagation doing? • It is “explicating” implicit constraints • A special subset of “3-consistency” enforcement • Recall that enforcing k-consistency involves adding (k-1)-ary constraints • *Not* full 3-consistency (which can be much costlier) • So enforcing the consistency on PG is cheaper than enforcing it after conversion to CSP...

  22. Should we bother with Proof-based encodings? • Clearly Graphplan-based encodings are better than backward-proof based encodings • Because Graphplan does Mutex propagation • And mutex propagation is harder to do on direct SAT • And yet, it is useful to know how to set up encodings directly—since if we don’t know how to do directed consistency enforcement, then we at least know how to solve the problem • Particularly useful for non-classical problems, where we haven’t yet found really effective mutex propagation procedures. • If “neural networks are often the second best way of solving any problem”, you can say that direct SAT(or other) encodings are the second best way of solving planning problems • Which is better than “no way at all”

  23. Life must be lived forward, but can only be understood backward.--Kierkegaard Lp relaxation

  24. Connections to Refinement Planning Framework

  25. Disjunctive Planning • Idea: Consider Partial plans with disjunctive step, ordering, and auxiliary constraints • Motivation: Provides a lifted search space, avoids re-generating the same failures multiple times (also, rich connections to combinatorial problems) • The “solution check” now involves searching through the space of all (exponentially many) minimal candidates of the plan to see if any of them is a solution • Issues: • Refining disjunctive plans • Graphplan (Blum & Furst, 95) • Solution extraction in disjunctive plans • Direct combinatorial search • Compilation to CSP/SAT/ILP

  26. 0 1: Load(A) 1: Load(A) or   0 0 1: Load(B) 2 : Load(B) or  0 3 : Fly(R) 1: Fly(R) Disjunctive Representations --Allow disjunctive step, ordering and auxiliary constraints in partial plans < 1,In(A),  > V < 1 ,In(B),  > In(A)  0 1: Load(A) Load(A) In(x)@   0 1: or In(B) Load(B)  0 1: Load(B) In(x)@  At(A,E)@1 V At(B,E)@1

  27. 1: Load(A) or  0 2 : Load(B) or 3 : Fly(R)  0 2: Fly(R) 1: Load(A)  0 2: Fly(R) 1: Load(B)  0 2: Unload(A,E) 1: Load(A)  0 2: Unload(B,E) 1: Load(B)  0 2: Load(B) 1: Load(A)  0 2: Load(A) 1: Load(B)  0 1: Fly(R) Refining Disjunctive Plans (1) Indirect unioned approach + Maximum pruning power - Exponential refinement cost INFEASIBLE

  28. union of states In(A) In(B) At(R,M) At(R,E) At(A,E) At(B,E) At(A,M) At(B,M) 1: Load(A) 1: Load(A) or or 0  0  2 : Load(B) 2 : Load(B) or or 3 : Fly(R) 3 : Fly(R) Refining Disjunctive plans (2) Direct naive approach Put all actions at all levels --Proposition list contains all conditions + Trivial to construct - Loss of pruning power (progressivity) => too many (minimal) candidates => Costlier solution extraction 1: Load(A,E) or 2 : Load(B,E) or 3 : Fly(R) or 4 : Unload(A,E) USED in SATPLAN or 5 : Unload(B,E) 6 : Unload(A,M) 7 : Unload(B,M) 8 : load(A,M) 9 : load(B,M)

  29. union of states In(A) In(B) At(R,M) At(R,E) At(A,E) At(B,E) At(A,M) At(B,M) 1: Load(A) or 2 : Load(B) or 3 : Fly(R) 1: Load(A) 1: Load(A) or or or 0  4 : Unload(A,E) 0  2 : Load(B) 2 : Load(B) or or or 5 : Unload(B,E) 3 : Fly(R) 3 : Fly(R) or 6 : Unload(A,M) or 7 : Unload(B,M) Refining Disjunctive plans (2) Direct naive approach Put all actions at all levels --Proposition list contains all conditions + Trivial to construct - Loss of pruning power (progressivity) => too many (minimal) candidates => Costlier solution extraction USED in SATPLAN

  30. In(A) In(B) At(R,M) At(R,E) At(A,E) At(B,E) At(A,M) At(B,M) 1: Load(A) 1: Load(A) or or 0  0  2 : Load(B) 2 : Load(B) or or 3 : Fly(R) 3 : Fly(R) Refining Disjunctive plans (3) Enforce partial 1-consistency Proposition list avoids unsupported conditions + Polynomial refinement time -- Loss of pruning power (progressivity) 1: Load(A,E) or 2 : Load(B,E) or 3 : Fly(R) or 4 : Unload(A,E) or 5 : Unload(B,E) 6 : Unload(A,M) 7 : Unload(B,M) 8 : load(A,M) 9 : load(B,M)

  31. In(A) In(B) At(R,M) At(R,E) At(A,E) At(B,E) 1: Load(A) 1: Load(A) or or 0  0  2 : Load(B) 2 : Load(B) or or 3 : Fly(R) 3 : Fly(R) Refining Disjunctive plans (4) Enforce (partial) 2-consistency Proposition list maintains interactions between pairs of conditions + Polynomial refinement time higher pruning power + Better balance between refinement cost and solution extraction cost 1: Load(A,E) or 2 : Load(B,E) or 3 : Fly(R) or 4 : Unload(A,E) or 5 : Unload(B,E) 6 : Unload(A,M) Can do higher level consistency, but it is rarely worth the cost 7 : Unload(B,M) 8 : load(A,M) 9 : load(B,M)

More Related