Efficient Automated Planning with New Formulations

1 / 58

# Efficient Automated Planning with New Formulations - PowerPoint PPT Presentation

Efficient Automated Planning with New Formulations. Ruoyun Huang Advisors : Dr. Yixin Chen and Dr. Weixiong Zhang Washington University in St. Louis. Efficient Automated Planning with New Formulations. Ruoyun Huang Advisors : Dr. Yixin Chen and Dr. Weixiong Zhang

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## PowerPoint Slideshow about 'Efficient Automated Planning with New Formulations' - lanza

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

### Efficient Automated Planning with New Formulations

Ruoyun Huang

Advisors: Dr. Yixin Chen and Dr. Weixiong Zhang

Washington University in St. Louis

• Efficient Automated Planning with New Formulations

Ruoyun Huang

Advisors: Dr. Yixin Chen and Dr. Weixiong Zhang

Washington University in St. Louis

Committee Members

Dr. Christopher D. Gill

Dr. Norman I. Katz

Dr. Yinjie Tang

Dr. KilianWeinburger

Classical Planning (STRIPS)

loc1

loc1

loc2

loc2

Init State:

(AT pkg loc1);

(AT truck loc2);

?

Applicable actions:

MOVE;

Goal: (AT pkg loc2)

• One solution with time step N=4:
• Time Step 1: MOVE truck loc2 loc1
• Time Step 2: LOAD pkg truck loc1
• Time Step 3: MOVE truck loc1 loc2
• Time Step 4: UNLOAD pkg truck loc2

Each action is consisted of a set of preconditions and a set of effects

3

Applications
• NASA Mars rover
• High speed printer in PARC
• Hubble space telescope[Zweben94]
• Biological network[Bryce10]
• Natural language processing[Geib07]

Research in “Automated” planning provides a basis for any domain specific planning applications

4

Key Challenge

state

action

Branching factor: O(2A)

Number of states: O(2AN)

… … …

A: Number of actions

N: Length of plan

… … …

… … …

• Expressiveness
• Complicated real world scenarios
• High complexity

5

Summayof Dissertation

Motivation:

Formulation/Representation is vital to problem solving

Agenda
• The SAS+ formulation
• SAS+ on planning algorithms
• Abstraction search
• SAS+ formulation as Satisfiability(SASE)
• Why SASE works?
• Temporally expressive planning
• A planning as SAT approach
• Apply SASE to temporal planning
• Further handle action costs
SAS+ Representation[Backstrom:96]

loc1

loc2

Strips

SAS+

Transitions

pkg:loc1truck

pkg:truckloc1

pkg:truckloc2

pkg:loc2truck

AT pkg loc1

V(pkg)

in truck

AT pkg loc2

at loc1

at loc2

IN pkg truck

AT truck loc1

V(truck)

DTG(Domain Transition Graph)

at loc1

at loc2

AT truck loc2

Transition: Change between values in a multi-valued variable

9

SAS+ is more compact

fact…

fact…

fact…

fact…

fact…

fact…

fact…

fact…

fact…

fact…

fact…

fact…

fact…

2F

10F/10

F: number of STRIPS facts

Assume avg. facts/value per DTG is 10

SAS+ also captures more hidden problem structures

10

Causal Dependency and Causal Graph

loc1

loc2

Pre: (AT truck loc2)

pkg’s DTG

pkg’s DTG

in truck

truck’s DTG

truck’s DTG

at loc1

at loc2

at loc1

at loc2

Causal Graph

Example of Hierarchy

To build a house

1.a Drive to the city call

1.b Fill out an application

1. Drive to the city hall

2. Fill out an application

3. Drive to the bank

5. Drive somewhere

6. Look for a builder

7. Sign a Contract

8. Build foundation

9. Build frame

10. Build roof

11. Build wall

… …

1. Get a permit

2. Get enough money

3. Construction

4. Interior

… …

2.a Drive to bank

2.b Fill out an application

3.a Find a builder

3.b Sign a contract

3.c Build foundation

3.d Build frame

3.e Build roof

3.f build wall

… …

Abstraction State Space

loc1

loc2

Abstraction

package’s DTG

truck’s DTG

full state space

in truck

1.a MOVE truck loc2 loc1

at loc1

at loc2

2.a MOVE truck loc1 loc2

Abstraction State Space Search (Iteration #1)

Abstraction State Space

DTG

DTG

Goal1

Goal2

DTG

DTG

DTG

DTG

DTG

DTG

DTG

DTG

Finds a plan quickly

Abstraction State Space Search (Iteration #2)

Abstraction State Space

G2

G1

DTG

DTG

DTG

DTG

DTG

DTG

DTG

DTG

DTG

DTG

Keep running to improve the plan quality

Agenda
• SAS+ on planning algorithms
• Abstraction search
• STRIPS based planning as SAT
• An encoding completely based on SAS+ (SASE)
• Recent work
• Why SASE works?
• Temporally expressive planning
• A planning as SAT approach
• Recent work:
• Apply SASE to temporal planning
• Further handle action costs
Agenda of SAT & SAS+
• Long distance mutual exclusion
• Limitations
• SAS+ Planning as Satisfiability, a new encoding
• Motivation
• How it works
• Experimental Results
• Why it works
Mutual Exclusion (Mutex)

(in pkgtruck)

(in pkgtruck)

(at truck loc1)

(at truck loc1)

(at truck loc2)

(at truck loc2)

(at pkg loc1)

(at pkg loc1)

(at pkg loc2)

(at pkg loc2)

Move-truck-loc1-loc2

Move-truck-loc2-loc1

A key reason for the efficiency of SAT-based planning

• Two facts/actions cannot be true at the same time
• Fact mutual exclusion
• Action mutual exclusion

18

Mutual Exclusion

Invalid

Time #1

Time #2

Time #3

#4

#5

#6

#7

#8

#9

#10

(at truck loc1)

(at truck loc2)

Long Distance Mutual Exclusion(Londex)

DTG of truck

(at truck loc1)

(at truck loc2)

(at truck loc3)

(at truck loc4)

Invalid

#4

Time#1

#5

Time #2

#6

Time #3

#9

#10

#8

#7

(at truck loc1)

(at truck loc4)

Several limitations

SAS+ Based Encoding (SASE)
• Limitations of long distance mutual exclusion
• SAS+ not fully exploited
• Not memory efficient
• We want to make better use of SAS+
Actions versus Transitions

Representing Actions:

Hierarchical relationships between actions and transitions

22

Motivation: Clumsy Actions

:action move

:parameters (?a - airplane ?t - planetype ?d1 ?d2- direction ?s1 ?s2 - segment)

:effects

(occupied ?s2)

(blocked ?s2 ?a)

(not (occupied ?s1))

(when (not (is-blocked ?s1 ?t ?s2 ?d2))

(not (blocked ?s1 ?a)))

(when (not (= ?d1 ?d2))

(not (facing ?a ?d1)))

(not (at-segment ?a ?s1))

(forall (?s - segment)

(when (is-blocked ?s ?t ?s2 ?d2)

(blocked ?s ?a)))

(at-segment ?a ?s2)

(when (not (= ?d1 ?d2))

(facing ?a ?d2)) )

… … More .. ….

:precondition

(has-type ?a ?t)

(is-moving ?a)

(not (= ?s1 ?s2))

(facing ?a ?d1)

(can-move ?s1 ?s2 ?d1)

(move-dir ?s1 ?s2 ?d2)

(at-segment ?a ?s1)

(not (exists (?a1 - airplane)

(and (not (= ?a1 ?a))

(blocked ?s2 ?a1))))

(forall (?s - segment)

(imply (and

(is-blocked ?s ?t ?s2 ?d2)

(not (= ?s ?s1)))

(not (occupied ?s)) ))

This action can be break into 7 transitions

• AIRPORT benchmark domain[Hoffmann:04]
• 10 actions (in first order logic)
A Typical Strips Encoding

Compile from planning graph

Time step 3,4,5,…

Time step 1

Time step 2

(at pkg loc1)

(at pkg loc1)

(at pkg loc1)

(in pkg truck)

(at truck loc2)

(at truck loc2)

(at truck loc2)

move

move

(at truck loc1)

(at truck loc1)

move

Planning Graph

Nodes (facts, actions)

Edges

SAT Instance:

Variables

Clauses

Results in a whole chunk of encoding

25

New Encoding, An Overview

Planning graph

SAT Instance:

Facts and actions

Actions&

Facts

t = 1

t =2

t =3, 4, …

Strips Based Encoding

Transitions

SAT Instance (Part 1):

transitions

SAT Instance (Part 2):

matching actions and transitions (multiple independent ones)

Actions

t = 1

t =2

t =3

Transition Based Encoding

26

Constraints

Time step 1

Time step 2

Time step 3,4,5,…

truck:loc2

truck:loc2

truck:loc2

truck:loc1

truck:loc1

truck:loc1

pkg: loc1

pkg: loc1

pkg: loc1

pkg: truck

pkg: truck

pkg: truck

pkg: loc2

pkg: loc2

pkg: loc2

Find matchings

… …

set of actions

set of actions

• Progression of transitions over time steps (blue one implies green ones)
• Initial state and goal (Bold ones)
• Matching actions and transitions
• Action mutual exclusions and transition mutual exclusions
Worst Case State Space Size

State Space:

O((2A)N)

Planning graph

Actions

A Number of actions

N Number of Time Step

Strips Based Encoding

t = 1

t =2

t =3, 4, …

Transitions

State Space (Part 1):

O((2T)N)

T Number of transitions

State Space (Part 2):

O(NK)

K

K

K

Actions

Total: O((2T)NNK)

t = 1

t =2

t =3

Transition Based Encoding

Note: T < A

30

Why SASE works?

32

• Worst case analysis has limitations
• Real SAT solvers do not enumerate
• Real SAT solvers may not favor transition variables
• We want better understanding
• Reason: significance of transition variables
• Why more significant?
• How much?
• Correlates to the speedup?
SAT Algorithms

at certain point

V10

V1

-

V2

V5

V7

Solution Found

… …

… …

No Solution

… …

No solution

Which variable to pick matters

33

• The DPLL procedure
• Depth First Search and Inference
• Used by almost all SAT solvers
VSIDS Heuristic

we probably want to augment the deviation of the h values of all variables

34

• VSIDS Heuristic
• Widely used, very effective
• h(), number of times a variable appears in all clauses
• EMA of all variables’ h(), clause database changes over time

Example: (a∨b∨c) ; (a∨d); (a∨b∨d)

h(a) = 3; h(b) = 2; h(c)=1

• What kind of instances make VSIDS effective and what kind of instances not?
• Consider an extreme case: random SAT
h() on SASE Instances

Transitions

K

K

K

t = 1

t =2

t =3

Actions

36

h() on IPC Domains

Action variables even do not exist in top percentage of variables

37

Significance of Transition Variables

38

• If we ranking all variables by h
• Transition variables dominates the head of the list
• More likely to be picked as decision variables earlier (statistically)
• But VSIDS heuristic dynamically updates …
• We need evidence that reflects the dynamic updating
Branching Frequency

V10

V1

-

V2

V7

V5

For every 1000 consecutive decision variables

… …

… …

… …

39

A measurement of how often transition variables (or action variables) are chosen as decision variable

Branching Frequency

Zenotravel#15

Speedup = 4.21

Openstack #5

Speedup = 7.88

40

Transition variables have higher branching frequency in most instances

Branching Frequency

Storage #12

Speedup = 0.83

Sokoban#8

Speedup = 0.62

41

SASE could be less efficient in a few domains

Transition Index

The ratio of transition

variables in the top ranked

p percent variables

Normalized by the ratio

of transition variables

among all variables

42

Transitions variables are more likely to be decision variables

The more branching on transition variables, the higher speedup

Transition Index : An indicator of the significance

Transition Index versus Speedup

Transition index of 1%

Transition index of 10%

43

Transition Index versus Speedup

Transition index of 50%

Transition index of 90%

44

Conclusions

45

Transition variables occurs more frequently

Transition index correlates with the speedup that SASE can achieve

Outline

Automated Planning

Classical Planning

Temporally Expressive Planning

STRIPS

SAS+

STRIPS

SAS+

Search

SAT

Search

SAT

Search

SAT

SAT

4

Temporal Planning

e.g. got a blackout:

Light a Match

Repair a fuse

find a manual

Time

47

Actions have durations

Temporally Expressive[W. Cushing et al. 2007]

condition

condition

requires condition

(has light)

Light a match

• A planning algorithm is temporally expressive if it can handle required concurrency, otherwise it is temporally simple

deletes fact (has light)

Time

• Temporally Expressive (Required Concurrency)
• Two actions need to be executed in parallel, in any valid plan.

48

• Required Concurrency is important in the real world
• PDDL2.1 supports temporally expressiveness
• e.g. Firework, Missile launch
• Most state of the art temporal planners are not able to handle temporally expressive
• Action cost is in addition an important feature
• No planner can handle both temporal expressiveness and action costs

49

Overview of Our Work

Temporally Expressive planning

PDDL2.1 Spec

STRIPS

A STRIPS based encoding [Huang09]

SAS+

Extend to support action costs

Cost Sensitive Temporally Expressive Planning

SASE for temporal planning

A MinCost SAT Formulation

MaxSAT solvers

A Branch & Bound Algorithm

50

Durative Actions as SAS+

Light a match

Light a match

deletes fact (has light)

Time

Time

51

Experimental Results

PET and SET are our SAT-based approach, with different encoding schemes;

Crikey2, Crikey3, TFD, LPG-c are several heuristic search based planners

53

Cost Sensitive Temporally Expressive Planning

MinCost

A Branch & Bound Algorithm

MaxSAT solvers

54

Cost Sensitive Temporally Expressive
• Extend the encoding into MinCost SAT problem
• MinCost: variables have costs, to be minimized
• Convert MinCost instances into MaxSAT
• MaxSAT is widely accept
• A planning specialized branch and bound algorithm
• Use relaxed planning graph to estimate a heuristic function

55

Summary
• More efficient algorithm
• Search
• Planning as SAT
• More expressive features
• Temporal expressiveness
• Action costs

57

Significance of Research
• Other planning algorithms
• Other planning models
• probabilistic, uncertainty, etc
• Beyond planning research
• Bounded model checking[Clarke01]

58

Future Directions
• Integrate existing techniques into SASE
• More understanding on what makes an SAT instance easy or hard
• Backdoor set / Backbone set
• Goal: improve general SAT solving
• Preferences, numerics, etc.
• SAS+ for other planning techniques

59

Publications

R. Huang, Y. Chen, and W. Zhang, “SAS+ planning as Satisfiability”. Journal of Artificial Intelligence Research, invited and submitted

R. Huang, Y. Chen, and W. Zhang, “A Novel Transition Based Encoding Scheme for Planning as Satisfiability”. AAAI’10

R. Huang, Y. Chen, and W. Zhang, “An Optimal Temporally Expressive Planner: Initial Results and Application to P2P Network Optimization”. ICAPS’09

Y. Chen, R. Huang, Z. Xing and W. Zhang, “Long-distance Mutual Exclusion for Planning”. Artificial Intelligence Journal, 2009

Y. Chen, R. Huang, and W. Zhang, “Fast Planning by Search in Domain Transition Graphs”. AAAI’08

C. Hsu, B. Wah, R. Huang, and Y. Chen, “ Constraint Partitioning for Solving Planning Problems with Trajectory Constraints and Goal Preferences”. IJCAI’07

60