Integer Programming Approaches for Automated Planning - PowerPoint PPT Presentation

1 / 45

Integer Programming Approaches for Automated Planning. Menkes van den Briel Department of Industrial Engineering Arizona State University menkes@asu.edu http://www.public.asu.edu/~dbvan1/. What is automated planning?. Ordering problem

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

Integer Programming Approaches for Automated Planning

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Integer Programming Approaches for Automated Planning

Menkes van den BrielDepartment of Industrial EngineeringArizona State Universitymenkes@asu.eduhttp://www.public.asu.edu/~dbvan1/

What is automated planning?

Ordering problem

Scheduling is the problem of deciding when to execute a set of actions

NP-complete

Selection and ordering problem

Planning is deciding both what actions need to be done and when to execute them

PSPACE-complete

Scheduling

Planning

What is automated planning?

• Creating a computer program to produce a plan, a sequence of actions that will transform the world from some given initial state to a desired goal state

1

2

1

2

Initial states0 S

Goalg S

PlanP = a1, …, an

Action

Actions are state transformation functions

What is automated planning?

• Creating a computer program to produce a plan, a sequence of actions that will transform the world from some given initial state to a desired goal state

Initial states0 S

Goalg S

PlanP = a1, …, an

Action

si

sj

Actions are state transformation functions

Planning applications

• Autonomous vehicles

• Mars rovers

• Underwater robotics

• Remote agent experiment

• Games

• Bridge Baron

• General game playing

• Others

• Manufacturing process planning

• Composition of web services

• Cyber Security

Planning by integer programming

Operations research (OR)

Scheduling problems typically involve solving hard optimization problems

Integer programming (IP), branch-and-bound

Artificial intelligence (AI)

Planning problems typically involve solving hard feasibility problems

Constraint satisfaction, satisfiability (SAT), A* search

Scheduling

Planning

Planning by integer programming

• Very little focus on integer programming approaches for planning

• [Bylander, 1997]

• [Bockmayr and Dimopoulos, 1998, 1999]

• [Kautz and Walser, 1999]

• [Vossen et al., 1999]

• [Dimopoulos, 2001]

• [Dimopoulos and Gerevini, 2002]

Why this lack of interest?

• IP-based approaches simply don’t work

• “Lplan [a linear programming-based heuristic for optimal planning] was often slower than the other algorithms primarily due to the time to evaluate the linear programming heuristic”[Bylander, 1997]

• SAT-based approaches are much faster

• SAT-based planners have successfully participated in IPC1, IPC2, IPC4, and IPC5

• Traditionally there has been little focus on plan quality

• Planning is PSPACE-complete, so finding a feasible plan is already hard enough

Counter arguments

• IP-based approaches do work

• Optiplan, first IP-based planner to take part in the IPC series

• Ranked 2nd in four out of seven domains in IPC4 in the optimal track for propositional domains

• IP-based approaches can compete with SAT-based approaches

• Represent planning as a set of interdependent network flow problems

• Generalize the notion of action parallelism

• Shift in focus towards optimal planning

• Applied formulations to partial satisfaction planning problems

• Developed a novel framework for optimal planning

• Utilized LP relaxations in deriving quality sensitive heuristics

Contributions

• IP-based approaches do work

• IP-based approaches can compete with SAT-based approaches

• Shift in focus towards optimal planning

• [Van den Briel, and Kambhampati. Journal of Artificial Intelligence Research, 2005]

• [Van den Briel, Vossen, and Kambhampati. ICAPS, 2005]

• [Van den Briel, Vossen, and Kambhampati. Journal of Artificial Intelligence Research, 2008]

• [Van den Briel, et al. AAAI, 2004]

• [Do, Benton, van den Briel, and Kambhampati. IJCAI, 2007]

• [J. Benton, van den Briel, and Kambhampati. ICAPS, 2007]

• [Van den Briel, Benton, Kambhampati, and Vossen. CP, 2007]

1. IP approaches do work

• Optiplan

• IP-based planner that extends the state change formulation by [Vossen et al., 1999]

[van den Briel, and Kambhampati, 2005]

Summary of results

• International planning competition (IPC)

• Bi-annual event

• Provides data sets (domains) that are used as benchmarks

• IPC4

• 7 competition domains

• 7 participating planners in the “optimal” track

• Domains

• Pipesworld

• Control the flow of oil derivatives through a pipeline network, obeying various constraints such as product compatibility and tankage restrictions

• Satellite

• Collect image data with a number of satellites

• Philosophers, Optical telegraph

• Involves finding deadlocks in communication protocols

2. IP versus SAT approaches

• Represent planning as a set of interdependent network flow problems

• One network flow problem for each state variable in the planning domain

• Nodes correspond to the values of the state variables, arcs correspond to the value transitions

• Generalize the notion of action parallelism

• Reduces the plan length of the solution plan (and thus the size of the formulation)

Logistics example

1

2

P

T

Truck

1

Drive(1,2)

Drive(2,1)

2

Package

1

2

T

States are described by state variables

Logistics example

1

2

Prevail

Truck

1

Drive(1,2)

Drive(2,1)

2

Package

1

Effect

2

T

Actions are state transformation functions

One state change (1SC)

• Network representation

• Logistics example

Prevail

f

f

f

Effect

g

g

g

h

h

h

Plan step

Truck

1

1

2

2

Planning involves considering

plans of increasing length

Package

1

1

2

2

t

t

t = 1

One state change (1SC)

• Network representation

• Logistics example

Prevail

f

f

f

Effect

g

g

g

h

h

h

Drive(1,2)

Truck

1

1

1

1

2

2

2

2

-

Package

1

1

1

1

2

2

2

2

t

t

t

t

t = 1

t = 2

t = 3

1SC formulation

• Constraints

• State changes (network flow), for allc  CgCycf,g,t= 1{f  I}for f  DchCycg,h,t+1= fCycg,h,t for f  Dc , 1  t < T fCycf,g,T= 1for g  G

• Effect implications, for allc  C, 1  t  TaA:(f,g)SC(a)xa,t = ycf,g,tfor f, g  Dc, f  g xa,t  ycf,f,tfor a  A, f PR(a)

Summary of results

• Experimental setup

• Domains from IPC2, IPC3

• Comparing 1SC formulation versus SATPLAN04 (winner of the “optimal“ track IPC4)

• 2.67GHz CPU with 1.0GB memory

• Domains

• Logistics, Driverlog

• Involves driving trucks (and flying airplanes) around to deliver packages between locations

• Blocksworld

• Stacking and unstacking towers of blocks

• Zenotravel

• Transporting people around in planes, using different modes of movement: fast and slow

2. IP versus SAT approaches

• Represent planning as a set of interdependent network flow problems

• One network flow problem for each state variable in the planning domain

• Nodes correspond to the values of the state variables, arcs correspond to the value transitions

• Generalize the notion of action parallelism

• Reduces the plan length of the solution plan (and thus the size of the formulation)

Generalized one state change (G1SC)

• Network representation

• Example

Prevail

f

f

f

Effect

g

g

g

h

h

h

Truck

1

1

1

2

2

2

Package

1

1

1

2

2

2

t

t

t

t = 1

t = 2

Implied precedences (G1SC)

• Example

A4

A1

A3

A1,A2

A3

A4

A2

Implied precendence graph

Implied precedences (G1SC)

• Example

• Ordering (cycle elimination) constraints ensure a feasible ordering of the actions

A4

A1

A3

A1,A2

A3

A4

A2

Implied precendence graph

A4

A1

xA1,t + xA3,t + xA4,t  2

G1SC formulation

• Constraints

• State changes (network flow), for allc  CgCycf,g,t= 1{f  I}for f  DchCycg,h,t+1= fCycg,h,t for f  Dc, 1  t  T fCycf,g,T= 1for g  G

• Effect implications, for allc  C, 1 t  T

aA:(f,f)SC(a)xa,t= ycf,g,tfor f, g  Dc, f  g,

xa,t  ycf,f,t + gDc:f≠g (ycg,f,t + ycf,g,t)for a  A, f PR(a)

• Ordering (Cycle elimination) constraints

 aV()xa,t  |V()| – 1for all cycles G, 1  t  T

START

STOP

Initialize LP

no

Nodes found?

yes

LP solver

Feasible?

no

Fathom

Node selection

yes

Z_lp < Z*?

no

yes

Cut generation

Cuts found?

yes

no

Integer?

no

Branching

yes

State change path (PathSC)

• Network representation

• Example

Prevail

f

f

f

Effect

g

g

g

h

h

h

Truck

1

1

2

2

Package

1

1

2

2

t

t

t = 1

Summary of results

[van den Briel, Vossen, and Kambhampati, 2005, 2008]

3. Shift towards optimal planning

• Applied formulations to partial satisfaction planning problems

• Developed a novel framework for optimal planning

• Utilized LP relaxations in deriving quality sensitive heuristic search approaches

Partial satisfaction planning

• PLAN LENGTH is PSPACE-complete

• [Bylander, 1994]

• PSP UTILITY COST is PSPACE-complete

• [Van den Briel, et al., 2004]

Total Satisfaction Problems

PSP UTILITY COST

PSP NET BENEFIT

PLAN COST

PSP UTILITY

PSP GOAL LENGTH

Partial SatisfactionProblems

PLAN LENGTH

PSP GOAL

PLAN EXISTENCE

Framework for optimal planning

• For step-based IP formulations optimality is restricted to the length of the plan

Plan step

Drive(1,2)

Truck

1

1

1

1

2

2

2

2

-

Package

1

1

1

1

2

2

2

2

t

t

t

t

t = 1

t = 2

t = 3

1

2

P

T

Truck

1

Drive(1,2)

Drive(2,1)

2

Package

1

2

T

Action selection formulation

• Variables

• xa Z+, for a  A; xa is equal to the number of times action a is executed

• yv(c,a) Z+, for v  V, a  A, a  –(c); yv(c,a) is equal to the number of times transition v(c,a) is executed

• Objective function

• MIN aAcaxa

• Constraints

• av+(e)yv(c,a) – a v–(e)yv(c,a)

• av+(e)yv(c,a) = xa

No time indicesNo upper bounds

1if c c0,v, c g–1if c= c0,v, c g0otherwise

Concurrent automata

• Given a set of state variables V = {v1, …, vn}

• For each v V we define a deterministic automaton Gv = (Dv, Av, v, v, c0,v, gv)

• Dv is a finite set of states corresponding to the domain of state variable v

• Av is a finite set of actions associated with the transitions in Gv

• v : Dv  A  Dv is the transition function

• v : Dv  2A is the active action function

• c0,v  S is the initial state of state variable v

• gv  S is a set of goal states of state variable v

Parallel composition

• The parallel composition of the two automata G1 and G2 is the automaton G1||G2 := (D1D2, A1A2, 1||2, 1||2, (c0,1, c0,2), g1g2)

• 1||2((c1,c2),a) :=

• 1||2(c1,c2) := [1(c1)2(c2)]  [1(c1)\A2][2(c2)\A1]

(1(c1,a), 2(c2,a)if a  1(c1)2(c2)(1(c1,a), c2)if a  1(c1)\A2(c1,2(c2,a))if a  2(c2)\A1undefinedotherwise

1

2

P

T

Truck

1

Drive(1,2)

Drive(2,1)

2

Package

1

2

T

1

2

P

T

Truck|| Package

2,1

Drive(2,1)

Drive(1,2)

1,2

1,1

Drive(2,1)

Drive(1,2)

2,2

1,T

Drive(2,1)

Drive(1,2)

2,T

Summary of results

Highlighted values equal optimal solution

Utilize LP in heuristic search

BBOP-LP planner

[Benton, van den Briel, and Kambhampati, 2007]

Summary

• IP-based approaches do work

• Optiplan, first IP-based planner to take part in the IPC series

• Ranked 2nd in four out of seven domains in IPC4 in the optimal track for propositional domains

• IP-based approaches can compete with SAT-based approaches

• Represent planning as a set of interdependent network flow problems

• Generalize the notion of action parallelism

• Shift in focus towards optimal planning

• Applied formulations to partial satisfaction planning problems

• Developed a novel framework for optimal planning

• Utilized LP relaxations in deriving quality sensitive heuristics

Publications status

• Journal

• [M.H.L. van den Briel, and S. Kambhampati. Optiplan: Unifying IP-based and graph-based planning. Journal of Artificial Intelligence Research, 24:623-635, 2005]

• [M.H.L van den Briel, T. Vossen, and S. Kambhampati. Loosely coupled formulation for automated planning: An integer programming perspective. Journal of Artificial Intelligence Research, 31:217-257, 2008]

• [(In progress) M.H.L van den Briel, T. Vossen, S. Kambhampati and J. Fowler. Optimal automated planning]

• Conference

• [M.H.L. van den Briel, R. Sanchez, M.B. Do, and S. Kambhampati. Effective approaches for partial satisfaction (oversubscription) planning. In Proceedings of AAAI, pages 562-569, 2004]

• [M.H.L. van den Briel, T. Vossen, and S. Kambhampati. Reviving integer programming approaches for AI planning: A branch-and-cut framework. In Proceedings of ICAPS, pages 161-170, 2005]

• [M.B. Do, J. Benton, M.H.L. van den Briel, and S. Kambhampati. Planning with goal utility dependencies. In Proceedings of IJCAI, pages 1872-1878, 2007]

• [J. Benton, M.H.L. van den Briel, and S. Kambhampati. A hybrid linear programming and relaxed plan heuristic for partial satisfaction planning problems. In Proceedings of ICAPS, pages 24-41, 2007]

• [M.H.L. van den Briel, J. Benton, S. Kambhampati, and T. Vossen. An LP-based heuristic for optimal planning. In Proceedings of CP, pages 651-665, 2007]

Cited by 6

Cited by 31

Cited by 15

Cited by 3

Cited by 4

Cited by 3

Publications status

• Workshop and posters

• [M.H.L. van den Briel, R. Sanchez, and S. Kambhampati. Over-Subscription in Planning: a Partial Satisfaction Problem. In Proceedings of ICAPS Workshop on Integrating Planning into Scheduling, 2005]

• [M.H.L. van den Briel,. Kambhampati, and T. Vossen. Planning with numerical state variables through mixed integer programming. In Proceedings of ICAPS Poster Session, pages 5-8, 2005]

• [M.H.L. van den Briel,. Kambhampati, and T. Vossen. Planning with preferences and trajectory constraints by integer programming. In Proceedings of ICAPS Workshop on Preferences and Soft Constraints in Planning, pages 19-22, 2006]

• [J. Benton, M.H.L. van den Briel,. Kambhampati. Finding admissible bounds for oversubscription planning problems. In Proceedings of ICAPS Workshop on Heuristics for Domain-Independent Planning: Progress, Ideas, Limitations, Challenges, 2007]

• [M.H.L. van den Briel,. Kambhampati, and T. Vossen. Fluent merging: A general technique to improve reachability heuristics and factored planning. In Proceedings of ICAPS Workshop on Heuristics for Domain-Independent Planning: Progress, Ideas, Limitations, Challenges, 2007]

Cited by 5

Cited by 1

Cited by 1