Integer programming approaches for automated planning
This presentation is the property of its rightful owner.
Sponsored Links
1 / 45

Integer Programming Approaches for Automated Planning PowerPoint PPT Presentation


  • 89 Views
  • Uploaded on
  • Presentation posted in: General

Integer Programming Approaches for Automated Planning. Menkes van den Briel Department of Industrial Engineering Arizona State University [email protected] http://www.public.asu.edu/~dbvan1/. What is automated planning?. Ordering problem

Download Presentation

Integer Programming Approaches for Automated Planning

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Integer programming approaches for automated planning

Integer Programming Approaches for Automated Planning

Menkes van den BrielDepartment of Industrial EngineeringArizona State [email protected]://www.public.asu.edu/~dbvan1/


What is automated planning

What is automated planning?

Ordering problem

Scheduling is the problem of deciding when to execute a set of actions

NP-complete

Selection and ordering problem

Planning is deciding both what actions need to be done and when to execute them

PSPACE-complete

Scheduling

Planning


What is automated planning1

What is automated planning?

  • Creating a computer program to produce a plan, a sequence of actions that will transform the world from some given initial state to a desired goal state

1

2

1

2

Initial states0 S

Goalg S

PlanP = a1, …, an

Action

Actions are state transformation functions


What is automated planning2

What is automated planning?

  • Creating a computer program to produce a plan, a sequence of actions that will transform the world from some given initial state to a desired goal state

Initial states0 S

Goalg S

PlanP = a1, …, an

Action

si

sj

Actions are state transformation functions


Planning applications

Planning applications

  • Autonomous vehicles

    • Mars rovers

    • Underwater robotics

    • Remote agent experiment

  • Games

    • Bridge Baron

    • General game playing

  • Others

    • Manufacturing process planning

    • Composition of web services

    • Cyber Security


Planning by integer programming

Planning by integer programming

Operations research (OR)

Scheduling problems typically involve solving hard optimization problems

Integer programming (IP), branch-and-bound

Artificial intelligence (AI)

Planning problems typically involve solving hard feasibility problems

Constraint satisfaction, satisfiability (SAT), A* search

Scheduling

Planning


Planning by integer programming1

Planning by integer programming

  • Very little focus on integer programming approaches for planning

    • [Bylander, 1997]

    • [Bockmayr and Dimopoulos, 1998, 1999]

    • [Kautz and Walser, 1999]

    • [Vossen et al., 1999]

    • [Dimopoulos, 2001]

    • [Dimopoulos and Gerevini, 2002]


Why this lack of interest

Why this lack of interest?

  • IP-based approaches simply don’t work

    • “Lplan [a linear programming-based heuristic for optimal planning] was often slower than the other algorithms primarily due to the time to evaluate the linear programming heuristic”[Bylander, 1997]

  • SAT-based approaches are much faster

    • SAT-based planners have successfully participated in IPC1, IPC2, IPC4, and IPC5

  • Traditionally there has been little focus on plan quality

    • Planning is PSPACE-complete, so finding a feasible plan is already hard enough


Counter arguments

Counter arguments

  • IP-based approaches do work

    • Optiplan, first IP-based planner to take part in the IPC series

    • Ranked 2nd in four out of seven domains in IPC4 in the optimal track for propositional domains

  • IP-based approaches can compete with SAT-based approaches

    • Represent planning as a set of interdependent network flow problems

    • Generalize the notion of action parallelism

  • Shift in focus towards optimal planning

    • Applied formulations to partial satisfaction planning problems

    • Developed a novel framework for optimal planning

    • Utilized LP relaxations in deriving quality sensitive heuristics


Contributions

Contributions

  • IP-based approaches do work

  • IP-based approaches can compete with SAT-based approaches

  • Shift in focus towards optimal planning

  • [Van den Briel, and Kambhampati. Journal of Artificial Intelligence Research, 2005]

  • [Van den Briel, Vossen, and Kambhampati. ICAPS, 2005]

  • [Van den Briel, Vossen, and Kambhampati. Journal of Artificial Intelligence Research, 2008]

  • [Van den Briel, et al. AAAI, 2004]

  • [Do, Benton, van den Briel, and Kambhampati. IJCAI, 2007]

  • [J. Benton, van den Briel, and Kambhampati. ICAPS, 2007]

  • [Van den Briel, Benton, Kambhampati, and Vossen. CP, 2007]


1 ip approaches do work

1. IP approaches do work

  • Optiplan

    • IP-based planner that extends the state change formulation by [Vossen et al., 1999]

[van den Briel, and Kambhampati, 2005]


Summary of results

Summary of results

  • International planning competition (IPC)

    • Bi-annual event

    • Provides data sets (domains) that are used as benchmarks

  • IPC4

    • 7 competition domains

    • 7 participating planners in the “optimal” track

  • Domains

    • Pipesworld

      • Control the flow of oil derivatives through a pipeline network, obeying various constraints such as product compatibility and tankage restrictions

    • Satellite

      • Collect image data with a number of satellites

    • Philosophers, Optical telegraph

      • Involves finding deadlocks in communication protocols


Summary of results1

Summary of results


2 ip versus sat approaches

2. IP versus SAT approaches

  • Represent planning as a set of interdependent network flow problems

    • One network flow problem for each state variable in the planning domain

    • Nodes correspond to the values of the state variables, arcs correspond to the value transitions

  • Generalize the notion of action parallelism

    • Reduces the plan length of the solution plan (and thus the size of the formulation)


Logistics example

Logistics example

1

2

P

T

Truck

Load(P,T,1)Unload(P,T,1)

1

Drive(1,2)

Drive(2,1)

2

Load(P,T,1)Unload(P,T,1)

Package

1

Load(P,T,1)

unload(P,T,1)

2

Load(P,T,2)

unload(P,T,2)

T

States are described by state variables


Logistics example1

Logistics example

1

2

Prevail

Truck

Load(P,T,1)Unload(P,T,1)

1

Drive(1,2)

Drive(2,1)

2

Load(P,T,1)Unload(P,T,1)

Package

1

Load(P,T,1)

unload(P,T,1)

Effect

2

Load(P,T,2)

unload(P,T,2)

T

Actions are state transformation functions


One state change 1sc

One state change (1SC)

  • Network representation

  • Logistics example

Prevail

f

f

f

Effect

g

g

g

h

h

h

Plan step

Truck

1

1

2

2

Planning involves considering

plans of increasing length

Package

1

1

2

2

t

t

t = 1


One state change 1sc1

One state change (1SC)

  • Network representation

  • Logistics example

Prevail

f

f

f

Effect

g

g

g

h

h

h

Load(P,T,1)

Drive(1,2)

Unload(P,T,2)

Truck

1

1

1

1

2

2

2

2

Load(P,T,1)

-

Unload(P,T,2)

Package

1

1

1

1

2

2

2

2

t

t

t

t

t = 1

t = 2

t = 3


1sc formulation

1SC formulation

  • Constraints

    • State changes (network flow), for allc  CgCycf,g,t= 1{f  I}for f  DchCycg,h,t+1= fCycg,h,t for f  Dc , 1  t < T fCycf,g,T= 1for g  G

    • Effect implications, for allc  C, 1  t  TaA:(f,g)SC(a)xa,t = ycf,g,tfor f, g  Dc, f  g xa,t  ycf,f,tfor a  A, f PR(a)


Summary of results2

Summary of results

  • Experimental setup

    • Domains from IPC2, IPC3

    • Comparing 1SC formulation versus SATPLAN04 (winner of the “optimal“ track IPC4)

    • 2.67GHz CPU with 1.0GB memory

  • Domains

    • Logistics, Driverlog

      • Involves driving trucks (and flying airplanes) around to deliver packages between locations

    • Blocksworld

      • Stacking and unstacking towers of blocks

    • Zenotravel

      • Transporting people around in planes, using different modes of movement: fast and slow


Summary of results3

Summary of results


2 ip versus sat approaches1

2. IP versus SAT approaches

  • Represent planning as a set of interdependent network flow problems

    • One network flow problem for each state variable in the planning domain

    • Nodes correspond to the values of the state variables, arcs correspond to the value transitions

  • Generalize the notion of action parallelism

    • Reduces the plan length of the solution plan (and thus the size of the formulation)


Generalized one state change g1sc

Generalized one state change (G1SC)

  • Network representation

  • Example

Prevail

f

f

f

Effect

g

g

g

h

h

h

Load(P,T,1)Drive(1,2)

Unload(P,T,2)

Truck

1

1

1

2

2

2

Load(P,T,1)

Unload(P,T,2)

Package

1

1

1

2

2

2

t

t

t

t = 1

t = 2


Implied precedences g1sc

Implied precedences (G1SC)

  • Example

A4

A1

A3

A1,A2

A3

A4

A2

Implied precendence graph


Implied precedences g1sc1

Implied precedences (G1SC)

  • Example

  • Ordering (cycle elimination) constraints ensure a feasible ordering of the actions

A4

A1

A3

A1,A2

A3

A4

A2

Implied precendence graph

A4

A1

xA1,t + xA3,t + xA4,t  2


G1sc formulation

G1SC formulation

  • Constraints

    • State changes (network flow), for allc  CgCycf,g,t= 1{f  I}for f  DchCycg,h,t+1= fCycg,h,t for f  Dc, 1  t  T fCycf,g,T= 1for g  G

    • Effect implications, for allc  C, 1 t  T

      aA:(f,f)SC(a)xa,t= ycf,g,tfor f, g  Dc, f  g,

      xa,t  ycf,f,t + gDc:f≠g (ycg,f,t + ycf,g,t)for a  A, f PR(a)

    • Ordering (Cycle elimination) constraints

       aV()xa,t  |V()| – 1for all cycles G, 1  t  T


Branch and cut

Branch-and-cut

START

STOP

Initialize LP

no

Nodes found?

yes

LP solver

Feasible?

no

Fathom

Node selection

yes

Z_lp < Z*?

no

yes

Cut generation

Cuts found?

yes

no

Integer?

no

Branching

yes


State change path pathsc

State change path (PathSC)

  • Network representation

  • Example

Prevail

f

f

f

Effect

g

g

g

h

h

h

Load(P,T,1)Drive(1,2)Unload(P,T,2)

Truck

1

1

2

2

load(P,T,1)unload(P,T,2)

Package

1

1

2

2

t

t

t = 1


Summary of results4

Summary of results


Summary of results5

Summary of results

[van den Briel, Vossen, and Kambhampati, 2005, 2008]


3 shift towards optimal planning

3. Shift towards optimal planning

  • Applied formulations to partial satisfaction planning problems

  • Developed a novel framework for optimal planning

  • Utilized LP relaxations in deriving quality sensitive heuristic search approaches


Partial satisfaction planning

Partial satisfaction planning

  • PLAN LENGTH is PSPACE-complete

    • [Bylander, 1994]

  • PSP UTILITY COST is PSPACE-complete

    • [Van den Briel, et al., 2004]

Total Satisfaction Problems

PSP UTILITY COST

PSP NET BENEFIT

PLAN COST

PSP UTILITY

PSP GOAL LENGTH

Partial SatisfactionProblems

PLAN LENGTH

PSP GOAL

PLAN EXISTENCE


Framework for optimal planning

Framework for optimal planning

  • For step-based IP formulations optimality is restricted to the length of the plan

Plan step

Load(P,T,1)

Drive(1,2)

Unload(P,T,2)

Truck

1

1

1

1

2

2

2

2

Load(P,T,1)

-

Unload(P,T,2)

Package

1

1

1

1

2

2

2

2

t

t

t

t

t = 1

t = 2

t = 3


Framework for optimal planning1

Framework for optimal planning

1

2

P

T

Truck

Load(P,T,1)Unload(P,T,1)

1

Drive(1,2)

Drive(2,1)

2

Load(P,T,1)Unload(P,T,1)

Package

1

Load(P,T,1)

unload(P,T,1)

2

Load(P,T,2)

unload(P,T,2)

T


Action selection formulation

Action selection formulation

  • Variables

    • xa Z+, for a  A; xa is equal to the number of times action a is executed

    • yv(c,a) Z+, for v  V, a  A, a  –(c); yv(c,a) is equal to the number of times transition v(c,a) is executed

  • Objective function

    • MIN aAcaxa

  • Constraints

    • av+(e)yv(c,a) – a v–(e)yv(c,a)

    • av+(e)yv(c,a) = xa

No time indicesNo upper bounds

1if c c0,v, c g–1if c= c0,v, c g0otherwise


Concurrent automata

Concurrent automata

  • Given a set of state variables V = {v1, …, vn}

  • For each v V we define a deterministic automaton Gv = (Dv, Av, v, v, c0,v, gv)

    • Dv is a finite set of states corresponding to the domain of state variable v

    • Av is a finite set of actions associated with the transitions in Gv

    • v : Dv  A  Dv is the transition function

    • v : Dv  2A is the active action function

    • c0,v  S is the initial state of state variable v

    • gv  S is a set of goal states of state variable v


Parallel composition

Parallel composition

  • The parallel composition of the two automata G1 and G2 is the automaton G1||G2 := (D1D2, A1A2, 1||2, 1||2, (c0,1, c0,2), g1g2)

    • 1||2((c1,c2),a) :=

    • 1||2(c1,c2) := [1(c1)2(c2)]  [1(c1)\A2][2(c2)\A1]

(1(c1,a), 2(c2,a)if a  1(c1)2(c2)(1(c1,a), c2)if a  1(c1)\A2(c1,2(c2,a))if a  2(c2)\A1undefinedotherwise


Logistics example2

Logistics example

1

2

P

T

Truck

Load(P,T,1)Unload(P,T,1)

1

Drive(1,2)

Drive(2,1)

2

Load(P,T,1)Unload(P,T,1)

Package

1

Load(P,T,1)

unload(P,T,1)

2

Load(P,T,2)

unload(P,T,2)

T


Simple logistics example

Simple logistics example

1

2

P

T

Truck|| Package

2,1

Drive(2,1)

Drive(1,2)

1,2

1,1

Drive(2,1)

Drive(1,2)

Unload(P, T,1)

Load(P, T,1)

2,2

1,T

Unload(P, T,2)

Drive(2,1)

Load(P, T,2)

Drive(1,2)

2,T


Summary of results6

Summary of results

Highlighted values equal optimal solution


Summary of results7

Summary of results


Utilize lp in heuristic search

Utilize LP in heuristic search

BBOP-LP planner

[Benton, van den Briel, and Kambhampati, 2007]


Summary

Summary

  • IP-based approaches do work

    • Optiplan, first IP-based planner to take part in the IPC series

    • Ranked 2nd in four out of seven domains in IPC4 in the optimal track for propositional domains

  • IP-based approaches can compete with SAT-based approaches

    • Represent planning as a set of interdependent network flow problems

    • Generalize the notion of action parallelism

  • Shift in focus towards optimal planning

    • Applied formulations to partial satisfaction planning problems

    • Developed a novel framework for optimal planning

    • Utilized LP relaxations in deriving quality sensitive heuristics


Publications status

Publications status

  • Journal

    • [M.H.L. van den Briel, and S. Kambhampati. Optiplan: Unifying IP-based and graph-based planning. Journal of Artificial Intelligence Research, 24:623-635, 2005]

    • [M.H.L van den Briel, T. Vossen, and S. Kambhampati. Loosely coupled formulation for automated planning: An integer programming perspective. Journal of Artificial Intelligence Research, 31:217-257, 2008]

    • [(In progress) M.H.L van den Briel, T. Vossen, S. Kambhampati and J. Fowler. Optimal automated planning]

  • Conference

    • [M.H.L. van den Briel, R. Sanchez, M.B. Do, and S. Kambhampati. Effective approaches for partial satisfaction (oversubscription) planning. In Proceedings of AAAI, pages 562-569, 2004]

    • [M.H.L. van den Briel, T. Vossen, and S. Kambhampati. Reviving integer programming approaches for AI planning: A branch-and-cut framework. In Proceedings of ICAPS, pages 161-170, 2005]

    • [M.B. Do, J. Benton, M.H.L. van den Briel, and S. Kambhampati. Planning with goal utility dependencies. In Proceedings of IJCAI, pages 1872-1878, 2007]

    • [J. Benton, M.H.L. van den Briel, and S. Kambhampati. A hybrid linear programming and relaxed plan heuristic for partial satisfaction planning problems. In Proceedings of ICAPS, pages 24-41, 2007]

    • [M.H.L. van den Briel, J. Benton, S. Kambhampati, and T. Vossen. An LP-based heuristic for optimal planning. In Proceedings of CP, pages 651-665, 2007]

Cited by 6

Cited by 31

Cited by 15

Cited by 3

Cited by 4

Cited by 3


Publications status1

Publications status

  • Workshop and posters

    • [M.H.L. van den Briel, R. Sanchez, and S. Kambhampati. Over-Subscription in Planning: a Partial Satisfaction Problem. In Proceedings of ICAPS Workshop on Integrating Planning into Scheduling, 2005]

    • [M.H.L. van den Briel,. Kambhampati, and T. Vossen. Planning with numerical state variables through mixed integer programming. In Proceedings of ICAPS Poster Session, pages 5-8, 2005]

    • [M.H.L. van den Briel,. Kambhampati, and T. Vossen. Planning with preferences and trajectory constraints by integer programming. In Proceedings of ICAPS Workshop on Preferences and Soft Constraints in Planning, pages 19-22, 2006]

    • [J. Benton, M.H.L. van den Briel,. Kambhampati. Finding admissible bounds for oversubscription planning problems. In Proceedings of ICAPS Workshop on Heuristics for Domain-Independent Planning: Progress, Ideas, Limitations, Challenges, 2007]

    • [M.H.L. van den Briel,. Kambhampati, and T. Vossen. Fluent merging: A general technique to improve reachability heuristics and factored planning. In Proceedings of ICAPS Workshop on Heuristics for Domain-Independent Planning: Progress, Ideas, Limitations, Challenges, 2007]

Cited by 5

Cited by 1

Cited by 1

Citation count by Google Scholar


  • Login