Dynamic programming – Minimisation problem

1 / 29

# Dynamic programming – Minimisation problem - PowerPoint PPT Presentation

Dynamic programming – Minimisation problem. Problem 1: Find the route through the network from A to H with minimal total weight. B. 5. F. 3. 3. 4. 4. 1. 2. 2. A. C. E. H. 5. 2. 3. 1. 3. 6. D. G. 5. Dynamic programming – Minimisation problem.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## PowerPoint Slideshow about ' Dynamic programming – Minimisation problem' - winter-myers

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Dynamic programming – Minimisation problem

Problem 1: Find the route through the network from A to H with minimal total weight.

B

5

F

3

3

4

4

1

2

2

A

C

E

H

5

2

3

1

3

6

D

G

5

Dynamic programming – Minimisation problem

The first step is to label all the nodes with stage and state variables.

The stage variable for a node is the maximum number of transitions required to get from this node to the end node, H. All nodes with the same stage variable are then given a different state variable, starting with 1 at the top of the page and working downwards.

H has stage variable 0, and is given the state variable 1.

B

5

F

3

3

4

4

1

2

2

A

C

E

H

5

(0, 1)

2

3

1

3

6

D

G

5

Dynamic programming – Minimisation problem

The first step is to label all the nodes with stage and state variables.

The stage variable for a node is the maximum number of transitions required to get from this node to the end node, H. All nodes with the same stage variable are then given a different state variable, starting with 1 at the top of the page and working downwards.

F and G have stage variables of 1.

F is given state variable 1, and G is given state variable 2.

B

5

F

(1, 1)

3

3

4

4

1

2

2

A

C

E

H

5

(0, 1)

2

3

1

3

6

D

G

5

(1, 2)

Dynamic programming – Minimisation problem

The first step is to label all the nodes with stage and state variables.

The stage variable for a node is the maximum number of transitions required to get from this node to the end node, H. All nodes with the same stage variable are then given a different state variable, starting with 1 at the top of the page and working downwards.

E is the only node with stage variable 2.

E is given state variable 1.

B

5

F

(1, 1)

You could go via G instead which also gives two transitions

3

3

4

4

1

2

2

(2, 1)

A

C

E

H

5

(0, 1)

2

3

1

3

6

D

G

5

(1, 2)

Dynamic programming – Minimisation problem

The first step is to label all the nodes with stage and state variables.

The stage variable for a node is the maximum number of transitions required to get from this node to the end node, H. All nodes with the same stage variable are then given a different state variable, starting with 1 at the top of the page and working downwards.

C is the only node with stage variable 3.

C is given state variable 1.

B

5

F

(1, 1)

3

3

4

4

1

2

2

(2, 1)

A

C

E

H

(3, 1)

5

(0, 1)

2

3

1

3

6

D

G

5

(1, 2)

Dynamic programming – Minimisation problem

The first step is to label all the nodes with stage and state variables.

The stage variable for a node is the maximum number of transitions required to get from this node to the end node, H. All nodes with the same stage variable are then given a different state variable, starting with 1 at the top of the page and working downwards.

B and D have stage variables of 4.

B is given state variable 1, and D is given state variable 2.

B

5

F

(4, 1)

(1, 1)

3

3

4

4

1

2

2

(2, 1)

A

C

E

H

(3, 1)

5

(0, 1)

2

3

1

3

6

D

G

(4, 2)

5

(1, 2)

Dynamic programming – Minimisation problem

The first step is to label all the nodes with stage and state variables.

The stage variable for a node is the maximum number of transitions required to get from this node to the end node, H. All nodes with the same stage variable are then given a different state variable, starting with 1 at the top of the page and working downwards.

A has stage variable 5.

A is given state variable 1.

B

5

F

(4, 1)

(1, 1)

3

3

4

4

1

2

2

(2, 1)

A

C

E

H

(5, 1)

(3, 1)

5

(0, 1)

2

3

1

3

6

D

G

(4, 2)

5

(1, 2)

Dynamic programming – Minimisation problem

The next step is to set up a table to show your working.

B

5

F

(4, 1)

(1, 1)

3

3

4

4

1

2

2

(2, 1)

A

C

E

H

(5, 1)

(3, 1)

5

(0, 1)

2

3

1

3

6

D

G

(4, 2)

5

(1, 2)

Dynamic programming – Minimisation problem

Current minimum

5

B

F

(1, 1)

Stage

State

Action

Value

(4, 1)

4

F (1, 1)

1

3

3

3

3

4

1

1

2

(2, 1)

2

E

C

H

A

5

(3, 1)

(0, 1)

(5, 1)

3

6

2

3

1

G

D

(4, 2)

5

(1, 2)

In Stage 1, consider nodes with stage variable 1. These are F and G.

There is only one possible action from F, which is FH.

This has value 3.

The minimum value of a route starting from F is therefore 3 .

Dynamic programming – Minimisation problem

Current minimum

5

B

F

(1, 1)

Stage

State

Action

Value

(4, 1)

4

F (1, 1)

1

3

3

3

3

4

1

1

G (1, 2)

1

6

6

2

(2, 1)

2

E

C

H

A

5

(3, 1)

(0, 1)

(5, 1)

3

6

2

3

1

G

D

(4, 2)

5

(1, 2)

In Stage 1, consider nodes with stage variable 1. These are F and G.

There is only one possible action from G, which is GH.

This has value 6.

The minimum value of a route starting from G is therefore 6 .

Dynamic programming – Minimisation problem

Current minimum

5

B

F

(1, 1)

Stage

State

Action

Value

(4, 1)

4

F (1, 1)

1

3

3

3

3

4

1

1

G (1, 2)

1

6

6

2

(2, 1)

2

E

C

H

A

1

1+3=4

5

(3, 1)

(0, 1)

(5, 1)

2

E (2, 1)

2

3

6

3

2

3

1

G

D

(4, 2)

5

(1, 2)

In Stage 2, consider nodes with stage variable 2. This is E only.

There are three possible actions from E.

Action 1 is EF.

The value of EF is 1. From stage 1, the minimum value of a route from F is 3.

So the value of this route is 4.

Dynamic programming – Minimisation problem

Current minimum

5

B

F

(1, 1)

Stage

State

Action

Value

(4, 1)

4

F (1, 1)

1

3

3

3

3

4

1

1

G (1, 2)

1

6

6

2

(2, 1)

2

E

C

H

A

1

1+3=4

5

(3, 1)

(0, 1)

(5, 1)

2

E (2, 1)

2

5

3

6

3

2

3

1

G

D

(4, 2)

5

(1, 2)

In Stage 2, consider nodes with stage variable 2. This is E only.

There are three possible actions from E.

Action 2 is EH.

The value of EH is 5.

Dynamic programming – Minimisation problem

Current minimum

5

B

F

(1, 1)

Stage

State

Action

Value

(4, 1)

4

F (1, 1)

1

3

3

3

3

4

1

1

G (1, 2)

1

6

6

2

(2, 1)

2

E

C

H

A

1

1+3=4

5

(3, 1)

(0, 1)

(5, 1)

2

E (2, 1)

2

5

3

6

3

3+6=9

2

3

1

G

D

(4, 2)

5

(1, 2)

In Stage 2, consider nodes with stage variable 2. This is E only.

There are three possible actions from E.

Action 3 is EG.

The value of EG is 3. From stage 1, the minimum value of a route from G is 6.

So the value of this route is 9.

Dynamic programming – Minimisation problem

Current minimum

5

B

F

(1, 1)

Stage

State

Action

Value

(4, 1)

4

F (1, 1)

1

3

3

3

3

4

1

1

G (1, 2)

1

6

6

2

(2, 1)

2

E

C

H

A

1

1+3=4

4

5

(3, 1)

(0, 1)

(5, 1)

2

E (2, 1)

2

5

3

6

3

3+6=9

2

3

1

G

D

(4, 2)

5

(1, 2)

In Stage 2, consider nodes with stage variable 2. This is E only.

The minimum value of a route starting from E is therefore 4.

Dynamic programming – Minimisation problem

Current minimum

5

B

F

(1, 1)

Stage

State

Action

Value

(4, 1)

4

F (1, 1)

1

3

3

3

3

4

1

1

G (1, 2)

1

6

6

2

(2, 1)

2

E

C

H

A

1

1+3=4

4

5

(3, 1)

(0, 1)

(5, 1)

2

E (2, 1)

2

5

3

6

3

3+6=9

2

3

1

1

4+3=7

G

D

3

C (3, 1)

(4, 2)

5

(1, 2)

2

In Stage 3, consider nodes with stage variable 3. This is C only.

There are two possible actions from C.

Action 1 is CF.

The value of CF is 4. From stage 1, the minimum value of a route from F is 3.

So the value of this route is 7.

Dynamic programming – Minimisation problem

Current minimum

5

B

F

(1, 1)

Stage

State

Action

Value

(4, 1)

4

F (1, 1)

1

3

3

3

3

4

1

1

G (1, 2)

1

6

6

2

(2, 1)

2

E

C

H

A

1

1+3=4

4

5

(3, 1)

(0, 1)

(5, 1)

2

E (2, 1)

2

5

3

6

3

3+6=9

2

3

1

1

4+3=7

G

D

3

C (3, 1)

(4, 2)

5

(1, 2)

2

2+4=6

In Stage 3, consider nodes with stage variable 3. This is C only.

There are two possible actions from C.

Action 2 is CE.

The value of CE is 2. From stage 2, the minimum value of a route from E is 4.

So the value of this route is 6.

Dynamic programming – Minimisation problem

Current minimum

5

B

F

(1, 1)

Stage

State

Action

Value

(4, 1)

4

F (1, 1)

1

3

3

3

3

4

1

1

G (1, 2)

1

6

6

2

(2, 1)

2

E

C

H

A

1

1+3=4

4

5

(3, 1)

(0, 1)

(5, 1)

2

E (2, 1)

2

5

3

6

3

3+6=9

2

3

1

1

4+3=7

G

D

3

C (3, 1)

(4, 2)

5

(1, 2)

2

2+4=6

6

In Stage 3, consider nodes with stage variable 3. This is C only.

The minimum value of a route starting from C is therefore 6.

Dynamic programming – Minimisation problem

Current minimum

5

B

F

(1, 1)

Stage

State

Action

Value

(4, 1)

4

F (1, 1)

1

3

3

3

3

4

1

1

G (1, 2)

1

6

6

2

(2, 1)

2

E

C

H

A

1

1+3=4

4

5

(3, 1)

(0, 1)

(5, 1)

2

E (2, 1)

2

5

3

6

3

3+6=9

2

3

1

1

4+3=7

G

D

3

C (3, 1)

(4, 2)

5

(1, 2)

2

2+4=6

6

1

5+3=8

In Stage 4, consider nodes with stage variable 4. These are B and D.

B (4, 1)

2

4

There are two possible actions from B.

Action 1 is BF.

The value of BF is 5. From stage 1, the minimum value of a route from F is 3.

So the value of this route is 8.

Dynamic programming – Minimisation problem

Current minimum

5

B

F

(1, 1)

Stage

State

Action

Value

(4, 1)

4

F (1, 1)

1

3

3

3

3

4

1

1

G (1, 2)

1

6

6

2

(2, 1)

2

E

C

H

A

1

1+3=4

4

5

(3, 1)

(0, 1)

(5, 1)

2

E (2, 1)

2

5

3

6

3

3+6=9

2

3

1

1

4+3=7

G

D

3

C (3, 1)

(4, 2)

5

(1, 2)

2

2+4=6

6

1

5+3=8

In Stage 4, consider nodes with stage variable 4. These are B and D.

B (4, 1)

2

3+6=9

4

There are two possible actions from B.

Action 2 is BC.

The value of BC is 3. From stage 3, the minimum value of a route from C is 6.

So the value of this route is 9.

Dynamic programming – Minimisation problem

Current minimum

5

B

F

(1, 1)

Stage

State

Action

Value

(4, 1)

4

F (1, 1)

1

3

3

3

3

4

1

1

G (1, 2)

1

6

6

2

(2, 1)

2

E

C

H

A

1

1+3=4

4

5

(3, 1)

(0, 1)

(5, 1)

2

E (2, 1)

2

5

3

6

3

3+6=9

2

3

1

1

4+3=7

G

D

3

C (3, 1)

(4, 2)

5

(1, 2)

2

2+4=6

6

1

5+3=8

8

In Stage 4, consider nodes with stage variable 4. These are B and D.

B (4, 1)

2

3+6=9

4

The minimum value of a route starting from B is therefore 8.

Dynamic programming – Minimisation problem

Current minimum

5

B

F

(1, 1)

Stage

State

Action

Value

(4, 1)

4

F (1, 1)

1

3

3

3

3

4

1

1

G (1, 2)

1

6

6

2

(2, 1)

2

E

C

H

A

1

1+3=4

4

5

(3, 1)

(0, 1)

(5, 1)

2

E (2, 1)

2

5

3

6

3

3+6=9

2

3

1

1

4+3=7

G

D

3

C (3, 1)

(4, 2)

5

(1, 2)

2

2+4=6

6

1

5+3=8

8

In Stage 4, consider nodes with stage variable 4. These are B and D.

B (4, 1)

2

3+6=9

4

1

3+6=9

There are three possible actions from D.

2

D (4, 2)

Action 1 is DC.

3

The value of DC is 3. From stage 3, the minimum value of a route from C is 6.

So the value of this route is 9.

Dynamic programming – Minimisation problem

Current minimum

5

B

F

(1, 1)

Stage

State

Action

Value

(4, 1)

4

F (1, 1)

1

3

3

3

3

4

1

1

G (1, 2)

1

6

6

2

(2, 1)

2

E

C

H

A

1

1+3=4

4

5

(3, 1)

(0, 1)

(5, 1)

2

E (2, 1)

2

5

3

6

3

3+6=9

2

3

1

1

4+3=7

G

D

3

C (3, 1)

(4, 2)

5

(1, 2)

2

2+4=6

6

1

5+3=8

8

In Stage 4, consider nodes with stage variable 4. These are B and D.

B (4, 1)

2

3+6=9

4

1

3+6=9

There are three possible actions from D.

2

2+4=6

D (4, 2)

Action 2 is DE.

3

The value of DE is 2. From stage 2, the minimum value of a route from E is 4.

So the value of this route is 6.

Dynamic programming – Minimisation problem

Current minimum

5

B

F

(1, 1)

Stage

State

Action

Value

(4, 1)

4

F (1, 1)

1

3

3

3

3

4

1

1

G (1, 2)

1

6

6

2

(2, 1)

2

E

C

H

A

1

1+3=4

4

5

(3, 1)

(0, 1)

(5, 1)

2

E (2, 1)

2

5

3

6

3

3+6=9

2

3

1

1

4+3=7

G

D

3

C (3, 1)

(4, 2)

5

(1, 2)

2

2+4=6

6

1

5+3=8

8

In Stage 4, consider nodes with stage variable 4. These are B and D.

B (4, 1)

2

3+6=9

4

1

3+6=9

There are three possible actions from D.

2

2+4=6

D (4, 2)

Action 3 is DG.

3

5+6=11

The value of DG is 5. From stage 1, the minimum value of a route from G is 6.

So the value of this route is 11.

Dynamic programming – Minimisation problem

Current minimum

5

B

F

(1, 1)

Stage

State

Action

Value

(4, 1)

4

F (1, 1)

1

3

3

3

3

4

1

1

G (1, 2)

1

6

6

2

(2, 1)

2

E

C

H

A

1

1+3=4

4

5

(3, 1)

(0, 1)

(5, 1)

2

E (2, 1)

2

5

3

6

3

3+6=9

2

3

1

1

4+3=7

G

D

3

C (3, 1)

(4, 2)

5

(1, 2)

2

2+4=6

6

1

5+3=8

8

In Stage 4, consider nodes with stage variable 4. These are B and D.

B (4, 1)

2

3+6=9

4

1

3+6=9

The minimum value of a route starting from D is therefore 6.

2

2+4=6

6

D (4, 2)

3

5+6=11

Dynamic programming – Minimisation problem

Current minimum

5

B

F

(1, 1)

Stage

State

Action

Value

(4, 1)

4

F (1, 1)

1

3

3

3

3

4

1

1

G (1, 2)

1

6

6

2

(2, 1)

2

E

C

H

A

1

1+3=4

4

5

(3, 1)

(0, 1)

(5, 1)

2

E (2, 1)

2

5

3

6

3

3+6=9

2

3

1

1

4+3=7

G

D

3

C (3, 1)

(4, 2)

5

(1, 2)

2

2+4=6

6

1

5+3=8

8

In Stage 5, consider nodes with stage variable 5. This is A only.

B (4, 1)

2

3+6=9

4

1

3+6=9

There are three possible actions from A.

2

2+4=6

6

D (4, 2)

Action 1 is AB.

3

5+6=11

The value of AB is 4. From stage 4, the minimum value of a route from B is 8.

1

4+8=12

A (5, 1)

2

5

So the value of this route is 12.

3

Dynamic programming – Minimisation problem

Current minimum

5

B

F

(1, 1)

Stage

State

Action

Value

(4, 1)

4

F (1, 1)

1

3

3

3

3

4

1

1

G (1, 2)

1

6

6

2

(2, 1)

2

E

C

H

A

1

1+3=4

4

5

(3, 1)

(0, 1)

(5, 1)

2

E (2, 1)

2

5

3

6

3

3+6=9

2

3

1

1

4+3=7

G

D

3

C (3, 1)

(4, 2)

5

(1, 2)

2

2+4=6

6

1

5+3=8

8

In Stage 5, consider nodes with stage variable 5. This is A only.

B (4, 1)

2

3+6=9

4

1

3+6=9

There are three possible actions from A.

2

2+4=6

6

D (4, 2)

Action 2 is AC.

3

5+6=11

The value of AC is 2. From stage 3, the minimum value of a route from C is 6.

1

4+8=12

A (5, 1)

2

2+6=8

5

So the value of this route is 8.

3

Dynamic programming – Minimisation problem

Current minimum

5

B

F

(1, 1)

Stage

State

Action

Value

(4, 1)

4

F (1, 1)

1

3

3

3

3

4

1

1

G (1, 2)

1

6

6

2

(2, 1)

2

E

C

H

A

1

1+3=4

4

5

(3, 1)

(0, 1)

(5, 1)

2

E (2, 1)

2

5

3

6

3

3+6=9

2

3

1

1

4+3=7

G

D

3

C (3, 1)

(4, 2)

5

(1, 2)

2

2+4=6

6

1

5+3=8

8

In Stage 5, consider nodes with stage variable 5. This is A only.

B (4, 1)

2

3+6=9

4

1

3+6=9

There are three possible actions from A.

2

2+4=6

6

D (4, 2)

3

5+6=11

The value of AD is 1. From stage 4, the minimum value of a route from D is 6.

1

4+8=12

A (5, 1)

2

2+6=8

5

So the value of this route is 7.

3

1+6=7

Dynamic programming – Minimisation problem

Current minimum

5

B

F

(1, 1)

Stage

State

Action

Value

(4, 1)

4

F (1, 1)

1

3

3

3

3

4

1

1

G (1, 2)

1

6

6

2

(2, 1)

2

E

C

H

A

1

1+3=4

4

5

(3, 1)

(0, 1)

(5, 1)

2

E (2, 1)

2

5

3

6

3

3+6=9

2

3

1

1

4+3=7

G

D

3

C (3, 1)

(4, 2)

5

(1, 2)

2

2+4=6

6

1

5+3=8

8

In Stage 5, consider nodes with stage variable 5. This is A only.

B (4, 1)

2

3+6=9

4

1

3+6=9

The minimum value of a route starting from A is therefore 7.

2

2+4=6

6

D (4, 2)

3

5+6=11

1

4+8=12

A (5, 1)

2

2+6=8

5

3

1+6=7

7

Dynamic programming – Minimisation problem

Current minimum

5

B

F

(1, 1)

Stage

State

Action

Value

(4, 1)

4

F (1, 1)

1

3

3

3

3

4

1

1

G (1, 2)

1

6

6

2

(2, 1)

2

E

C

H

A

1

1+3=4

4

5

(3, 1)

(0, 1)

(5, 1)

2

E (2, 1)

2

5

3

6

3

3+6=9

2

3

1

1

4+3=7

G

D

3

C (3, 1)

(4, 2)

5

(1, 2)

2

2+4=6

6

1

5+3=8

8

From the completed table, the minimum weight for a route from A to H is 7.

B (4, 1)

2

3+6=9

4

1

3+6=9

The table shows that you need to take the third action from A, which is AD,

2

2+4=6

6

D (4, 2)

3

5+6=11

then the second action out of D, which is DE,

1

4+8=12

then the first action out of E, which is EF,

A (5, 1)

2

2+6=8

and finally the action FH.

5

3

1+6=7

7

So the minimum weight route is ADEFH.