1 / 29

Dynamic programming – Minimisation problem

Dynamic programming – Minimisation problem. Problem 1: Find the route through the network from A to H with minimal total weight. B. 5. F. 3. 3. 4. 4. 1. 2. 2. A. C. E. H. 5. 2. 3. 1. 3. 6. D. G. 5. Dynamic programming – Minimisation problem.

winter-myers
Download Presentation

Dynamic programming – Minimisation problem

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Dynamic programming – Minimisation problem Problem 1: Find the route through the network from A to H with minimal total weight. B 5 F 3 3 4 4 1 2 2 A C E H 5 2 3 1 3 6 D G 5

  2. Dynamic programming – Minimisation problem The first step is to label all the nodes with stage and state variables. The stage variable for a node is the maximum number of transitions required to get from this node to the end node, H. All nodes with the same stage variable are then given a different state variable, starting with 1 at the top of the page and working downwards. H has stage variable 0, and is given the state variable 1. B 5 F 3 3 4 4 1 2 2 A C E H 5 (0, 1) 2 3 1 3 6 D G 5

  3. Dynamic programming – Minimisation problem The first step is to label all the nodes with stage and state variables. The stage variable for a node is the maximum number of transitions required to get from this node to the end node, H. All nodes with the same stage variable are then given a different state variable, starting with 1 at the top of the page and working downwards. F and G have stage variables of 1. F is given state variable 1, and G is given state variable 2. B 5 F (1, 1) 3 3 4 4 1 2 2 A C E H 5 (0, 1) 2 3 1 3 6 D G 5 (1, 2)

  4. Dynamic programming – Minimisation problem The first step is to label all the nodes with stage and state variables. The stage variable for a node is the maximum number of transitions required to get from this node to the end node, H. All nodes with the same stage variable are then given a different state variable, starting with 1 at the top of the page and working downwards. E is the only node with stage variable 2. E is given state variable 1. B 5 F (1, 1) You could go via G instead which also gives two transitions 3 3 4 4 1 2 2 (2, 1) A C E H 5 (0, 1) 2 3 1 3 6 D G 5 (1, 2)

  5. Dynamic programming – Minimisation problem The first step is to label all the nodes with stage and state variables. The stage variable for a node is the maximum number of transitions required to get from this node to the end node, H. All nodes with the same stage variable are then given a different state variable, starting with 1 at the top of the page and working downwards. C is the only node with stage variable 3. C is given state variable 1. B 5 F (1, 1) 3 3 4 4 1 2 2 (2, 1) A C E H (3, 1) 5 (0, 1) 2 3 1 3 6 D G 5 (1, 2)

  6. Dynamic programming – Minimisation problem The first step is to label all the nodes with stage and state variables. The stage variable for a node is the maximum number of transitions required to get from this node to the end node, H. All nodes with the same stage variable are then given a different state variable, starting with 1 at the top of the page and working downwards. B and D have stage variables of 4. B is given state variable 1, and D is given state variable 2. B 5 F (4, 1) (1, 1) 3 3 4 4 1 2 2 (2, 1) A C E H (3, 1) 5 (0, 1) 2 3 1 3 6 D G (4, 2) 5 (1, 2)

  7. Dynamic programming – Minimisation problem The first step is to label all the nodes with stage and state variables. The stage variable for a node is the maximum number of transitions required to get from this node to the end node, H. All nodes with the same stage variable are then given a different state variable, starting with 1 at the top of the page and working downwards. A has stage variable 5. A is given state variable 1. B 5 F (4, 1) (1, 1) 3 3 4 4 1 2 2 (2, 1) A C E H (5, 1) (3, 1) 5 (0, 1) 2 3 1 3 6 D G (4, 2) 5 (1, 2)

  8. Dynamic programming – Minimisation problem The next step is to set up a table to show your working. B 5 F (4, 1) (1, 1) 3 3 4 4 1 2 2 (2, 1) A C E H (5, 1) (3, 1) 5 (0, 1) 2 3 1 3 6 D G (4, 2) 5 (1, 2)

  9. Dynamic programming – Minimisation problem Current minimum 5 B F (1, 1) Stage State Action Value (4, 1) 4 F (1, 1) 1 3 3 3 3 4 1 1 2 (2, 1) 2 E C H A 5 (3, 1) (0, 1) (5, 1) 3 6 2 3 1 G D (4, 2) 5 (1, 2) In Stage 1, consider nodes with stage variable 1. These are F and G. There is only one possible action from F, which is FH. This has value 3. The minimum value of a route starting from F is therefore 3 .

  10. Dynamic programming – Minimisation problem Current minimum 5 B F (1, 1) Stage State Action Value (4, 1) 4 F (1, 1) 1 3 3 3 3 4 1 1 G (1, 2) 1 6 6 2 (2, 1) 2 E C H A 5 (3, 1) (0, 1) (5, 1) 3 6 2 3 1 G D (4, 2) 5 (1, 2) In Stage 1, consider nodes with stage variable 1. These are F and G. There is only one possible action from G, which is GH. This has value 6. The minimum value of a route starting from G is therefore 6 .

  11. Dynamic programming – Minimisation problem Current minimum 5 B F (1, 1) Stage State Action Value (4, 1) 4 F (1, 1) 1 3 3 3 3 4 1 1 G (1, 2) 1 6 6 2 (2, 1) 2 E C H A 1 1+3=4 5 (3, 1) (0, 1) (5, 1) 2 E (2, 1) 2 3 6 3 2 3 1 G D (4, 2) 5 (1, 2) In Stage 2, consider nodes with stage variable 2. This is E only. There are three possible actions from E. Action 1 is EF. The value of EF is 1. From stage 1, the minimum value of a route from F is 3. So the value of this route is 4.

  12. Dynamic programming – Minimisation problem Current minimum 5 B F (1, 1) Stage State Action Value (4, 1) 4 F (1, 1) 1 3 3 3 3 4 1 1 G (1, 2) 1 6 6 2 (2, 1) 2 E C H A 1 1+3=4 5 (3, 1) (0, 1) (5, 1) 2 E (2, 1) 2 5 3 6 3 2 3 1 G D (4, 2) 5 (1, 2) In Stage 2, consider nodes with stage variable 2. This is E only. There are three possible actions from E. Action 2 is EH. The value of EH is 5.

  13. Dynamic programming – Minimisation problem Current minimum 5 B F (1, 1) Stage State Action Value (4, 1) 4 F (1, 1) 1 3 3 3 3 4 1 1 G (1, 2) 1 6 6 2 (2, 1) 2 E C H A 1 1+3=4 5 (3, 1) (0, 1) (5, 1) 2 E (2, 1) 2 5 3 6 3 3+6=9 2 3 1 G D (4, 2) 5 (1, 2) In Stage 2, consider nodes with stage variable 2. This is E only. There are three possible actions from E. Action 3 is EG. The value of EG is 3. From stage 1, the minimum value of a route from G is 6. So the value of this route is 9.

  14. Dynamic programming – Minimisation problem Current minimum 5 B F (1, 1) Stage State Action Value (4, 1) 4 F (1, 1) 1 3 3 3 3 4 1 1 G (1, 2) 1 6 6 2 (2, 1) 2 E C H A 1 1+3=4 4 5 (3, 1) (0, 1) (5, 1) 2 E (2, 1) 2 5 3 6 3 3+6=9 2 3 1 G D (4, 2) 5 (1, 2) In Stage 2, consider nodes with stage variable 2. This is E only. The minimum value of a route starting from E is therefore 4.

  15. Dynamic programming – Minimisation problem Current minimum 5 B F (1, 1) Stage State Action Value (4, 1) 4 F (1, 1) 1 3 3 3 3 4 1 1 G (1, 2) 1 6 6 2 (2, 1) 2 E C H A 1 1+3=4 4 5 (3, 1) (0, 1) (5, 1) 2 E (2, 1) 2 5 3 6 3 3+6=9 2 3 1 1 4+3=7 G D 3 C (3, 1) (4, 2) 5 (1, 2) 2 In Stage 3, consider nodes with stage variable 3. This is C only. There are two possible actions from C. Action 1 is CF. The value of CF is 4. From stage 1, the minimum value of a route from F is 3. So the value of this route is 7.

  16. Dynamic programming – Minimisation problem Current minimum 5 B F (1, 1) Stage State Action Value (4, 1) 4 F (1, 1) 1 3 3 3 3 4 1 1 G (1, 2) 1 6 6 2 (2, 1) 2 E C H A 1 1+3=4 4 5 (3, 1) (0, 1) (5, 1) 2 E (2, 1) 2 5 3 6 3 3+6=9 2 3 1 1 4+3=7 G D 3 C (3, 1) (4, 2) 5 (1, 2) 2 2+4=6 In Stage 3, consider nodes with stage variable 3. This is C only. There are two possible actions from C. Action 2 is CE. The value of CE is 2. From stage 2, the minimum value of a route from E is 4. So the value of this route is 6.

  17. Dynamic programming – Minimisation problem Current minimum 5 B F (1, 1) Stage State Action Value (4, 1) 4 F (1, 1) 1 3 3 3 3 4 1 1 G (1, 2) 1 6 6 2 (2, 1) 2 E C H A 1 1+3=4 4 5 (3, 1) (0, 1) (5, 1) 2 E (2, 1) 2 5 3 6 3 3+6=9 2 3 1 1 4+3=7 G D 3 C (3, 1) (4, 2) 5 (1, 2) 2 2+4=6 6 In Stage 3, consider nodes with stage variable 3. This is C only. The minimum value of a route starting from C is therefore 6.

  18. Dynamic programming – Minimisation problem Current minimum 5 B F (1, 1) Stage State Action Value (4, 1) 4 F (1, 1) 1 3 3 3 3 4 1 1 G (1, 2) 1 6 6 2 (2, 1) 2 E C H A 1 1+3=4 4 5 (3, 1) (0, 1) (5, 1) 2 E (2, 1) 2 5 3 6 3 3+6=9 2 3 1 1 4+3=7 G D 3 C (3, 1) (4, 2) 5 (1, 2) 2 2+4=6 6 1 5+3=8 In Stage 4, consider nodes with stage variable 4. These are B and D. B (4, 1) 2 4 There are two possible actions from B. Action 1 is BF. The value of BF is 5. From stage 1, the minimum value of a route from F is 3. So the value of this route is 8.

  19. Dynamic programming – Minimisation problem Current minimum 5 B F (1, 1) Stage State Action Value (4, 1) 4 F (1, 1) 1 3 3 3 3 4 1 1 G (1, 2) 1 6 6 2 (2, 1) 2 E C H A 1 1+3=4 4 5 (3, 1) (0, 1) (5, 1) 2 E (2, 1) 2 5 3 6 3 3+6=9 2 3 1 1 4+3=7 G D 3 C (3, 1) (4, 2) 5 (1, 2) 2 2+4=6 6 1 5+3=8 In Stage 4, consider nodes with stage variable 4. These are B and D. B (4, 1) 2 3+6=9 4 There are two possible actions from B. Action 2 is BC. The value of BC is 3. From stage 3, the minimum value of a route from C is 6. So the value of this route is 9.

  20. Dynamic programming – Minimisation problem Current minimum 5 B F (1, 1) Stage State Action Value (4, 1) 4 F (1, 1) 1 3 3 3 3 4 1 1 G (1, 2) 1 6 6 2 (2, 1) 2 E C H A 1 1+3=4 4 5 (3, 1) (0, 1) (5, 1) 2 E (2, 1) 2 5 3 6 3 3+6=9 2 3 1 1 4+3=7 G D 3 C (3, 1) (4, 2) 5 (1, 2) 2 2+4=6 6 1 5+3=8 8 In Stage 4, consider nodes with stage variable 4. These are B and D. B (4, 1) 2 3+6=9 4 The minimum value of a route starting from B is therefore 8.

  21. Dynamic programming – Minimisation problem Current minimum 5 B F (1, 1) Stage State Action Value (4, 1) 4 F (1, 1) 1 3 3 3 3 4 1 1 G (1, 2) 1 6 6 2 (2, 1) 2 E C H A 1 1+3=4 4 5 (3, 1) (0, 1) (5, 1) 2 E (2, 1) 2 5 3 6 3 3+6=9 2 3 1 1 4+3=7 G D 3 C (3, 1) (4, 2) 5 (1, 2) 2 2+4=6 6 1 5+3=8 8 In Stage 4, consider nodes with stage variable 4. These are B and D. B (4, 1) 2 3+6=9 4 1 3+6=9 There are three possible actions from D. 2 D (4, 2) Action 1 is DC. 3 The value of DC is 3. From stage 3, the minimum value of a route from C is 6. So the value of this route is 9.

  22. Dynamic programming – Minimisation problem Current minimum 5 B F (1, 1) Stage State Action Value (4, 1) 4 F (1, 1) 1 3 3 3 3 4 1 1 G (1, 2) 1 6 6 2 (2, 1) 2 E C H A 1 1+3=4 4 5 (3, 1) (0, 1) (5, 1) 2 E (2, 1) 2 5 3 6 3 3+6=9 2 3 1 1 4+3=7 G D 3 C (3, 1) (4, 2) 5 (1, 2) 2 2+4=6 6 1 5+3=8 8 In Stage 4, consider nodes with stage variable 4. These are B and D. B (4, 1) 2 3+6=9 4 1 3+6=9 There are three possible actions from D. 2 2+4=6 D (4, 2) Action 2 is DE. 3 The value of DE is 2. From stage 2, the minimum value of a route from E is 4. So the value of this route is 6.

  23. Dynamic programming – Minimisation problem Current minimum 5 B F (1, 1) Stage State Action Value (4, 1) 4 F (1, 1) 1 3 3 3 3 4 1 1 G (1, 2) 1 6 6 2 (2, 1) 2 E C H A 1 1+3=4 4 5 (3, 1) (0, 1) (5, 1) 2 E (2, 1) 2 5 3 6 3 3+6=9 2 3 1 1 4+3=7 G D 3 C (3, 1) (4, 2) 5 (1, 2) 2 2+4=6 6 1 5+3=8 8 In Stage 4, consider nodes with stage variable 4. These are B and D. B (4, 1) 2 3+6=9 4 1 3+6=9 There are three possible actions from D. 2 2+4=6 D (4, 2) Action 3 is DG. 3 5+6=11 The value of DG is 5. From stage 1, the minimum value of a route from G is 6. So the value of this route is 11.

  24. Dynamic programming – Minimisation problem Current minimum 5 B F (1, 1) Stage State Action Value (4, 1) 4 F (1, 1) 1 3 3 3 3 4 1 1 G (1, 2) 1 6 6 2 (2, 1) 2 E C H A 1 1+3=4 4 5 (3, 1) (0, 1) (5, 1) 2 E (2, 1) 2 5 3 6 3 3+6=9 2 3 1 1 4+3=7 G D 3 C (3, 1) (4, 2) 5 (1, 2) 2 2+4=6 6 1 5+3=8 8 In Stage 4, consider nodes with stage variable 4. These are B and D. B (4, 1) 2 3+6=9 4 1 3+6=9 The minimum value of a route starting from D is therefore 6. 2 2+4=6 6 D (4, 2) 3 5+6=11

  25. Dynamic programming – Minimisation problem Current minimum 5 B F (1, 1) Stage State Action Value (4, 1) 4 F (1, 1) 1 3 3 3 3 4 1 1 G (1, 2) 1 6 6 2 (2, 1) 2 E C H A 1 1+3=4 4 5 (3, 1) (0, 1) (5, 1) 2 E (2, 1) 2 5 3 6 3 3+6=9 2 3 1 1 4+3=7 G D 3 C (3, 1) (4, 2) 5 (1, 2) 2 2+4=6 6 1 5+3=8 8 In Stage 5, consider nodes with stage variable 5. This is A only. B (4, 1) 2 3+6=9 4 1 3+6=9 There are three possible actions from A. 2 2+4=6 6 D (4, 2) Action 1 is AB. 3 5+6=11 The value of AB is 4. From stage 4, the minimum value of a route from B is 8. 1 4+8=12 A (5, 1) 2 5 So the value of this route is 12. 3

  26. Dynamic programming – Minimisation problem Current minimum 5 B F (1, 1) Stage State Action Value (4, 1) 4 F (1, 1) 1 3 3 3 3 4 1 1 G (1, 2) 1 6 6 2 (2, 1) 2 E C H A 1 1+3=4 4 5 (3, 1) (0, 1) (5, 1) 2 E (2, 1) 2 5 3 6 3 3+6=9 2 3 1 1 4+3=7 G D 3 C (3, 1) (4, 2) 5 (1, 2) 2 2+4=6 6 1 5+3=8 8 In Stage 5, consider nodes with stage variable 5. This is A only. B (4, 1) 2 3+6=9 4 1 3+6=9 There are three possible actions from A. 2 2+4=6 6 D (4, 2) Action 2 is AC. 3 5+6=11 The value of AC is 2. From stage 3, the minimum value of a route from C is 6. 1 4+8=12 A (5, 1) 2 2+6=8 5 So the value of this route is 8. 3

  27. Dynamic programming – Minimisation problem Current minimum 5 B F (1, 1) Stage State Action Value (4, 1) 4 F (1, 1) 1 3 3 3 3 4 1 1 G (1, 2) 1 6 6 2 (2, 1) 2 E C H A 1 1+3=4 4 5 (3, 1) (0, 1) (5, 1) 2 E (2, 1) 2 5 3 6 3 3+6=9 2 3 1 1 4+3=7 G D 3 C (3, 1) (4, 2) 5 (1, 2) 2 2+4=6 6 1 5+3=8 8 In Stage 5, consider nodes with stage variable 5. This is A only. B (4, 1) 2 3+6=9 4 1 3+6=9 There are three possible actions from A. 2 2+4=6 6 D (4, 2) Action 3 is AD. 3 5+6=11 The value of AD is 1. From stage 4, the minimum value of a route from D is 6. 1 4+8=12 A (5, 1) 2 2+6=8 5 So the value of this route is 7. 3 1+6=7

  28. Dynamic programming – Minimisation problem Current minimum 5 B F (1, 1) Stage State Action Value (4, 1) 4 F (1, 1) 1 3 3 3 3 4 1 1 G (1, 2) 1 6 6 2 (2, 1) 2 E C H A 1 1+3=4 4 5 (3, 1) (0, 1) (5, 1) 2 E (2, 1) 2 5 3 6 3 3+6=9 2 3 1 1 4+3=7 G D 3 C (3, 1) (4, 2) 5 (1, 2) 2 2+4=6 6 1 5+3=8 8 In Stage 5, consider nodes with stage variable 5. This is A only. B (4, 1) 2 3+6=9 4 1 3+6=9 The minimum value of a route starting from A is therefore 7. 2 2+4=6 6 D (4, 2) 3 5+6=11 1 4+8=12 A (5, 1) 2 2+6=8 5 3 1+6=7 7

  29. Dynamic programming – Minimisation problem Current minimum 5 B F (1, 1) Stage State Action Value (4, 1) 4 F (1, 1) 1 3 3 3 3 4 1 1 G (1, 2) 1 6 6 2 (2, 1) 2 E C H A 1 1+3=4 4 5 (3, 1) (0, 1) (5, 1) 2 E (2, 1) 2 5 3 6 3 3+6=9 2 3 1 1 4+3=7 G D 3 C (3, 1) (4, 2) 5 (1, 2) 2 2+4=6 6 1 5+3=8 8 From the completed table, the minimum weight for a route from A to H is 7. B (4, 1) 2 3+6=9 4 1 3+6=9 The table shows that you need to take the third action from A, which is AD, 2 2+4=6 6 D (4, 2) 3 5+6=11 then the second action out of D, which is DE, 1 4+8=12 then the first action out of E, which is EF, A (5, 1) 2 2+6=8 and finally the action FH. 5 3 1+6=7 7 So the minimum weight route is ADEFH.

More Related