1 / 24

Dynamic Programming

Dynamic Programming. Study Guide for ES205 Yu-Chi Ho Jonathan T. Lee Jan. 11, 2001. Outline. Sample Problem General Formulation Linear-Quadratic Problem General Problems. 5. 6. 8. 1. 2. 5. 1. N. 8. 6. 1. 5. 2. 0. 6. 3. 4. 2. 3. 8. 3. 1. 9. 3. 6. 3. 1. 2. 3. 4.

abel
Download Presentation

Dynamic Programming

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Dynamic Programming Study Guide for ES205 Yu-Chi Ho Jonathan T. Lee Jan. 11, 2001

  2. Outline • Sample Problem • General Formulation • Linear-Quadratic Problem • General Problems

  3. 5 6 8 1 2 5 1 N 8 6 1 5 2 0 6 3 4 2 3 8 3 1 9 3 6 3 1 2 3 4 Path-Cost Problem • Find the path with minimal cost

  4. N Principle of Optimality • “An optimal policy has the property that whatever the initial state and initial decision are, the remaining decisions must constitute an optimal policy with regard to the state resulting from the first decision.”

  5. 7 6 5 5 6 6 6 8 1 2 5 1 N 8 6 1 5 2 0 6 3 4 2 3 8 3 1 9 3 6 3 1 2 3 4 Path-Cost Problem (cont.)

  6. 7 12 7 5 15 6 8 1 2 5 1 N 8 6 1 5 2 0 6 3 4 2 3 8 3 1 9 3 6 3 1 2 3 4 Path-Cost Problem (cont.)

  7. 11 15 5 9 15 6 8 1 2 5 1 N 8 6 1 5 2 0 6 3 4 2 3 8 3 1 9 3 6 3 1 2 3 4 Path-Cost Problem (cont.)

  8. N Formulation for Cost-Path Pb.

  9. N Formulation (cont.) • More generally,

  10. N General Formulation • Multistage optimization problem: • The cost-to-go with initial condition

  11. Multistage or Optimal Control Problem • Can be approached as static optimization problem with specialized equality (staircase) constraints • See study guides titled “Dynamic Systems” • These two equivalent ways will be made clear below in the solution of a specific class of problems

  12. N Linear-Quadratic Problem subject to linearsystem dynamics given the initial state x(0) where x(i) is the state variables at time i u(i) is the control variable at time i a(i) and b(i) are the cost factor at time i

  13. N LQ Problem (cont.)

  14. N LQ Problem (cont.) • Substituteinto • Set • We get

  15. N LQ Problem (cont.) • With some work, we have • LetThen, we have

  16. N LQ Problem (cont.) • With

  17. N LQ Problem (cont.) • Substitute the optimum u(N-1), then we have • Define

  18. N LQ Problem (cont.)

  19. N LQ Problem (cont.) • By induction, we have the optimal solution to bewherewith boundary condition

  20. N General Problems • Stochastic problems • Combinatorial problems • Variable termination time • Constraints in the problem

  21. N Stochastic Problem • The cost-to-gowith initial condition

  22. N Combinatorial Problem • The cost-to-gowith initial condition

  23. References: • Bellman, R., Dynamic Programming, Princeton University Press, 1957. • Bryson, Jr., A. E. and Y.-C. Ho, Applied Optimal Control: Optimization, Estimation, and Control, Taylor & Francis, 1975. • Dreyfus, S. E. and A. M. Law, The Art and Theory of Dynamic Programming, Academic Press, 1977. • Ho, Y.-C., Lecture Notes, Harvard University, 1997.

  24. References: • National Institute of Standards and Technology, Dictionary of Algorithms, Data Structures, and Problems, http://hissa.nist.gov/dads/HTML/principle.html • Ortega, A. and K. Ramchandran, “Rate-Distortion Methods for Image and Video Compression: An Overview,” IEEE Signal Processing Magazine, Nov. 1998. http://sipi.usc.edu/~ortega/RD_Examples/boxDP.html

More Related