1 / 24

Topic 10

Topic 10. Dynamic Programming (DP). Dynamic Programming: A Breakdown Solution Approach. Overview:

vienna
Download Presentation

Topic 10

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Topic 10 Dynamic Programming (DP)

  2. Dynamic Programming: A Breakdown Solution Approach Overview: Dynamic programming (DP) is perhaps the most fascinating tool of MS. It can be used to solve certain types of difficult problems which are not solvable, or which are difficult to solve, by other tools. DP is an approach rather than a structured tool. It decomposes problems to sub problems in a certain manner. The tool is also capable of handling multiperiod sequential decision situations. Dynamic programming is a technique that can be used to solve many optimization problems. In most applications, dynamic programming obtains solutions by working backward from the end of a problem toward the beginning, thus breaking up a large unwieldy problem into a series of smaller, more tractable problems.

  3. Two Puzzles In this section, we show how working backward can make a seemingly difficult problem almost trivial to solve. Example 1 Suppose there are 30 matches on a table. I begin by picking up 1, 2, or 3 matches. Then my opponent must pick up 1, 2, or 3 matches. We continue in this fashion until last match is picked up. The player who picks up the last match is the loser. How can I (the first player) be sure of winning the game? Example 2 I have a 9-oz cup and a 4-oz cup. My mother has ordered me to bring home exactly 6-oz of milk. How can I accomplish this goal? By starting near the end of the problem, I cleverly realized that the problem can easily be solved if I can somewhat get 1 oz of milk into a 4-oz cup. Then I can fill the 9-oz cup and empty 3-oz from the 9-oz cup into the partially filled 4-oz cup. At this point, I will be left with 6-oz of milk. After I have this flash of insight, the solution to the problem may easily be described as in Table 1 (initial situation is written last, and the final situation is written first).

  4. The Stagecoach Problem

  5. Dynamic Programming (DP) Sequential-Decision Problems: a "big" decision problem consists of a series of interrelated sub problem decisions at a series of stages. Or a "big" decision problem that can be divided into several interrelated sub problems, and the whole problem is solved when all sub problems are solved. DP: an efficient solution technique designed to solve multi-period sequential decision problem in which, an "optimal" solution to the whole problem is achieved by collecting a series of "local" solutions (may be not optimal in local sense) to all sub problems through a backward (rollback) or forward approach. Formulation of DP: There is no standard expression of DP, a specific recursive relation must be developed for each problem. • In general, all DP nature problems can be classified as: • Multi-Unit Allocation Process. • Multi-Period Planning Process • Network Process (CPM/PERT). • Markov-Chain Process.

  6. DP: Formulation Basic Elements of DP: • Stage: a decision point where a "local" decision must be made. • State: the choices of action at each stage (one or more). • Output: dependent performance measures (e.g., cost/distance) • Local (immediate) output value from a stage decision. • Total (accumulative) output value from all stage decisions. • Optimal Total output value from optimal solution decision. • Recursive relation: an expression to link two adjacent stages. • Policy: a consistent and complete plan for decision-making. • Principle of Optimality (Basic Principle of DP): At any stage, the optimal policy (decision) for all remaining stages is only dependent on the state of "current" stage (no relationship to the previous stage decisions). • Structure of DP: Steps in Formulation of DP Problem: • Determine Stages, States, Decision Variables, Dependent • Measure (Output Values).*Establish the recursive relations between stages. • Solve the problem stage-by-stage through a backward procedure

  7. Stage 3 Input and Output

  8. The Stagecoach Problem

  9. Dynamic ProgrammingAdvertising Strategy

  10. DP: Advantage and Limitation Advantages of DP: • Simplify a relatively "large" and complex problem through solving a series of relatively small and simple sub problems. • Sensitivity analysis is performed along with solving process (e.g., if reduce one stage, the best policy is already known). • The alternative secondary "optimal" solutions are identified automatically along the solving process. • Great efficiency can be achieved when problem size is large. Limitations of DP: • There is no standard formulation method. Specific recursive relationship must be developed for each problem. • The computational requirements of DP is still NP-complete. That is, for a realistic large size problem, a huge computational time effort is unavoidable.

  11. The Highway Map between Rice and Dixieville

  12. Solution for the Three-Stage Problem

  13. Steps in Dynamic Programming • Divide the Problem into stages • Solve the last stage first • Work backwards solving stages • Optimal solution obtained from all solved stages

  14. Dynamic Programming Terminology • Stage- A logical sub problem • State Variable- Possible condition • Decision Variable – Alternatives • Decision Criterion –problem objectives • Optimal Policy- A set of decision rules • Transformation – Relationship between stages

More Related