Dynamic Programming Examples

1. Dynamic ProgrammingExamples Based on: Michael A. Trick: A Tutorial on Dynamic Programming http://mat.gsia.cmu.edu/classes/dynamic/dynamic.html M. A. Rosenman: Tutorial - Dynamic Programming Formulation http://people.arch.usyd.edu.au/~mike/DynamicProg/DPTutorial.95.html

2. Motivation Problems • Capital Budgeting Problem • Minimum cost from Sydney to Perth • Economic Feasibility Study • Travelling Salesman Problem

3. Capital Budgeting Problem • A corporation has $5 million to allocate to its three plants for possible expansion. • Each plant has submitted a number of proposals specifying (i) the cost of the expansion c and (ii) the total revenue expected r. • Each plant is only permitted to realize one of its proposals. • The goal is to maximize the firm’s revenues resulting from the allocation of the $5 million. Investment proposals

4. Minimum cost from Sydney to Perth • Travelling from home in Sydney to a hotel in Perth – choose from 3 hotels. • Three stopovers on the way – a number of choices of towns for each stop. • Each trip has a different distance resulting in a different cost (petrol). • Hotels have different costs. • The goal is to select a route to and a hotel in Perth so that the overall cost of the trip is minimized.

5. Economic Feasibility Study • We are asked for an advice how to best utilize a large urban area in an expanding town. • Envisaged is a mixed project of housing, retil, office and hotel areas. • Rental income is a function of the floor areas allocated to each activity. • The total floor area is limited to 7 units. • The goal is to find the mix of development which will maximize the return. • Additional constraint determines that the project must include at least 1 unit of offices.

6. Random tour Optimized tour Traveling Salesman Problem • Given a number of cities and the costs of travelling from any city to any other city, what is the cheapest round-trip route that visits each city exactly once and then returns to the starting city? • Example

7. How to Solve it?  No problem, just try all possibilities and choose the best.  It is not that easy for problem instances of a realistic size. Try it out!

8. DP: Common characteristics • The problem can be divided into stages with a decision required at each stage. • Each stage has a number of states associated with it. • The decision at one stage transforms one state into a state in the next stage. • Principal of optimality: Given the current state, the optimal decision for each of the remaining states does not depend on the previous states or decisions. • There exists a recursive relationship that identifies the optimal decision for stage j, given that stage j+1 has already been solved. • The final stage must be solvable by itself. • It is an art to determine stages and states.

9. DP: Capital Budgeting Problem • Stages – money allocated to a single plant • States variables x1 = {0,1,2,3,4,5} … amount of money spent on plant 1 x2 = {0,1,2,3,4,5} … amount of money spent on plant 1 and 2 x3 = {5} … amount of money spent on palnts 1, 2 and 3 We want x3 to be 5. • Ordering on the stages – first allocate to plant 1, then plant 2, then plant 3.

10. DP: Capital Budgeting Problem • Stage 1 computations

11. DP: Capital Budgeting Problem • Stage 2 computations • Example: Determine the best allocation for state x2=4. 1. Proposal 1 gives revenue of 0, leaves 4 for stage 1, which returns 6. Total=6. 2. Proposal 2 gives revenue of 8, leaves 2 for stage 1, which returns 6. Total=14. 3. Proposal 3 gives revenue of 9, leaves 1 for stage 1, which returns 5. Total=14. 4. Proposal 4 gives revenue of 12, leaves 0 for stage 1, which returns 0. Total=12.

12. DP: Capital Budgeting Problem • Stage 3 computations, we are interested in x3=5 1. Proposal 1 gives revenue 0, leaves 5. Previous stages give 17. Total=17. 2. Proposal 2 gives revenue 4, leaves 4. Previous stages give 14. Total=18. • The optimal solution, that gives a revenue 18, is to implement - proposal 2 at plant 3, - proposal 2 or 3 at plant 2, and - proposal 3 or 2 at plant 1.

13. DP: Minimum cost from Sydney to Perth • Stages – stopovers • States variables – cost of the trip • Decision variables - towns with its hotels

14. DP: Minimum cost from Sydney to Perth

15. DP: Minimum cost from Sydney to Perth

16. DP: Minimum cost from Sydney to Perth

17. DP: Economic Feasibility Study • Stage variable – the development type (1, 2, 3, 4) • State variable – the amount of floor area allocated to all dev. types • Decision variables – the amount of floor area allocated to the nth dev. type

18. DP: Economic Feasibility Study Expected annual profit

19. DP: Economic Feasibility Study

20. DP: Economic Feasibility Study

21. DP: Economic Feasibility Study How we get it?

22. DP: Economic Feasibility Study(Constrained Solution)

23. DP: Economic Feasibility Study(Constrained Solution)

24. DP: Economic Feasibility Study(Constrained Solution) Solution?

25. DP: Economic Feasibility Study(Constrained Solution)

26. DP: TSP • Stages – stage t represents t cities visited . • Decisions – where to go next. • States – we need to know where we have already gone, so states are represented by a pair (i, S), where S is the set of t cities already visited and i is the last city visited. • Start with stage 3 calculations: • f3(2, {2,3,4}) = 1334 • f3(3, {2,3,4}) = 1559 • f3(4, {2,3,4}) = 809 • For other stages, the recursion is ft(i,S)=minj1 and jS{cij + ft+1(j, Sj)} • Problems?  Number of states. • N=100 generates more than 51030 states at stage 50

27. Need for Something Better! • Curse of dimmensionality • Not enough memory, not enough computational power to solve hard problems. • Softcomputing techniques - Evolutionary algortihms, Artificial neural networks, Fuzzy. • The guiding principle of soft computing is: Exploit the tolerance for imprecision, uncertainty, partial truth, and approximation to achieve tractability, robustness and low solution cost.