Dynamic Programming

1. Dynamic Programming Lecture 9 Asst. Prof. Dr. İlker Kocabaş

2. Dynamic Programming History • Bellman. Pioneered the systematic study of dynamic programming in the 1950s. • Etymology. • Dynamic programming = planning over time. • Secretary of Defense was hostile to mathematical research. • Bellman sought an impressive name to avoid confrontation. • "it's impossible to use dynamic in a pejorative sense" • "something not even a Congressman could object to" Reference: Bellman, R. E. Eye of the Hurricane, An Autobiography.

3. Introduction • Dynamic Programming(DP) applies to optimiza- tion problems in which a set of choices must be made in order to arrive at an optimal solution. • As choices are made, subproblems of the same form arise. • DP is effective when a given problem may arise from more than one partial set of choices. • The key technique is to store the solution to each subproblem in case it should appear

4. Introduction (cont.) • Divide and Conquer algorithms partition the problem into independent subproblems. • Dynamic Programming is applicable when the subproblems are not independent.(In this case DP algorithm does more work than necessary) • Dynamic Programming algorithm solves every subproblem just once and then saves its answer in a table.

5. Dynamic Programming Applications • Areas. • Bioinformatics. • Control theory. • Information theory. • Operations research. • Computer science: theory, graphics, AI, systems, …. • Some famous dynamic programming algorithms. • Viterbi for hidden Markov models. • Unix diff for comparing two files. • Smith-Waterman for sequence alignment. • Bellman-Ford for shortest path routing in networks. • Cocke-Kasami-Younger for parsing context free grammars.

6. Assembly-line Scheduling

7. Assembly-line Scheduling • If we are given a list of which stations to use in line 1 and which to use in line 2, then T(n)=Θ(n). • There are 2n possible ways to choose stations. Thus, the solution is obtained by enumarating all posible ways and computing how long each takes would take Ω(2n).

8. The steps of a dynamic programming • Characterize the structure of an optimal solution • Recursively define the value of an optimal solution • Compute the value of an optimal solution in a bottom-up fashion • Construct an optimal solution from computed information

9. Step 1:The structure of the fastest way through the factory • Consider the fastest possible way(fpw) to get from the starting point through station S1,j. • If j=1, there isonly one way thatthe chassis could have gone. • For j=2, 3, ..., n there are two choices: • The chassis could have come from S1, j-1and then directly to station S1, j. • The chassis could have come from S2, j-1and then directly to station S1, j-1.

10. Step 1:The structure of the fastest way through the factory (cont.) • First, suppose that the fastest way(fw) through station S1, j is throughstation S1, j-1. • The key observation is that the chassis must have taken a fw from the starting point through station S1, j-1.

11. Step 1:The structure of the fastest way through the factory (cont.) • Similarly, suppose that the fw through station S2, j is throughstation S2, j-1. • We observe that the chassis must have taken a fw from tha starting point through station S2, j-1.

12. Step 1:The structure of the fastest way through the factory (cont.) • We can say that for assembly-line scheduling, an optimal solution to a problem contains within it an optimal solution to sub-problems (finding the fw through either S1, j-1or S2, j-1). • We refer to this property as optimal substructure.

13. The fw through stations S1, j • The fw through station S1, j-1and then directly through station S1, j. or • The fw through station S2, j-1and then through station S1, j.

14. The fw through station S2, j • The fw through station S2, j-1and then directly through station S2, j. or • The fw through station S1, j-1and then through station S2, j.

15. Step 1:The structure of the fastest way through the factory (cont.) • To solve the problem of finding the fw through station j of either line, we solve the subproblems of finding the fw through station j-1 on both lines. • Thus, we can build an optimal solution to an instance of the assemly line scheduling problem by building on optimal solutions to subproblems

16. Step 2: A recursive solution • We define the value of an optimal solution recursively in tems of the optimal solutions to subproblems. • Let fi[j] denote the fastest possible time to get a chassis from the starting point throuhg station Si, j. • Let fi* denote the fastest time to get a chassis all the way through the factory.

17. Step 2: A recursive solution (cont.)

18. Step 2: A recursive solution (cont.)

19. Step 2: A recursive solution: An example Station S1,1 Station S1,2 Station S1,3 Station S1,4 Station S1,5 Station S1,6 9 3 4 Line 1 7 4 8 2 3 2 1 3 3 4 Chassis Enters 5 2 1 2 1 4 2 5 6 4 8 5 7 Line 2 Station S1,1 Station S1,2 Station S1,3 Station S1,4 Station S1,5 Station S1,6 j 1 2 3 4 5 6 j 2 3 4 5 6 f1[j] l1[j] f*=0 l* =0 f2[j] l2[j]

20. Step 2: A recursive solution: An example Station S1,1 Station S1,2 Station S1,3 Station S1,4 Station S1,5 Station S1,6 9 3 4 Line 1 9 4 8 3 2 1 3 3 4 Chassis Enters 5 2 1 2 1 2 5 6 4 12 5 7 Line 2 Station S2,1 Station S2,2 Station S2,3 Station S2,4 Station S2,5 Station S2,6 j 1 2 3 4 5 6 j 2 3 4 5 6 f1[j] 9 l1[j] f*=0 l* =0 12 f2[j] l2[j]

21. Step 2: A recursive solution: An example Station S1,1 Station S1,2 Station S1,3 Station S1,4 Station S1,5 Station S1,6 4 Line 1 9 4 18 3 8 3 1 3 3 4 Chassis Enters 5 1 2 1 2 6 4 16 12 5 7 Line 2 Station S2,1 Station S2,2 Station S2,3 Station S2,4 Station S2,5 Station S2,6 j 1 2 3 4 5 6 j 2 3 4 5 6 f1[j] 9 18 1 l1[j] f*=0 l* =0 12 16 f2[j] 1 l2[j]

22. Step 2: A recursive solution: An example Station S1,1 Station S1,2 Station S1,3 Station S1,4 Station S1,5 Station S1,6 4 Line 1 9 4 18 20 8 3 1 3 4 Chassis Enters 5 2 1 2 4 16 22 12 5 7 Line 2 Station S2,1 Station S2,2 Station S2,3 Station S2,4 Station S2,5 Station S2,6 j 1 2 3 4 5 6 j 2 3 4 5 6 f1[j] 9 18 20 1 2 l1[j] f*=0 l* =0 12 16 22 f2[j] 1 2 l2[j]

23. Step 2: A recursive solution: An example Station S1,1 Station S1,2 Station S1,3 Station S1,4 Station S1,5 Station S1,6 24 Line 1 9 4 18 20 8 3 3 4 Chassis Enters 2 1 2 16 22 12 25 5 7 Line 2 Station S2,1 Station S2,2 Station S2,3 Station S2,4 Station S2,5 Station S2,6 j 1 2 3 4 5 6 j 2 3 4 5 6 f1[j] 9 18 20 24 1 2 1 l1[j] f*=0 l* =0 12 16 22 25 f2[j] 1 2 1 l2[j]

24. Step 2: A recursive solution: An example Station S1,1 Station S1,2 Station S1,3 Station S1,4 Station S1,5 Station S1,6 24 Line 1 9 4 18 20 32 3 4 Chassis Enters 1 2 16 22 12 25 30 7 Line 2 Station S2,1 Station S2,2 Station S2,3 Station S2,4 Station S2,5 Station S2,6 j 1 2 3 4 5 6 j 2 3 4 5 6 f1[j] 9 18 20 24 32 1 2 1 1 l1[j] f*=0 l* =0 12 16 22 25 30 f2[j] 1 2 1 2 l2[j]

25. Step 2: A recursive solution: An example( Correction !!! ) Station S1,1 Station S1,2 Station S1,3 Station S1,4 Station S1,5 Station S1,6 24 Line 1 9 4 18 20 32 3 4 Chassis Enters 1 2 16 22 12 25 30 7 Line 2 Station S2,1 Station S2,2 Station S2,3 Station S2,4 Station S2,5 Station S2,6 j 1 2 3 4 5 6 j 2 3 4 5 6 f1[j] 9 18 20 24 32 1 2 1 1 l1[j] f*=0 l* =0 12 16 22 25 30 f2[j] 1 2 1 2 l2[j]

26. Step 2: A recursive solution: An example Station S1,1 Station S1,2 Station S1,3 Station S1,4 Station S1,5 Station S1,6 24 Line 1 9 35 18 20 20 32 3 Chassis Enters 2 16 22 22 12 25 30 37 Line 2 Station S2,1 Station S2,2 Station S2,3 Station S2,4 Station S2,5 Station S2,6 j 1 2 3 4 5 6 j 2 3 4 5 6 f1[j] 9 18 20 24 32 35 1 2 1 1 1 l1[j] f*=0 l* =0 12 16 22 25 30 37 f2[j] 1 2 1 2 2 l2[j]

27. Step 2: A recursive solution: An example Station S1,1 Station S1,2 Station S1,3 Station S1,4 Station S1,5 Station S1,6 24 Line 1 9 35 18 20 20 32 3 Chassis Enters 2 16 22 22 12 25 30 37 Line 2 Station S2,1 Station S2,2 Station S2,3 Station S2,4 Station S2,5 Station S2,6 j 1 2 3 4 5 6 j 2 3 4 5 6 f1[j] 9 18 20 24 32 35+3 1 2 1 1 2 l1[j] f*=38 l* =1 12 16 22 25 30 37+2 f2[j] 1 2 1 2 2 l2[j]

28. Step 2: A recursive solution (cont.) • fi[j] values give the values of optimal solutions to subproblems. • Let li[j] to be the line number 1 or 2, whose station j-1 is used in a fw throgh station Si, j. • We also define l* to be the line whose station nis used in a fw through the entire factory.

29. Step 2: A recursive solution (cont.) The fw through the entire factory:

30. Step 3: Computing the fastest times We can write a simple recursive algorithm based on the equations for f1[j] and f2[j] to compute the fw through the factory. However its running time is exponential. Let ri( j ) be the number of references made tofi [ j ] in a recursive algorithm:

31. Step 3: Computing the fastest times(cont.) • We observe that for each value of fi[j] depends only on the values of f1[j-1] and f2[j-1]. • By computing the fi[j] values in order of increasing station numbers we can compute the fw through the factory in time.

32. The fastest way procedure

33. The fastest way procedure (cont.) line 1, station 6 line 2, station 5 line 2, station 4 line 1, station 3 line 2, station 2 line 1, station 1

34. Matrix-Chain multiplication • We are given a sequence • And we wish to compute

35. Matrix-Chain multiplication (cont.) • Matrix multiplication is assosiative, and so all parenthesizations yield the same product. • For example, if the chain of matrices is then the product can be fully paranthesized in five distinct way:

36. Matrix-Chain multiplication MATRIX-MULTIPLY (A,B) if columns[A] ≠ rows[B] then error “incompatible dimensions” else for i←1 to rows[A] do for j←1 to columns[B] do C[i,j]←0 for k←1 to columns[A] do C[ i,j ]← C[i,j]+A[i,k]*B[k,j] return C

37. Matrix-Chain multiplication (cont.) Cost of the matrix multiplication: An example:

38. Matrix-Chain multiplication (cont.)

39. Matrix-Chain multiplication (cont.) • The problem: Given a chain of nmatrices, where matrix Ai has dimension pi-1xpi, fully paranthesize the product in a way that minimizes the number of scalar multiplications.

40. Matrix-Chain multiplication (cont.) • Counting the number of alternative paranthesization : bn

41. Matrix-Chain multiplication (cont.) • Based on the above defination we have

42. Matrix-Chain multiplication (cont.) • Substituting the initial value for b we obtain

43. Matrix-Chain multiplication (cont.) • Let B(x) be the generating function

44. Matrix-Chain multiplication (cont.) • Substituting in B(x) we can write that B(x)

45. Matrix-Chain multiplication (cont.) • Summing over the both sides

46. Matrix-Chain multiplication (cont.) • One way to express B(x) in closed form is Expanding into a Taylor series about x=0 we get Comparing the coefficients of x we finally show that

47. Matrix-Chain multiplication (cont.) Step 1: The structure of an optimal paranthesization(op) • Find the optimal substructure and then use it to construct an optimal solution to the problem from optimal solutions to subproblems. • Let Ai...jwhere i ≤ j, denote the matrix product Ai Ai+1 ... Aj • Anyparenthesization of Ai Ai+1 ... Aj must split the product between Ak andAk+1 fori ≤ k < j.

48. Matrix-Chain multiplication (cont.) The optimal substructure of the problem: • Suppose that an op of Ai Ai+1 ... Aj splits the productbetween Akand Ak+1 then the paranthesization of the subchain Ai Ai+1 ... Ak within this parantesization of Ai Ai+1 ... Aj must be an op of Ai Ai+1 ... Ak

49. Matrix-Chain multiplication (cont.) Step 2: A recursive solution: • Let m[i,j] be the minimum number of scalar multiplications needed to compute the matrix Ai...jwhere 1≤ i ≤ j ≤ n. • Thus, the cost of a cheapest way to compute A1...nwould be m[1,n]. • Assume that the op splits the product Ai...jbetween Akand Ak+1.wherei ≤ k <j. • Then m[i,j] =Theminimum cost for computing Ai...k and Ak+1...j+ the cost of multiplying these two matrices.

50. Matrix-Chain multiplication (cont.) Recursive defination for the minimum cost of paranthesization: