Dynamic Programming

Dynamic Programming Lets begin by looking at the Fibonacci sequence.

The Fibonacci sequence is defined by the recursive formula Fib(x) = Fib(x-1) + Fib(x-2) However, actually using recursion to calculate Fibonacci numbers is extremely inefficient. Does anyone remember why? Consider what happens if we attempt to calculate Fib(100) using recursion. We get a binary tree with leaf nodes at a minimum depth of 100/2 and a maximum depth of 100. More than 250 calculations will be needed.

Fib(100) Fib(100) Fib(99) Fib(98) Fib(96) Fib(97) Fib(97) Fib(98) 48 more levels 97 more levels Fib(0) Fib(0) Fib(1) Fib(0) Fib(1) Fib(1)

Clearly, the recursive approach gives us an Ω(2n/2) algorithm. It is much faster to use the recursive formula to build from the bottom up. Fib(0) = 1, Fib(1) = 1, Fib(2) = 2, Fib(3) = 3, Fib(4) = 5, Fib(5) = 8 etc. up to Fib(100) This approach gives us an Θ(n) algorithm. If building a table of Fibonacci numbers instead of just one, the algorithm is Θ(c), since each new number only requires two additions.

The Fibonacci sequence is NOT a dynamic programming problem, since it does not involve optimization. However, it does illustrate an important point: Sometimes it is much faster to use a recursive formula to work from the bottom up, building a table of values as you go. Each new value depends on the values that were calculated previously. This is similar to the dynamic programming approach.

Colonel Motors Problemfrom Cormen Leiserson Rivest Stein Second Edition Chapter 15 pages 324 to 330

Problem: Two assembly lines both produce the same cars, but take different times at each station. Partially completed cars can move to the next station on the same line, or cross over to the next station on other line. Crossing over takes extra time. Problem is to find the fastest path through the entire line. If there are n stations and 2 lines, how many different ways can we select a path? Brute force method is an exponential problem since there are 2n possible ways to select stations. We will use dynamic programming.

First step is to define the structure of an optimal solution. Suppose we have taken the fastest path through station S1, j There are only 2 ways we could have arrived there, from S1, j-1 or by crossing over from S2, j-1 If the fastest path was from S1, j-1 then the car must have taken the fastest path from the start to S1, j-1 If there was a faster path to S1, j-1 we could substitute this faster path to get a faster path to S1, j as well. But this is a contradiction since we assumed we already had the fastest path to S1, j.

We can use similar reasoning for the path crossing over from S2, j-1 This property is called optimal substructure, and it is necessary for a problem to have this property for a dynamic programming solution to be applicable. The next step is to define the recursive formula. Denote fastest path as f*. Denote fi, j as the fastest time to get from the start through station Si, j Recursive formula (on board) Now fill the total cost array using the recursive formulas.

F1, 1 = (2+7) = 9 from base case F2, 1 = (4+8) = 12 from base case

F1, 2 = min(9 + 9, 12 + 2 + 9) = 18 F2, 2 = min(12 + 5, 9 + 2 + 5) = 16

Continue in this fashion until the entire table is filled. The last column just adds the exit times. Are we done? No. We still do not know the optimum selection of assembly stations. With some problems (e.g. edit distance) we only need the optimum number, not the sequence of steps need to achieve it. Here, we need the sequence.

Can anyone suggest a method of retrieving the station sequence from the total cost array? There are two ways. One – Maintain a system of pointers that tells us where each optimum was obtained from. For example use an up arrow ↑ to indicate the value was obtained by moving from the previous station on the same line, and a right arrow → to indicate is was obtained by crossing over from the other line.

Trace back using arrows to give shaded path.

The other alternative is to trace back through the total cost array using the recursive formula.

For example, f1, 6 = min(32 + 4, 30 + 1 + 4) 35 = min(36, 35) So the vaue 35 was obtained by crossing from the other line. Repeat this process to the start. • Dynamic programming steps: • Verify the problem has optimal substructure. • Define the recursive formula. • Use the recusive formula to build a table of partial solutions from the bottom up. • If required, trace back to obtain the sequence to steps comprising the optimal solution.

Matrix Chain Multiplication Review of matrix multiplication Matrix multiplication is not commutative. AB ≠ BA Matrix multiplication is associative. So that ABC = (AB)C = A(BC) For AB to be defined the number of colums in A must equal the number of rows in B. If A is pxq and B is qxr AB will be pxr.

If A is pxq and B is qxr then C = AB is given by the following formula: • C has pr elements and each element takes q multiplications to compute so matrix multiplicaiton is θ(pqr).

Consider the multiplication ABC where A is 10x100, B is 100x5 and C is 5x50. There are two possible parenthesizations. • ((AB)C) first = 10*100*5 = 5000 multiplications. Result is a 10x5 matrix. Second = 10*5*50 = 2500 multiplications. Total = 7500. • (A(BC)) first = 100*5*50 = 25,000 multiplications. Result is a 100x50 matrix. Second = 10*100*50 = 50,000 multiplications. Total = 75,000 • Brute force is a poor strategy since it is Ω(4n / n3/2 ).

Step 1. Demonstrate that the problem has optimal substructure. Suppose (A1....Ak) (Ak+1....An) is an optimum parenthesization of A1....An for some k. Then (A1....Ak) and (Ak+1....An) must also be optimal. (Use the normal proof by contradiction here.) Step 2. Define a recursive solution. Let m[i][j] be the minimum number of multiplications needed to compute Ai....Aj where i<j. The cost of the best overall solution is A1....An If i=j then m[i][j] = 0 since there is only 1 matrix.

Suppose we are looking for the optimum way to parenthesize Ai....Aj We will place the parenthesis between Ak and Ak+1 where i ≤ k < j. The trick is to find the best value for k to minimize m[i][j]. m[i][j] = 0 for i = j = Min( m[i][k] + m[k+1][j] + Pi-1* Pk * Pj) for i ≤ k < j where Pi-1* Pk * Pj is the cost of multiplying the resulting matrices of size Pi-1x Pk and Pk x Pj Here Pi refers to the number of columns in Ai (or the number of rows in Ai+1)

It is important to note that when we use the recursive formula we must already have calculated m[i][k] and m[k+1][j]. Thus we should fill the cost array in increasing order of the length of the chain. Thus we fill in this order: • m[1][2], m[2][3], m[3][4]... all chains of length 2 • m[1][3], m[2][4], m[3][5]....all chains of length 3 • m[1][4], m[2][5], m[3][6]....all chains of length 4 • … • m[1][n] the only chain of length n

Fill this diagonal first j = 1 to n final result m[1][n] i = 1 to n

Example with 4 matrices A1 = 5x4, A2= 4x6, A3= 6x2, A4 = 2x7. This makes P0 = 5, P1 = 4, P2 = 6, P3 = 2, P4 = 7. J =1 2 3 4 i =1 2 3 4

M[1][2] = Min( m[1][k] + m[k+1][2] + P0* Pk* P2) for 1 ≤ k < 2 = Min( m[1][1] + m[2][2] + P0* P1* P2) for k = 1 = 0 + 0 + 5*4*6 = 120 J = 1 3 4 2 i =1 2 3 4

M[1][3] = Min( m[1][k] + m[k+1][3] + P0* Pk* P3) for 1 ≤ k < 3 = Min ( m[1][1] + m[2][3] + P0* P1* P3) where k = 1 ( m[1][2] + m[3][3] + P0* P2* P3) where k = 2 = Min ( 0 + 48 + 5*4*2), ( 120 + 0 + 5*6*2) = 88 J = 1 3 4 2 i =1 2 3 4

M[2][4] = Min( m[2][k] + m[k+1][4] + P1* Pk* P4) for 2 ≤ k < 4 = Min ( m[2][2] + m[3][4] + P1* P2* P4) where k = 2 ( m[2][3] + m[4][4] + P1* P3* P4) where k = 3 = Min ( 0 + 84 + 4*6*7), ( 48 + 0 + 4*2*7) = 104 J = 1 3 4 2 i =1 2 3 4

M[1][4] = Min( m[1][k] + m[k+1][4] + P0* Pk* P4) for 1 ≤ k < 4 = Min ( m[1][1] + m[2][4] + P0* P1* P4) where k = 1 ( m[1][2] + m[3][4] + P0* P2* P4) where k = 2 ( m[1][3] + m[4][4] + P0* P3* P4) where k = 3 = Min ( 0 + 104 + 5*4*7), ( 120 + 84 + 5*6*7), ( 88 + 0 + 5*2*7) = 158 J = 1 3 4 2 i =1 2 3 4

Are we done? No. Just like the Colonel Motors problem, we have the lowest cost, but we don't know the sequence needed to achieve it. The best way is to maintain a separate array s[n][n] that stores the k value that was used to achieve the optimal splitting in the total cost array. The correct placement of parenthesis can then be recovered with the following recursive algorithm.

PrintBrackets( i, j) if i = j then print “Ai” else print “(“ PrintBrackets( i, s[i][j]) PrintBrackets( s[i][j] + 1, j) print “)”

Dynamic Programming