1.15k likes | 1.17k Views
Explore dynamic programming techniques, from recursive to matrix multiplication, for optimal Fibonacci number computation. Learn how to boost efficiency by minimizing redundant calculations.
E N D
Chapter 06 Dynamic Programming
Dynamic Programming (i.e., Dynamic Planning) • An algorithm design technique. • Invented by a US Mathematician, Richard Bellman in 1950. • A method for optimizing multistage decision process. • The word “programming” in “Dynamic programming” stands for “planning” and does not refer to computer programming.
Dynamic programming • A technique for solving problems with overlapping subproblems. • These subproblems arise from a recurrence relating a given problem’s solution to solutions of its smaller subproblems. • Instead solving overlapping subproblems again and again, The technique suggests solving each of the smaller subproblems only once and recording the results in a table from which a solution to the original problem can then be obtained.
Consider again the Fibonacci numbers. The Fibonacci numbers are the elements of the sequence 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233, … which can be defined by the simple recurrence F(n) = F(n – 1) + F(n – 2) for n > 1 … 8.1 with two initial conditions F(0) = 0, F(1) = 1 … 8.2 For computing the nth Fibonacci number F(n), using directly the recurrence (8.1), the same values of this function would have to recomputed many times. The following algorithm is constructed based on this definition.
Algorithm F(n) //Computes the nth Fibonacci number recursively by using definition //Input: A nonnegative integer n //Output: The nth Fibonacci number if n ≤ 1 return n else return F(n-1) + F(n-2); The efficiency of this algorithm can be computed according to: the recurrence relation, which is: A(n) = A(n-1) + A(n-2) + 1. A(1) = 0. A(0) = 0.
F(5) F(4) F(3) F(3) F(2) F(2) F(1) F(2) F(1) F(1) F(0) F(1) F(0) F(1) F(0) Algorithm F(n) … if n ≤ 1 return n else return F(n-1) + F(n-2); Figure 2.6 Tree of recursive calls for computing the 5th Fibonacci number by the definition-based algorithm.
The algorithm yields A(n) = Θ(Øn ), where A(n) is the number of additions performed by the algorithm in computing F(n) and Ø = (1 + )/2 ≈ 1.61803. That means, F(n) grows exponentially in n. If we measure the size of n by the number of bits b, (i.e., n = ), where b = floor(log2 n) + 1 in its binary representation, the efficiency class will be even worse, namely, doubly exponential: A(b) = Θ( ). 4 bits is
The problem of computing F(n) is expressed in terms of its smaller and overlapping subproblems of computing F(n – 1) and F(n – 2). • We can fill elements of a one-dimensional array with the n + 1 consecutive values of F(n) by • starting with 0 and 1 using initial conditions (8.2) F(0) = 0, F(1) = 1 and • producing all the other elements using equation (8.1) • F(n) = F(n – 1) + F(n – 2) for n > 1, as the rule. • Obviously, the last element of this array will contain F(n). • A single-loop faster algorithm can be used by computing successive elements of the Fibonacci sequence iteratively:
Polynomial Algorithm Fib(n) //Computes the nth Fibonacci number recursively by using definition //Input: A nonnegative integer n //Output: The nth Fibonacci number create an array F[0..n]; F[0] ← 0; F[1] ← 1; for i← 2 to n do F[i] ← F[i-1] + F[i-2] return F[n]. This algorithm clearly makes n-1 additions. Hence, it is linear as a function of n, A(n) = Θ(n), and it is “only” exponential as a function of the number of bits b in n’s binary representation. A(b) = Θ(2b ). Since b is log2n, then A(b) = Θ(2log n).
Furthermore, We can avoid using an extra array to accomplish this task by recording the values of just the last two elements of the Fibonacci sequence. – This Improve algorithm Fib requires only Ɵ(1) space. We can improve further the time efficiency for computing the Fibonacci sequence. We begin with the equations F1 = F1 and F2 = F0 + F1 in matrix notation.
F1 0 1 F0 F1 = F1 F2 1 1 F1 F2 = F0 + F1 F2 0 1 F1 F2 = F1 F3 1 1 F2 F3 = F1 + F2 0 1 2 F0 0 1 F0 1 1 F1 1 2 F1 F3 = F0 + 2F1 Fn0 1 n F0 F4 = 2F0 + 3F1 Fn+1 1 1 F1 Fn+1 = (n-1)F0 + nF1? = = Similarly, = = = = And in general =
For computing Fn , it suffices to raise this 2 x 2 matrix, (called it X), to the nth power. We can show: Two 2 x 2 matrices can be multiplied using 4 additions and 8 multiplications. We should be able to take O(log n) matrix multiplications for computing Xn. X2 takes 4 additions. X4 = X2X2 takes 2*4 additions. X8 = X2X2X2X2 takes 3*4 additions. X16 = X2X2X2X2X2X2X2X2 takes 4*4 additions. X32 = X16X16takes 5*4 additions. X64 = X32X32takes 6*4 additions X128 = X64X64takes 7*4 additions. Thus, the number of arithmetic operations needed by our matrix-based algorithm for computing Fibonacci numbers is just O(log n), as compared to O(n). 0 1 0 1 0*0 + 1*1 0*1 + 1*1 1 1 1 1 1*0 + 1+1 1*1 + 1*1 = *
The catch is that This new algorithm involves multiplication, not just addition; and multiplications of large numbers are slower than additions. In conclusion, whether this algorithm is faster than the previous algorithm for generating the Fibonacci numbers depends on whether we can multiply n-bit integers faster than O(n2 ). A divide-and-conquer algorithms for integer multiplication will take O(n1.59 ). Using Strassen’s algorithm we can do O() steps for n x n matrices. Can we do better? It turns out that even faster algorithms for multiplying numbers exist, based on another important divide-and conquer-algorithm: the fast Fourier transform. Can we bring down the running time to O(nlogn)?.
An application of dynamic programming can be interpreted as a special variety of space-for-time trade-off. A dynamic programming algorithm can sometimes be refined to avoid using extra space.
Certain algorithms compute the nth Fibonacci number without computing all the preceding elements of this sequence. • Based on the classic bottom-up dynamic programming approach, • an algorithm can be used to solve all smaller subproblems of a given problem. e.g., Algorithm F(n), A(n) = Θ(Øn ), • To avoid solving unnecessary subproblems, this technique exploits so-called memory functions and can be considered a top-down variation of dynamic programming. e.g., Algorithm Fib(n), A(n) = Θ(n ), • Whether one uses the classical bottom-up version of dynamic programming or its top-down variation, the crucial step in designing such as algorithm remains the same: deriving a recurrence relating a solution to the problem to solutions to its smaller subproblems. • The immediate availability of equation (8.1) for computing the nth Fibonacci number is one of the few exceptions to this rule. • F(n) = F(n – 1) + F(n – 2) for n > 1 … 8.1 • with two initial conditions • F(0) = 0, F(1) = 1 … 8.2
Since a majority of dynamic programming applications deal with optimization problems, we need • the principle of optimality: • an optimal solution to any instance of an optimization problem is composed of optimal solutions to its sub-instances. • Three Basic Examples • Let introduce dynamic programming via three typical examples: • Coin-row problem, • Change-making problem and • Coin-collecting problem.
Example 1:Coin-row problem • Given: A row of n coins whose values are some positive integers c1 , c2 , …, cn , not necessarily distinct. • Output: Pick up the maximum amount of money subject to the constraint that no two coins adjacent in the initial row can be picked up. • Solution: • Let F(n) be the maximum amount that can be picked up from the row of n coins. • To derive a recurrence for F(n), we partition all the allow coin selections into two groups: those that include the last(previous one) coin and those without it (the last coin). • …
Example 1:Coin-row problem [Page 285-286, Levitin] • Given: A row of n coins whose values are some positive integers c1 , c2 , …, cn , not necessarily distinct. • Output: Pick up the maximum amount of money subject to the constraint that no two coins adjacent in the initial row can be picked up. • Solution: • … • The largest amount we can get from the first group is equal to cn + F(n – 2) -- the value of the nth coin plus the maximum amount we can pick up from the first n-2 coins. • The maximum amount we can get from the second group is equal to F(n – 1) by the definition of F(n). • Thus, we have the following recurrence subject to the obvious initial conditions: • F(n) = max(cn + F(n – 2), F(n-1)) for n > 1 … (8.3) • F(0) = 0, F(1) = c1 .
We can compute F(n) by filling the one-row table left to right in the manner similar to the way it was done for the nth Fibonacci number by Algorithm Fib(n), i.e., Polynomial Algorithm Fib(n) .
Algorithm CoinRow( C[1 .. n] ) //Applies formula (8.3) bottom up to find the maximum amount //of money that can be picked up from a coin row without //picking two adjacent coins. Input: Array C[1 .. n] ) of positive integers indicating the coin values Output: The maximum amount of money that can be picked up F[0] ← 0; F[1] ← C[1]; fori← 2 to n do F[ i ] ← max ( C[ i ] + F[ i – 2 ], F[ i – 1 ] ); return F[ n ] T(n) = θ(n)
The application of the algorithm to the coin row of denominations 5, 1, 2, 10, 6, 2 is shown in Figure 8.1. It yields the maximum amount of 17. We solve the problem for the first i coins in the row given for every 1 ≤i≤ 6. For example, for i = 3, the maximum amount is F(3) = 7. Figure 8.1 Solving the coin-row problem by dynamic programming for the coin row 5, 1, 2, 10, 6, 2
F[0] = 0, F[1] = C[1] = c1 = 5 F[2] = max{ C(2) + F[0], F[1] } : take next one with one before the current. F[2] = max{ 1 + 0, 5 } = 5 (i.e., (C(2) + F(0), F(1)) = (C(2) + C(0), C(1)))
F[3] = max{ C(3) + F[1], F[2]}: take next one with one before the current. F[3] = max{ 2 + 5, 5 } = 7 (i.e., (C(3)+F(1), F(2)) = (C(3) + C(1), C(1) ) ) F[4] = max{ C(4) + F[2], F[3]} F[4] = max{ 10 + 5, 7 } = 15 (i.e., (C(4)+F(2), F(3))=(C(4)+C(1), C(3)+C(1)))
F[5] = max{C(5) + F[3], F[4]} F[5] = max{ 6+7, 15 } = 15 (i.e.,(C(5)+F(3),F(4))=(C(5)+C(3)+C(1), C(4)+C(1))) F[6] = max{C(6) + F[4], F[5]} F[6] = max{ 2 + 15, 15 } = 17 (i.e., (C(6)+F(4), F(5))=(C(6)+C(4)+C(1), C(4)+C(1)))
To find the coins with the maximum total value found, we need to back-trace the computations to see which of the two possibilities -- cn + F(n – 2) or F(n – 1) -- produced the maxima in formula (8.3). • In the last application of the formula, it was the sum c6 + F(4), which means that the coin c6 = 2 is a part of an optimal solution. • F(n) = max(cn + F(n – 2), F(n-1)) for n > 1 … (8.3) • F(0) = 0, F(1) = c1 . F[6] = max{C(6) + F[4], F[5]} F[6] = max{ 2 + 15, 15 } = 17 (i.e., (C(6)+F(4), F(5))=(C(6)+C(4)+C(1), C(4)+C(1)))
Moving to compute F(4), the maximum was produced by the sum c4 + F(2), which means that the coin c4 = 10 is also a part of an optimal solution. F[4] = max{ C(4) + F[2], F[3]} F[4] = max{ 10 + 5, 7 } = 15 (i.e., (C(4)+F(2), F(3))=(C(4)+C(1), C(3)+C(1))) F(n) = max(cn + F(n – 2), F(n-1)) for n > 1 … (8.3) F(0) = 0, F(1) = c1 .
Finally, the maximum in computing F(2) was produced by F(1), implying that the coin c2 is not the part of an optimal solution and the coin c1 = 5 is. F[2] = max{ C(2) + F[0], F[1] } : take next one with one before the current. F[2] = max{ 1 + 0, 5 } = 5 (i.e., (C(2) + F(0), F(1)) = (C(2) + C(0), C(1))) F(n) = max(cn + F(n – 2), F(n-1)) for n > 1 … (8.3) F(0) = 0, F(1) = c1 .
Thus, the optimal solution is [c1 , c4 , c6 ]. • To avoid repeating the same computations during the backtracking, the information about which of the two terms in (8.3) was larger can be recorded in an extra array which the values of F are computed. • F(n) = max(cn + F(n – 2), F(n-1)) for n > 1 … (8.3) • F(0) = 0, F(1) = c1 .
Using the CoinRow to find F(n), the largest amount of money that can be picked up, as well as the coins composing an optimal set, clearly take ϴ(n) time and ϴ(n) space. This is by far superior to the alternatives: the straightforward top-down application of recurrence (8.3) (which is exponential) F(n) = max(cn + F(n – 2), F(n-1)) for n > 1 … (8.3) F(0) = 0, F(1) = c1 . and solving the problem by exhaustive search (which is at least exponential).
This is by far superior to the alternatives: the straightforward top-down application of recurrence (8.3) (which is exponential) F[6] c[6] + F[4] F[5] c[4] + F[2] F[3] c[5] + F[3] F[4] c[2] + F[0] F[1] c[3] + F[1] F[2] c[3] + F[1] F[2] c[4] + F[2] F[3] c[0] c[1] c[1] c[2] + F[0] F[1] c[1] c[2] + F[0] F[1] c[3] + F[1] F[2] c[1] c[1] c[2] + F[0] F[1]
This is by far superior to the alternatives: the straightforward top-down application of recurrence (8.3) (which is exponential) and solving the problem by exhaustive search (which is at least exponential).
Example 2: Change-making problem. Consider the general instance of the following well-known problem. Given change for amount n using the minimum number of coins denominations d1 < d2 < … < dm, assume that the availability of unlimited quantities of coins for each of the m denominations d1 < d2 < … < dm where d1 = 1. Consider a dynamic programming algorithm for the general case: Let F(n) be the minimum number of coins whose values add up to n. Let define F(0) = 0. The amount n can only be obtained by adding one coin of denomination dj to the amount n – dj for j = 1, 2, …, m such that n ≥ dj .
Therefore we can consider all such denominations and select the one minimizing F(n – dj ) + 1. Since 1 is a constant, we can find the smallest F(n – dj ) first and then add 1 to it. Hence, we have the following recurrence for F(n): F(n) = minj: n ≥ dj { F(n – dj ) } + 1 for n > 0 …………… 8.4 F(0) = 0. We can compute F(n) by filling a one-row table left to right in the manner similar to the way it was done above for the coin-row problem, but computing a table entry here requires finding the minimum of up to m numbers d1 < d2 < … < dm.
Algorithm ChangeMaking ( D[ 1.. m], n) //Applies dynamic programming to find the minimum number of //coins of denominations d1 < d2 < … < dm where d1 = 1 that add up //to a given amount n. Input: Positive integer n and array D[1 .. m] of increase positive integers indicating the coin denominators where D[1] = 1 Output: The minimum number of coins that add up to n F[0] ← 0 fori← 1 tondo { //add up to n. temp ←∞ ; j ← 1; while j ≤mandi≥ D[j] do { //1 ≤ j ≤ m and if the coin D(j) ≤ n=i temp ← min(F[i – D[j]], temp);//what will make up F[i-D(1)], j ← j + 1 } //end while //F[i-D(2)], … F[i-D(k)]. F[i] ← tempt + 1; } //end for return F[n] T(m,n) = O(n m)
The application of the algorithm to amount n = 6 and denominations 1, 3, 4 is shown in Figure 8.2. The answer it yields is two coins. The time and space efficiencies of the algorithm are obvious O(n m) and ϴ(n), respectively.
Given D[1] = 1, D[2] = 3 and D[3] = 4, and total amount n = 6, the best solution is 2*D[2] = 2*3 = 6. That is, we can use minimal number of two 3-coins to make up 6. But the other possible solutions, but not the best solutions, could be: 6*D[1] = 6*1 = 6; //We use six 1-coins to make up 6. 3*D[1] + 1*D[2] = 3*1 + 1*3 = 6; //we use four coins to make up 6; 2*D[1] + 1*D[3] = 2*1 + 1*4 = 6. //Use three coins to make up 6.
Algorithm ChangeMaking ( D[ 1.. m], n) = Algorithm ChangeMaking(D[1..m=3]{=1, 3, 4}, n=6) m = 3 and n =6 F[0] ← 0 F[0] = 0 F[0] = 0 Given D[1] = 1, D[2] = 3 and D[3] = 4, and if total amount n = 0 to begin with, n has to go up to 6.
fori← 1 to n=6 do { i =1 temp ←∞ ; j ← 1 temp = ∞ j = 1 while j ≤ m=3 andi≥ D[j] do { j=1 ≤m=3 and i=1 ≥ D[1]=1 temp ← min(F[i – D[j]], temp); temp = min(F[1-D[1]], ∞) = min(F[1-1],∞) = F[0]=0 j ← j + 1 } //end while j = 1+1=2 j=2 ≤ m=3 and i=1 ≥D[2]=3 F[i] ← tempt + 1; } //end for F[1] = F[1-1]+1 =F[0]+1 = 0+1 Need D[1]=1 to get total n=1 F[1] = min { F[1 - 1]} + 1 = 1 Given D[1] = 1, D[2] = 3 and D[3] = 4, and if total amount n = 1, then after pick up one 1-coin, the balance would be 1-1 = 0. Then how many coins need to make up F[0]
fori← 1 to n do { i =2 temp ←∞ ; j ← 1 temp = ∞ j = 1 while j ≤ m andi≥ D[j] do { j=1 ≤m=3 and i=2 ≥D[1]=1 temp ← min(F[i – D[j]], temp); temp = min(F[2-D[1]], ∞) = min(F[2-1], ∞) = F[1]=1 j ← j + 1 } //end while j=1+1 = 2 j=2 ≤ m=3 and i=2 ≥D[2]=3 F[i] ← tempt + 1; } //end for F[2] =F[2-1]+1=F[1]+1=1+1=2 Need D[1] + D[1] to get total n = 2 F[2] = min { F[2 - 1]} + 1 = 2 Given D[1] = 1, D[2] = 3 and D[3] = 4, and if total amount n = 2, then after using one 1-coin, the balance is F[2-1] = F[1].
fori← 1 to n=6 do { i =3 temp ←∞ ; j ← 1 temp = ∞ j = 1 while j ≤ m=3 andi≥ D[j] do { j=1 ≤ m=3 andi=3 ≥D[1]=1 temp ← min(F[i – D[j]], temp); temp = min(F[3-D[1]], ∞) = min(F[3-1],∞) = F[2]=2 j ← j + 1 } //end while j = 1+1=2 j=2 ≤m=3 andi=3 ≥D[2]=3 temp = min(F[3-D[2]], 2=F[3-1]) = min(F[3-3], 2=F[3-1]) = F[0]=0 j = 2+1=3 j=3 ≤ m=3 and i=3 ≥D[3]=4 F[i] ← tempt + 1; } //end for F[3] = min(F[3-3], 2=F[3-1]) + 1 = F[3-3]+1=F[0]+1=0+1 =1 Need D[3]=3 to get total n = 3 Given D[1] = 1, D[2] = 3 and D[3] = 4, and if total amount n = 3, then use one 1-coin or one 3-coins the balance would be 2 and 0. What is the min number of coins to make up 2 and 0. State the min number. F[3] = min { F[3 - 1], F[3 – 3]} + 1 = 1
fori← 1 to n=6 do { i =4 temp ←∞ ; j ← 1 temp = ∞ j = 1 while j ≤ m=3 andi≥ D[j] do { j=1 ≤ m=3 and i=4 ≥D[1]=1 temp ← min(F[i – D[j]], temp); temp = min(F[4-D[1]], ∞) = min(F[4-1],∞) = F[3]=1 j ← j + 1 } //end while j = 1+1=2 j=2 ≤ m=3 and i=4 ≥D[2]=3 temp = min(F[4-D[2]], 1=F[4-1]) = min(F[4-3], 1=F[4-1]) = min(F[1]=1, 1=F[3]) = F[1]= F[3] =1 j=2+1=3 j=3 ≤ m=3 and i=4 ≥D[3]=4 temp = min(F[4-D[3]], min(F[4-3], F[4-1])) = min(F[4-4], F[4-3], F[4-1]) = min(F[0]=0, 1=F[1]=1, F[3]=1) = 0 j=3+1=4 j=4 ≤ m=3 and i=4 ≥D[4]? F[i] ← tempt + 1; } //end for F[4] = min(F[4-4], F[4-3], F[4-1])+1 =F[0]+1=0+1 =1 Need D[4] = 4 to get total n = 4
F[4] = min { F[4 - 1], F[4 – 3], F[4 – 4] } + 1 = 1 Given D[1] = 1, D[2] = 3 and D[3] = 4, and if total amount n = 4, use one 1-coins, one 3-coins or one 4-coins currently, then the balance would be 3, 1 and 0. What is the number of coins F[3], F[1] and F[0] to make up 3, 1, and 0? What is the min number of these?
fori← 1 to n=6 do { i =5 temp ←∞ ; j ← 1 temp = ∞ j = 1 while j ≤ m=3 andi≥ D[j] do { j=1 ≤ m=3 and i=5 ≥D[1]=1 temp ← min(F[i – D[j]], temp); temp = min(F[5-D[1]], ∞) = min(F[5-1],∞) = F[4]=1 j ← j + 1 } //end while j = 1+1=2 j=2 ≤ m=3 and i=5 ≥D[2]=3 temp = min(F[5-D[2]], 1=F[5-1]) = min(F[5-3], 1=F[5-1]) = min(F[2]=2, 1=F[4]) = F[4]=1 j=2+1=3 j=3 ≤ m=3 and i=5 ≥D[3]=4 temp = min(F[5-D[3]], min(F[5-3], F[5-1])) = min(F[5-4], F[5-3], F[5-1]) = min(F[1]=1, 2=F[2], F[4]=1) = 1 j=3+1=4 j=4 ≤ m=3 and i=5 ≥D[4]=? F[i] ← tempt + 1; } //end for F[5] = min(F[5-4], F[5-3], F[5-1])+1 =F[1]+1=1+1 =2 Need D[1]=1 + D[4]=4 to get n=5
fori← 1 to n=6 do { i =6 temp ←∞ ; j ← 1 temp = ∞ j = 1 while j ≤ m=3 andi≥ D[j] do { j=1 ≤ m=3 and i=6 ≥D[1]=1 temp ← min(F[i – D[j]], temp); temp = min(F[6-D[1]], ∞) = min(F[6-1],∞) = F[5]=2 j ← j + 1 } //end while j = 1+1=2 j=2 ≤ m=3 and i=6 ≥D[2]=3 temp = min(F[6-D[2]], 2=F[6-1]) = min(F[6-3], 2=F[6-1]) = min(F[3]=1, 2=F[5]) = F[3]=1 j=2+1=3 j=3 ≤ m=3 and i=6 ≥D[3]=4 temp = min(F[6-D[3]], min(F[6-3], F[6-1])) = min(F[6-4], F[6-3], F[6-1]) = min(F[2]=2, 1=F[3], F[5]=2) = F[3] =1 j=3+1=4 j=4 ≤ m=3 and i=6 ≥D[4]=? F[i] ← tempt + 1; } //end for F[6] = min(F[6-4], F[6-3], F[6-1])+1 =F[3]+1=1+1 =2 Need D[3]=3 + D[3] to get n = 6.
fori← 1 to n=6 do { i =7 … return F[n] return F[6] = 2
F[0] = 0 F[3] = min { F[3 - 1], F[3 – 3]} + 1 = 1 F[1] = min { F[1 - 1]} + 1 = 1 F[4] = min {F[4 - 1], F[4 – 3], F[4 – 4]} + 1 = 1 F[2] = min { F[2 - 1]} + 1 = 2 F[5] = min {F[5 - 1], F[5 – 3], F[5 – 4]} + 1 = 2
F[6] = min { F[6 - 1], F[6 – 3], F[6 – 4] } + 1 = 2 Figure 8.2 Application of Algorithm MinCoinChange to amount n = 6 and coin denominations 1, 3, 4.
F[6] = min { F[6 - 1], F[6 – 3], F[6 – 4] } + 1 = 2 Figure 8.2 Application of Algorithm MinCoinChange to amount n = 6 and coin denominations 1, 3, 4. To find the coins of an optimal solution, we need to backtrace the computations to see which of the denominations produced the minima in formula (8.4). For the instance considered, the last application of the formula (for n = 6), the minimum was produced by d2 = 3. The second minimum (for n = 6 – 3) was also produced for a coin of that denomination. Thus, the minimum-coin set for n = 6 is two 3’s. F[6] = min(F[6-4], F[6-3], F[6-1])+1 =F[3]+1=1+1 =2 That is, we need D[3]=3 + D[3]=3 to get total n=6.