1 / 43

Dynamic Programming: Design and Analysis of Algorithms

This reading assignment explains the concept of dynamic programming and its application to various optimization problems. It covers topics such as Fibonacci numbers, computing binomial coefficients, longest common subsequence problem, and matrix chain multiplication.

daviddjohnson
Download Presentation

Dynamic Programming: Design and Analysis of Algorithms

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. King Fahd University of Petroleum & Minerals Information & Computer Science Department ICS 353: Design and Analysis of Algorithms Dynamic Programming

  2. Reading Assignment • M. Alsuwaiyel, Introduction to Algorithms: Design Techniques and Analysis, World Scientific Publishing Co., Inc. 1999. • Chapter 7

  3. Dynamic Programming • Dynamic Programming algorithms address problems whose solution is recursive in nature, but has the following property: The direct implementation of the recursive solution results in identical recursive calls that are executed more than once. • Dynamic programming implements such algorithms by evaluating the recurrence in a bottom-up manner, saving intermediate results that are later used in computing the desired solution

  4. Fibonacci Numbers • What is the recursive algorithm that computes Fibonacci numbers? What is its time complexity? • Note that it can be shown that

  5. Computing the Binomial Coefficient • Recursive Definition • Actual Value

  6. Computing the Binomial Coefficient • What is the direct recursive algorithm for computing the binomial coefficient? How much does it cost? • Note that

  7. Optimization Problems and Dynamic Programming • Optimization problems with certain properties make another class of problems that can be solved more efficiently using dynamic programming. • Development of a dynamic programming solution to an optimization problem involves four steps • Characterize the structure of an optimal solution • Optimal substructures, where an optimal solution consists of sub-solutions that are optimal. • Overlapping sub-problems where the space of sub-problems is small in the sense that the algorithm solves the same sub-problems over and over rather than generating new sub-problems. • Recursively define the value of an optimal solution. • Compute the value of an optimal solution in a bottom-up manner. • Construct an optimal solution from the computed optimal value.

  8. Longest Common Subsequence Problem • Problem Definition: Given two strings A and B over alphabet , determine the length of the longest subsequence that is common in A and B. • A subsequence of A=a1a2…an is a string of the form ai1ai2…aik where 1i1<i2<…<ik n • Example: Let  = { x , y , z }, A = xyxyxxzy, B=yxyyzxy, and C= zzyyxyz • LCS(A,B)=yxyzy Hence the length = • LCS(B,C)= Hence the length = • LCS(A,C)= Hence the length =

  9. Straight-Forward Solution • Brute-force search • How many subsequences exist in a string of length n? • How much time needed to check a string whether it is a subsequence of another string of length m? • What is the time complexity of the brute-force search algorithm of finding the length of the longest common subsequence of two strings of sizes n and m?

  10. Dynamic Programming Solution • Let L[i,j] denote the length of the longest common subsequence of a1a2…ai and b1b2…bj, which are substrings of A and B of lengths n and m, respectively. Then L[i,j] = when i = 0 or j = 0 L[i,j] = when i > 0, j > 0, ai=bj L[i,j] = when i > 0, j > 0, aibj

  11. LCS Algorithm Algorithm LCS(A,B) Input: A and B strings of length n and m respectively Output: Length of longest common subsequence of A and B Initialize L[i,0] and L[0,j] to zero; for i ← 1 to n do for j ← 1 to m do if ai = bj then L[i,j] ← 1 + L[i-1,j-1] else L[i,j] ← max(L[i-1,j],L[i,j-1]) end if end for; end for; return L[n,m];

  12. Example (Q7.5 pp. 220) • Find the length of the longest common subsequence of A=xzyzzyx and B=zxyyzxz

  13. Example (Cont.)

  14. Example (Cont.)

  15. Complexity Analysis of LCS Algorithm • What is the time and space complexity of the algorithm?

  16. Matrix Chain Multiplication • Assume Matrices A, B, and C have dimensions 210, 102, and 210 respectively. The number of scalar multiplications using the standard Matrix multiplication algorithm for • (A B) C is • A (B C) is • Problem Statement: Find the order of multiplying n matrices in which the number of scalar multiplications is minimum.

  17. Straight-Forward Solution • Again, let us consider the brute-force method. We need to compute the number of different ways that we can parenthesize the product of n matrices. • e.g. how many different orderings do we have for the product of four matrices? • Let f(n) denote the number of ways to parenthesize the product M1, M2, …, Mn. • (M1M2…Mk) (M k+1M k+2…Mn) • What is f(2), f(3) and f(1)?

  18. Catalan Numbers • Cn=f(n+1) • Using Stirling’s Formula, it can be shown that f(n) is approximately

  19. Cost of Brute Force Method • How many possibilities do we have for parenthesizing n matrices? • How much does it cost to find the number of scalar multiplications for one parenthesized expression? • Therefore, the total cost is

  20. The Recursive Solution • Since the number of columns of each matrix Mi is equal to the number of rows of Mi+1, we only need to specify the number of rows of all the matrices, plus the number of columns of the last matrix, r1, r2, …, rn+1 respectively. • Let the cost of multiplying the chain Mi…Mj (denoted by Mi,j) be C[i,j] • If k is an index between i+1 and j, what is the cost of multiplying Mi,j considering multiplying Mi,k-1 with Mk,j? • Therefore, C[1,n]=

  21. The Dynamic Programming Algorithm

  22. Example (Q7.11 pp. 221-222) • Given as input 2 , 3 , 6 , 4 , 2 , 7 compute the minimum number of scalar multiplications:

  23. Example (Q7.11 pp. 221-222)

  24. Example (Q7.11 pp. 221-222)

  25. Another Example (Activity Sheet) • Given as input 5 , 2 , 3 , 6 , 4 , 2, 4 • Compute the minimum number of scalar multiplications. • Find the optimal parenthesization

  26. MatChain Algorithm Algorithm MatChain Input: r[1..n+1] of +ve integers corresponding to the dimensions of a chain of matrices Output: Least number of scalar multiplications required to multiply the n matrices for i := 1 to n do C[i,i] := 0; // diagonal d0 for d := 1 to n-1 do // for diagonals d1 to dn-1 for i := 1 to n-d do j := i+d; C[i,j] := ; for k := i+1 to j do C[i,j] := min{C[i,j],C[i,k-1]+C[k,j]+r[i]r[k]r[j+1]; end for; end for; return C[1,n];

  27. Time and Space Complexity of MatChain Algorithm • Time Complexity • Space Complexity

  28. All-Pairs Shortest Paths • Problem: For every vertex u, v V, calculate (u, v). • Possible Solution 1: • Cost of Possible Solution 1:

  29. Dynamic Programming Solution • Define a k-path from u to v, where u,v  {1 , 2 , … , n} to be any path whose intermediate vertices all have indices less than or equal to k. • What is a 0-path? • What is a 1-path? • … • What is an n-path?

  30. Floyd’s Algorithm Algorithm Floyd Input: An n  n matrix length[1..n, 1..n] such that length[i,j] is the weight of the edge (i,j) in a directed graph G = ({1,2,…,n}, E) Output: A matrix D with D[i,j] = [i,j] 1 D = length;//copy the input matrix length into D 2for k =1 to n do 3for i = 1 to n do 4for j =1 to ndo 5 D[i,j] = min{D[i,j] , D[i,k] + D[k,j]}

  31. Example 2 11 5 4 12 15 8 2 4 1 11 1 3 5

  32. Example (Cont.)

  33. Example (Cont.)

  34. Example (Cont.)

  35. Time and Space Complexity • Time Complexity: • Space Complexity:

  36. The Knapsack Problem • Let U = {u1, u2, …, un} be a set of n items to be packed in a knapsack of size C. • Let sj and vj be the size and value of the jth item, where sj, vj, 1  j  n. • The objective is to fill the knapsack with some items from U whose total size does not exceed C and whose total value is maximum. • Assume that the size of each item does not exceed C.

  37. The Knapsack Problem Formulation • Given n +ve integers in U, we want to find a subset SU s.t. is maximized subject to the constraint

  38. Inductive Solution • Let V[i,j] denote the value obtained by filling a knapsack of size j with items taken from the first i items {u1, u2, …, ui} in an optimal way: • The range of i is • The range of j is • The objective is to find V[ , ] • V[i,0] = V[0,j] = • V[i,j] = V[i-1,j] if = max {V[i-1,j], V[ , ]+vi} if

  39. Example (pp. 223 Question 7.22) • There are five items of sizes 3, 5, 7, 8, and 9 with values 4, 6, 7, 9, and 10 respectively. The size of the knapsack is 22.

  40. Example (pp. 223 Question 7.22) • There are five items of sizes 3, 5, 7, 8, and 9 with values 4, 6, 7, 9, and 10 respectively. The size of the knapsack is 22.

  41. Algorithm Knapsack Algorithm Knapsack Input: A set of items U = {u1,u2,…,un} with sizes s1,s2,…,sn and values v1,v2,…,vn, respectively and knapsack capacity C. Output: the maximum value of subject to for i := 0 to n do V[i,0] := 0; for j := 0 to C do V[0,j] := 0; for i := 1 to n do for j := 1 to C do V[i,j] := V[i-1,j]; if si j then V[i,j] := max{V[i,j], V[i-1,j-si]+vi} end for; end for; return V[n,C];

  42. Time and Space Complexity of the Knapsack Algorithm

More Related