Chapter 15: Dynamic Programming III

1. Chapter 15: Dynamic Programming III

2. Subsequence of a String • Let S = s1s2…sm be a string of length m • Any string of the form • si1 si2 … sik • with i1i2 … ik is a subsequence of S • E.g., if S = farmers • fame, arm, mrs, farmers, are some of the subsequences of S

3. Longest Common Subsequence • Let S and T be two strings • If a string is both • a subsequence of S and • a subsequence of T, • it is a common subsequence of S and T • In addition, if it is the longest possible one, it is alongest common subsequence

4. Longest Common Subsequence • E.g., • S = algorithms • T = logarithms • Then, aim, lots, ohms, grit, are some of the common subsequences of S and T • Longest common subsequences: • lorithms , lgrithms

5. Longest Common Subsequence • Let S = s1s2…sm be a string of length m • Let T = t1t2…tn be a string of length n • Can we quickly find a longest common subsequence (LCS) of S and T ?

6. Optimal Substructure • Let X = x1x2…xk be an LCS of S1,i = s1 s2…si and T1,j=t1t2…tj. • Lemma: • If si = tj, then xk = si = tj, and x1x2…xk-1 must be the LCS of S1,i-1 and T1,j-1 • If sitj, then Xmust either be • (i) an LCS of S1,i and T1,j-1 , or • (ii) an LCS of S1,i-1 and T1,j

7. Optimal Substructure Let leni,j = length of the LCS of S1,i and T1,j • Lemma: For any i, j1, • if si = tj, leni,j = leni-1,j-1 + 1 • if sitj,leni,j=max {leni,j-1 , leni-1,j }

8. Length of LCS Define a function Compute_L(i,j) as follows: Compute_L(i, j)/* Finding leni,j */ 1. if (i == 0 or j == 0 ) return 0; /* base case */ 2. if (si==tj) return Compute_L(i-1,j-1) + 1; 3. else return max {Compute_L(i-1,j), Compute_L(i,j-1)}; Compute_L(m, n) runs in O(2m+n) time

9. Overlapping Subproblems To speed up, we can see that : To Compute_L(i,j-1) and Compute_L(i-1,j), has a common subproblem: Compute_L(i-1,j-1) In fact, in our recursive algorithm, there are many redundant computations ! Question: Can we avoid it ?

10. Bottom-Up Approach • Let us create a 2D table L to store all leni,j values once they are computed BottomUp_L() /* Finding min #operations */ 1. For all i and j, set L[i,0] = L[0, j] = 0; 2. for (i = 1,2,…, m) Compute L[i,j] for all j; // Based on L[i-1,j-1], L[i-1,j], L[i,j-1] 3. return L[m,n] ; Running Time = Q(mn)

11. Example Run: After Step 1

12. Example Run: After Step 2, i = 1

13. Example Run: After Step 2, i = 2

14. Example Run: After Step 2, i = 3

15. Example Run: After Step 2, i = 4

16. Example Run: After Step 2

17. Extra information to obtain an LCS

18. Extra Info: After Step 2, i = 2

19. Extra Info: After Step 2, i = 3

20. Extra Info: After Step 2

21. LCS obtained by tracing from L[m,n]

22. Computing the length of an LCS LCS_LENGTH(X,Y) 1 m X.length 2 n Y.length 3 fori1tom 4 c[i, 0]  0 5 forj 1ton 6 c[0, j]  0 7 fori 1 tom 8 forj 1ton

23. Computing the length of an LCS 9 ifxi == yj 10 c[i, j]  c[i-1, j-1]+1 11 b[i, j]  “” 12 elseif c[i–1, j] c[i, j-1] 13 c[i, j]  c[i-1, j] 14 b[i, j]  “” 15 elsec[i, j] c[i, j-1] 16 b[i, j]  “” 17 returnc and b

24. Remarks • Again, a slight change in the algorithm allows us to obtain a particular LCS • Also, we can make minor changes to the recursive algorithm and obtain a memoized version (whose running time is O(mn))

25. Writing a Translation Program • In real life, different words may be searched with different frequencies • E.g.,apple may be more often than pizza • Also, there may be different frequencies for the unsuccessful searches • E.g., we may unluckily search for a word in the range (hotdog, pizza) more often than in the range (apple, banana)

26.  apple 0.01 = hotdog 0.02 = apple 0.21 (hotdog, pizza) 0.04 (apple, banana) 0.10 = pizza 0.04 = banana 0.18 0.11 (pizza, spaghetti) 0.05 (banana, burger) = spaghetti 0.07 = burger 0.01 0.04  spaghetti (burger, hotdog) 0.12 • Suppose your friend in Google gives you the probabilities of what a search will be:

27. banana pizza apple spaghetti hotdog burger • Given these probabilities, we may want words that are searched more frequently to be nearer the root of the search tree This tree has better expected performance

28. Expected Search Time • To handle unsuccessful searches, we can modify the search tree slightly (by adding dummy leaves), and define the expected search time as follows: • Let k1 k2 … kn denote the n keys, which correspond to the internalnodes • Let d0d1d2 … dnbedummykeys for ranges of the unsuccessful search •  dummy keys correspond to leaves

29. k2 k5 k1 d0 d1 k4 k6 d4 d5 d6 k3 d3 d2 Search tree of Page 28 after modification

30. Search Time • Lemma: Based on the modified search tree: • when we search for a word ki, • search time = node-depth(ki) • when we search for a word in range dj, • search time = node-depth(dj)

31. Expected Search Time • Let pi = Pr( ki is searched ) • Let qj = Pr( word in dj is searched ) • So, Sipi + Sjqj= 1 • Expected search time • = Sipi node-depth(ki) + Sjqj node-depth(dj)

32. 15.5 Optimal Binary search trees cost:2.75 optimal!! cost:2.80

33. Optimal Binary Search Tree Question: Given the probabilities pi and qj, can we construct a binary search tree whose expected search time is minimized? Such a search tree is called an Optimal Binary Search Tree

34. Optimal Substructure • Let T = optimal BST for the keys ( ki, ki+1, … , kj; di-1, di, … , dj ). • Let L and R be its left and right subtrees. • Lemma: Suppose kr is the root of T. Then, • L must be an optimal BST for the keys (ki , ki+1, … , kr-1; di-1, di, … , dr-1) • R must be an optimal BST for the keys (kr+1, kr+2, … , kj; dr, dr+1, … , dj)

35. Optimal Substructure Let Ei,j = expected time spent with the keys ( ki, ki+1, …, kj; di-1, di, …, dj ) in optimal BST Let wi,j =Ss=i to jps + St=i-1 to jqt = sum of the probabilities of keys ( ki, ki+1, … , kj; di-1, di, … , dj )

36. kr Contribute wi,j Contribute Er+1,j Contribute Ei,r-1 Optimal Substructure Lemma: For any ji, Ei,j = minr{Ei,r-1 + Er+1,j + wi,j}

37. Optimal Binary Search Tree Define a function Compute_E(i,j) as follows: Compute_E(i, j)/* Finding ei,j */ 1. if (i == j+1) return qj; /* Exp time with key dj */ 2. min = ; 3. for (r = i, i+1, …, j) { g = Compute_E(i,r-1)+ Compute_E(r+1,j) + wi,j ; if (gmin) min = g; } 4. return min ;

38. Catalan Number Optimal Binary Search Tree • Question: We want to get Compute_E(1,n) • What is its running time? • Similar to Matrix-Chain Multiplication, the recursive function runs in W(3n) time • In fact, it will examine at most once for all possible binary search tree Running time = O(C(2n-2,n-1)/n)

39. Overlapping Subproblems Here, we can see that : To Compute_E(i,j) and Compute_E(i,j+1), there are many COMMON subproblems: Compute_E(i,i+1), …, Compute_E(i,j-1) So, in our recursive algorithm, there are many redundant computations ! Question: Can we avoid it ?

40. Bottom-Up Approach • Let us create a 2D table E to store all Ei,j values once they are computed • Let us also create a 2D table W to store all wi,j • We first compute all entries in W. • Next, we compute Ei,j for j-i = 0,1,2,…,n-1

41. Bottom-Up Approach BottomUp_E() /* Finding min #operations */ 1. Fill all entries of W 2. for j = 1, 2, …, n, set E[j+1,j] = qj ; 3. for (length = 0,1,2,…, n-1) Compute E[i,i+length] for all i; // From W and E[x,y] with |x-y| < length 4. return E[1,n] ; Running Time = Q(n3)

42. The table e[i,j], w[i,j], and root[i,j] compute by OPTIMAL-BST on the key distribution.

43. Remarks • Again, a slight change in the algorithm allows us to get the exact structure of the optimal binary search tree • Also, we can make minor changes to the recursive algorithm and obtain a memoized version (whose running time is O(n3))

44. Homework • Use the dynamic programming approach to write a program to find the maximum sum in any contiguous sub-array of a given array of n real values. Show the time complexity of your algorithm. (Due Nov. 23) • Practice at home: exercises 15.4-4, 15.4-5, 15.5-2, 15.5-3

45. Brainstorm • There is a staircase with n steps. Your friend, Jack, wants to count how many different ways he can walk up this staircase. Because John is quite tall, in one move, he can choose either to walk up 1 step, 2 steps, or 3 steps. Let Fk denote the number of ways John can walk up a staircase with k steps. So, F1 = 1, F2 = 2, F3 = 4, and F4 = 7. Derive a recurrence for Fk, and show that Fncan be computed in O(n) time.

46. Brainstorm • There is a staircase with n stages, and on each stage there is a coupon to your favorite restaurant. Each coupon has an associated value which may be different. You can climb up 1, 2, or 3 stages in a step. Since you are in a hurry, you need to reach the top of the stair in at most L steps, where L ≤ n. When you visit a particular stage, you can collect the coupon on that stage. Find a way to climb the stair within L steps so that you can collect coupons with maximum total value. Your algorithm should run in O(n2) time.

47. Brainstorm • 外遇村裡有50對夫妻，丈夫全都有外遇。妻子都知道所有外遇男人，就是不知道自己的先生有外遇，妻子之間彼此也不會互相交換訊息。村子有一個規定，妻子若能證明自己的先生外遇，就必需在當天晚上12:00殺死他。有一天，公認不會說謊的皇后來到外遇村，宣布至少有一位丈夫不忠。結果會發生甚麼事？