1 / 54

Divide & Conquer

Divide & Conquer. Themes Reasoning about code (correctness and cost) recursion, induction, and recurrence relations Divide and Conquer Examples sorting (insertion sort & merge sort) computing powers. Sorting.

richard-rivers
Download Presentation

Divide & Conquer

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Divide & Conquer • Themes • Reasoning about code (correctness and cost) • recursion, induction, and recurrence relations • Divide and Conquer • Examples • sorting (insertion sort & merge sort) • computing powers

  2. Sorting • Given a list L = (L1 … Ln) of elements from a totally ordered set (e.g. integers with ≤) return a list M = (M1 … Mn) such that • (sorted M), i.e. M1 ≤ M2 ≤  ≤ Mn • M = (permutation L), i.e. Mi = L(i) where  is a bijection from {1,…,n} to {1,…,n} • Use divide and conquer to derive several recursive algorithms and the principle of balance to obtain an efficient algorithm

  3. sortedp • Predicate to test to see if a list is sorted • Should check to see that L is a list of integers • (x1 x2 … xn) with x1 ≤ x2 ≤  ≤ xn • xi ≤ xj for 1 ≤ i < j ≤ n (defun sortedp (L) (cond ((equal L nil) t) ((equal (length L) 1) t) ((and (<= (first L) (second L)) (sortedp (rest L))))))

  4. Correctness of sortedp • L = (x1 x2 … xn) • By induction on len(L) • Base cases L nil or of length 1 • Show xi ≤ xj for 1 ≤ i < j ≤ n • x1 ≤ x2 and by induction xi ≤ xj for 2 ≤ i < j ≤ n • x1 ≤ x2 ≤ xj  x1 ≤ xj for 2 ≤ j ≤ n

  5. Precise Proof is Difficult • We take advantage of informal reasoning and names which provide intuitive meaning • L = (x1 x2 … xn) • Prove (fn123 L) = t if xi ≤ xj for 1 ≤ i < j ≤ n and nil otherwise using equational properties of fn13, fn14, fn15, fn17, fn81, fn100 and fn150 (defun fn123 (L) (fn13 ((fn81 L nil) t) ((fn81 (fn150 L) 1) t) ((fn82 (fn100 (fn14 L) (fn17 L)) (fn123 (fn15 L))))))

  6. Insertion Sort • To sort x1,…,xn, • recursively sort x2,…,xn • insert x1 into x2,…,xn • (see code for details) • Prove correctness by induction

  7. Insertion Sort (defun insertionsort (L) (if (equal L nil) nil (insert (first L) (insertionsort (rest L))))) (defun insert (x L) (cond ((equal L nil) (list x)) ((<= x (first L)) (cons x L)) ((cons (first L) (insert x (rest L))))))

  8. Insertion Sort (Example) • (insertionsort ‘(7,6,5,4,3,2,1,0)) • recursive call returns (0,1,2,3,4,5,6) • (insert 7 ‘(0,1,2,3,4,5,6)) • (0,1,2,3,4,5,6,7) • Number of comparisons to sort inputs that are in reverse sorted order (worst case) C(n) = C(n-1) + (n-1) C(1) = 0

  9. Correctness of insertionsort L = (L1 … Ln) • (insertionsort L) terminates and returns a list of length n • (insertionsort L) is sorted • (insertionsort L) is a permutation of L Proof by induction on n • For n  0, recursive call with smaller n • Recursive call produces a sorted list of (rest L) by induction and (insert (first L) (insertionsort (rest L))) returns a sorted list (by induction) • Recursive call returns a permutation of (rest L) and insert returns a list containing (first L) and elements of (rest L)

  10. Solve Recurrence for C(n)

  11. Merge Sort • To sort x1,…,xn, • recursively sort x1,…,xa and xa+1,…,xn, where a = n/2 • merge two sorted lists • Insertion sort is a special case where a=1

  12. Mergesort (defun mergesort (L) (cond ((equal L nil) nil) ((equal (length L) 1) L) ((merge (mergesort (take (floor (length L) 2) L)) (mergesort (drop (floor (length L) 2) L))))))

  13. Merge (defun merge (L M) (cond ((equal L nil) M) ((equal M nil) L) ((<= (first L) (first M)) (cons (first L) (merge (rest L) M))) ((cons (first M) (merge L (rest M))))))

  14. Take and Drop (defun take (n L) (cond ((equal L nil) nil) ((equal n 0) nil) ((cons (first L) (take (- n 1) (rest L)))))) (defun drop (n L) (cond ((equal L nil) nil) ((equal n 0) L) ((drop (- n 1) (rest L)))))

  15. Merge Sort (Example) • (7,6,5,4,3,2,1,0) • after recursive calls (4,5,6,7) and (0,1,2,3) • Number of comparisons needed to sort, worst case (the merged lists are perfectly interleaved) M(n) = 2M(n/2) + n-1 M(1) = 0 • What is the best case (all in one list > other list)? M(n) = 2M(n/2) + n/2

  16. Comparison of Insertion and Merge Sort • Count the number of comparisons for different n=2k • M(n)/C(n) 0 as n increases • C(n) grows faster than M(n) • C(2n)/C(n)  4 • So, apparently quadratic. • C(n) = Θ(n2) • M(2n)/M(n)  2 as n increases

  17. Solve Recurrence for M(n)

  18. Mergesort What is wrong with the following? (defun mergesort (L) (if (equal L nil) nil (merge (mergesort (mytake (floor (length L) 2) L)) (mergesort (drop (floor (length L) 2) L)))))

  19. Correctness of mergesort L = (L1 … Ln) • (mergesort L) terminates and returns a list of length n • (mergesort L) is sorted • (mergesort L) is a permutation of L • Proof by induction on n • For n  2, recursive calls with smaller n • Recursive calls produce sorted lists by induction and merge produces a sorted list • Recursive calls contain lists whose concatenation is L and hence by induction return lists whose concatenation is L. merge returns a permutation of L

  20. Properties of nth L = (L1 … Llen(L)) • 0 < n ≤ len(L), then (nth n L) = Ln • Proof by induction on n • n > len(L), then (nth n L) = nil • Proof by induction on len(L) (defun nth (n L) (if (equal L nil) nil (if (equal n 1) (first L) (nth (- n 1) (rest L)))))

  21. Properties of nth L = (L1 … Llen(L)) • 0 < n ≤ len(L), then (nth n L) = Ln • Proof by induction on n • Base case. n = 1 return (first L) • By induction (nth (- n 1) (rest L)) returns the (n-1)st element of (rest L) which is the nth element of L (n > 1)

  22. Properties of nth L = (L1 … Llen(L)) • n > len(L), then (nth n L) = nil • Proof by induction on len(L) • Base case. Len(L) = 0. L = nil and return nil • n > len(L)  n-1 > len((rest L)), therefore by induction (nth (- n 1) (rest L)) returns nil

  23. Properties of take and drop L = (L1 … Llen(L)) • 0 ≤ n ≤ len(L), then len((take n L)) = n • Proof by induction on n • n > len(L), then (take n L) = L • Proof by induction on len(L) • 0 < k ≤ n ≤ len(L), then (nth k (take n L)) = Lk • Proof by property of nth and induction on k • 0 ≤ n ≤ len(L), then len((drop n L)) = len(L) – n • n > len(L), then (drop n L) = nil • 0 ≤ n ≤ len(L), 0 < k ≤ len(L)-n, then (nth k (drop n L)) = Ln+k • 0 ≤ n ≤ len(L), (append (take n L) (drop n L)) = L

  24. Properties of merge L = (L1 … Lm) and M = (M1 … Mn) • Assume L is sorted and M is sorted • (merge L M) is a list of length m+n • (merge L M) is sorted • (merge L M) is a permutation of (L1 … Lm M1 … Mn)

  25. Exercise • Use check= to verify properties of take and drop for several examples • Use induction to prove the properties

  26. Solution Property 1 L = (L1 … Llen(L)) • 0 ≤ n ≤ len(L), then len((take n L)) = n • Proof by induction on n • Base case. n = 0 return nil and len(nil) = 0. • Assume n > 0 and show that len((take n L)) = n. Assume that take returns a list of length k for k < n [Inductive hypothesis] • In this case take returns (cons (first L) (take (- n 1) (rest L))) and since n-1 ≤ len((rest L)) (take (- n 1) (rest L)) by induction is a list of length n-1. Therefore (cons (first L) (take (- n 1) (rest L))) has length (n-1) + 1 = n.

  27. Solution Property 2 L = (L1 … Llen(L)) • n > len(L), then (take n L) = L • Proof by induction on len(L) • Base case. len(L) = 0 and take returns L = nil independent of n. • Assume n > len(L) = and show that (take n L) = L. Assume that take returns M when len(M) <  [Inductive hypothesis] • In this case take returns (cons (first L) (take (- n 1) (rest L)). By induction, (take (- n 1) (rest L)) returns (rest L) and therefore (take n L) returns (cons (first L) (rest L)) = L.

  28. Solution Property 3 L = (L1 … Llen(L)) • 0 < k ≤ n ≤ len(L), then (nth k (take n L)) = Lk • Proof by property of nth and induction on k • Base case. k = 1and by the first property of nth, (nth 1 (take n L)) = the first element of (take n L) which by the definition of take is the first element of L. • Assume the jth element of (take n L) = Lj for j < k [Inductive hypothesis] and show that the kth element of (take n L) = Lk. • By the first property of nth (nth k (take n L)) is the kth element of (take n L) and by the definition of take, that is the (k-1)st element of (take (- n 1) (rest L)) which by induction is the (k-1)st element of (rest L) which is the kth element of L.

  29. Solution of Properties 4-7 L = (L1 … Llen(L)) • Properties 4-6 for drop are analogous to properties 1-3 for take and their proofs are similar to those for properties 1-3. • Property 7 follows from properties 3 and 6.

  30. Computing Powers • Recursive definition: • an = a  an-1, n > 0 • a0 = 1 • Number of multiplications • M(n) = M(n-1) + 1, M(0) = 0 • M(n) = n

  31. Exercise • Write a recursive ACL program to compute the power of a number using the recursive definition on the previous slide • Prove by induction on n that (power a n) = an

  32. Solution ; inputs a rational number a and a natural number n ; output a rational number equal to a^n (defun power (a n) (if (= n 0) 1 (* a (power a (- n 1))))) (check= (power 2 0) 1) (check= (power 2 10) 1024)

  33. Solution Continued (defun power (a n) (if (= n 0) 1 (* a (power a (- n 1))))) • Base case n = 0 • (power a 0) = 1 = a0 • Assume n > 0 and (power a (- n 1)) = an-1 [Inductive Hypothesis] • (power a n) = (* a (power a (- n 1)) = a×an-1 [by induction] = an

  34. Binary Powering (Recursive) • Binary powering • x16 = (((x2)2)2)2, 16 = (10000)2 • x23 = (((x2)2x)2x)2x, 23 = (10111)2 • Recursive (right-left) xn = (xn/2)2 x(n % 2) M(n) = M(n/2) + [n % 2] + 1

  35. Exercise • Write a recursive ACL program to compute the power of a number using the binary powering algorithm • Prove by induction on n that (fastpower a n) = an

  36. Solution ; inputs a rational number a and a natural number n ; output a rational number equal to a^n (defun fastpower (a n) (if (= n 0) 1 (let ((b (fastpower a (floor n 2)))) (if (evenp n) (* b b) (* b b a))))) (check= (power 2 0) 1) (check= (fastpower 2 16) 65536) (check= (fastpower 2 15) 32768)

  37. Solution Continued • Base case n = 0 • (fastpower a 0) = 1 = a0 • Assume n > 0 and (fastpower a k) = ak for k < n [Inductive Hypothesis] • Even case n = 2k • (fastpower a n) = (* (fastpower a k) (fastpower a k)) = ak×ak [by induction] = a2k = an • Odd case n = 2k+1 • (fastpower a n) = (* (fastpower a k) (fastpower a k) a) = ak×ak×a [by induction] = a2k+1 = an

  38. Binary Powering • Number of multiplications • Let (n) = number of 1bits in binary representation of n • num bits = lg(n) +1 • M(n) = lg(n) + (n) • (n) ≤ lg(n) => • M(n) ≤ 2lg(n)

  39. Asymptotic Growth Rates f(n) = O(g(n)) [grows at the same rate or slower] There exists positive constants c and n0 such that f(n)  c g(n) for all n  n0 Ignore constants and low order terms

  40. Asymptotic Growth Rates f(n) = o(g(n)) [grows slower] f(n) = O(g(n)) and g(n)  O(f(n)) limn f(n)/g(n) = 0 f(n) = (g(n)) [grows at the same rate] f(n) = O(g(n)) and g(n) = O(f(n))

  41. Asymptotic Growth Rates (log(n)) – logarithmic [log(2n)/log(n) = 1 + log(2)/log(n)] (n) – linear [double input  double output] (n2) – quadratic [double input  quadruple output] (n3) – cubit [double input  output increases by factor of 8] (nk) – polynomial of degree k (cn) – exponential [double input  square output]

  42. Merge Sort and Insertion Sort Insertion Sort TI(n) = TI(n-1) + (n) =(n2) [worst case] TI(n) = TI(n-1) + (1) =(n) [best case] Merge Sort TM(n) = 2TM(n/2) + (n) =(nlogn) [worst case] TM(n) = 2TM(n/2) + (n) =(nlogn) [best case]

  43. Divide & Conquer Recurrence Assume T(n) = aT(n/b) + (n) T(n) = (n) [a < b] T(n) = (nlog(n)) [a = b] T(n) = (nlogb(a)) [a > b]

  44. Karatsuba’s Algorithm Using the classical pen and paper algorithm two n digit integers can be multiplied in O(n2) operations. Karatsuba came up with a faster algorithm. Let A and B be two integers with A = A110k + A0, A0 < 10k B = B110k + B0, B0 < 10k C = A*B = (A110k + A0)(B110k + B0) = A1B1102k + (A1B0 + A0 B1)10k + A0B0 Instead this can be computed with 3 multiplications T0 = A0B0 T1 = (A1 + A0)(B1 + B0) T2 = A1B1 C = T2102k + (T1 - T0 - T2)10k + T0

  45. Complexity of Karatsuba’s Algorithm Let T(n) be the time to compute the product of two n-digit numbers using Karatsuba’s algorithm. Assume n = 2k. T(n) = (nlg(3)), lg(3)  1.58 T(n)  3T(n/2) + cn  3(3T(n/4) + c(n/2)) + cn = 32T(n/22) + cn(3/2 + 1)  32(3T(n/23) + c(n/4)) + cn(3/2 + 1) = 33T(n/23) + cn(32/22 + 3/2 + 1) …  3iT(n/2i) + cn(3i-1/2i-1 + … + 3/2 + 1) ...  cn[((3/2)k - 1)/(3/2 -1)] --- Assuming T(1)  c  2c(3k - 2k)  2c3lg(n) = 2cnlg(3)

  46. Identities for Sums

  47. Some Common Sums

  48. Arithmetic Series • Sum of consecutive integers (written slightly differently):

  49. Arithmetic Series (Proof)

  50. Geometric Series = 1 + x + x2 + … + xn-1 =

More Related