Divide & Conquer

1. Divide & Conquer • Themes • Reasoning about code (correctness and cost) • iterative code, loop invariants, and sums • recursion, induction, and recurrence relations • Divide and Conquer • Examples • sorting (insertion sort & merge sort) • computing powers • Euclidean algorithm (computing gcds)

2. Identities for Sums

3. Some Common Sums

4. Arithmetic Series • Sum of consecutive integers (written slightly differently): See http://www.cs.drexel.edu/~kschmidt/CS520/Lectures/2/arithmetic.swf

5. Arithmetic Series (Proof)

6. Geometric Series = 1 + x + x2 + … + xn-1 = See http://www.cs.drexel.edu/~kschmidt/CS520/Lectures/2/geometric.swf

7. Geometric Series (Proof)

8. Floor and Ceiling • Let x be a real number • The floor of x, x, is the largest integer less than or equal to x • If an integer k satisfies k  x < k+1, k = x • E.G. 3.7 = 3, 3 = 3 • The ceiling of x, x, is the smallest integer greater than or equal to x • If an integer k satisfies k-1 < x  k, k = x • E.G. 3.7 = 4, 3 = 3

9. logarithm • y = logb(x)  by = x • Two important cases • ln(x) = loge(x) • lg(x) = log2(x) [frequently occurs in CS] • Properties • log(cd) = log(c) + log(d) • log(cx) = xlog(c) • logb(bx) = x = blogb(x) • d ln(x)/dx = 1/x

10. logarithm • 2k x < 2k+1 k = lg(x) • E.G. 16  25 < 32  4  lg(25) < 5 • lg(25)  4.64 • Change of base • logc(x) = logb(x) / logb(c) Proof. y = logc(x)  cy = x ylogb(c) = logb(x)  y = logb(x) / logb(c)

11. Insertion Sort • To sort x0,…,xn-1, • recursively sort x0,…,xn-2 • insert xn-1 into x0,…,xn-2 • (see code for details) • Loop invariant (just before test, i<n) • x0,…, xi-1 sorted • initialize t = xi

12. Insertion Sort (Example) • (7,6,5,4,3,2,1,0) • after recursive call (1,2,3,4,5,6,7,0) • Insert 0 into sorted subarray • Let 0 “fall” as far as it can • Number of comparisons to sort inputs that are in reverse sorted order (worst case) C(n) = C(n-1) + (n-1) C(1) = 0 See (http://www.cs.drexel.edu/~kschmidt/CS520/Programs/sorts.cc)

13. Merge Sort • To sort x0,…,xn-1, • recursively sort x0,…,xa-1 and xa,…,xn-1, where a = n/2 • merge two sorted lists • Insertion sort is a special case where a=1 • loop invariant for merge similar to insert (depends on implementation)

14. Merge Sort (Example) • (7,6,5,4,3,2,1,0) • after recursive calls (4,5,6,7) and (0,1,2,3) • Number of comparisons needed to sort, worst case (the merged lists are perfectly interleaved) M(n) = 2M(n/2) + (2n-2) M(1) = 0 • What is the best case (all in one list > other list)? M(n) = 2M(n/2) + n/2

15. Comparison of Insertion and Merge Sort • Count the number of comparisons for different n=2k (see and run sort.cpp) • M(n)/C(n) 0 as n increases • C(n) is of higher order. I.e., C(n) bounds M(n) from above, but not tightly • C(2n)/C(n)  4 • So, apparently quadratic. • C(n) = Θ(n2) • M(2n)/M(n)  2 as n increases

16. Solve Recurrence for C(n) See (http://www.cs.drexel.edu/~kschmidt/CS520/Lectures/2/insertionSortSwaps.swf)

17. Solve Recurrence for M(n) See http://www.cs.drexel.edu/~kschmidt/CS520/Lectures/2/mergeSortSwaps.swf

18. Computing Powers • Recursive definition, multiplying as we learned in 4th grade (whatever): • an = a  an-1, n > 0 • a0 = 1 • Number of multiplications • M(n) = M(n-1) + 1, M(0) = 0 • M(n) = n

19. Binary Powering (Recursive) • Binary powering (see http://www.cs.drexel.edu/~kschmidt/CS520/Programs/power.cc) • x16 = (((x2)2)2)2, 16 = (10000)2 • x23 = (((x2)2x)2x)2x, 23 = (10111)2 • Recursive (right-left) xn = (xn/2)2 x(n % 2) M(n) = M(n/2) + [n % 2]

20. Binary Powering (Iterative) (See http://www.cs.drexel.edu/~kschmidt/CS520/Programs/power.cc • Example N y z • 1 x 11 x x2 5 x3 x4 2 x7 x8 1 x7 x16 0 x23 x32 • Loop invariant • xn = y  zN N = n; y = 1; z = x; while (N != 0) { if (N % 2 == 1) y = z*y; z = z*z; N = N/2; }

21. Binary Powering • Number of multiplications • Let (n) = number of 1bits in binary representation of n • num bits = lg(n) +1 • M(n) = lg(n) + (n) • (n) ≤ lg(n) => • M(n) ≤ 2lg(n)

22. Greatest Common Divisors • g = gcd(a,b)  g|a and g|b if e|a and e|b  e|g • gcd(a,0) = a • gcd(a,b) = gcd(b,a%b) • since if g|a and g|b then g|a%b and if g|b and g|a%b then g|a

23. Euclidean Algorithm (Iterative) • (see http://www.cs.drexel.edu/~kschmidt/CS520/Programs/gcd.cc) • a0 = a, a1 = b • ai = qi ai+1+ ai+2 , 0  ai+2 < ai+1 • … • an = qn an+1 • g = an+1

24. Number of Divisions ai = qi ai+1+ ai+2 , 0  ai+2 < ai+1 => ai qi ai+2+ ai+2 = (qi + 1)ai+2  2ai+2 => an 2an+2 => an / an+2 2 a1 / a3  a2 / a4  …  an-1 / an+1  an / an+2  2n • n  lg(a1a2 /g2) • if a,b  N, n  2lg(N)

25. (blank, for notes)