Theory of Algorithms: Divide and Conquer

1. Theory of Algorithms:Divide and Conquer

2. Objectives • To introduce the divide-and-conquer mind set • To show a variety of divide-and-conquer solutions: • Merge Sort • Quick Sort • Convex Hull by Divide-and-Conquer • Strassen’s Matrix Multiplication • To discuss the strengths and weaknesses of a divide-and-conquer strategy

3. Divide-and-Conquer • Best known algorithm design strategy: • Divide instance of problem into two or more smaller instances • Solve smaller instances recursively • Obtain solution to original (larger) instance by combining these solutions • Recurrence Templates apply • Recurrences are of the form T(n) = aT(n/b) + f (n), a  1, b  2 • Silly Example (Addition): • a0 + …. + an-1 = (a0 + … + an/2-1) + (an/2 + … an-1)

4. Divide-and-Conquer Illustrated A PROBLEM OF SIZE n SUBPROBLEM 1 OF SIZE n/2 SUBPROBLEM 2 OF SIZE n/2 A SOLUTION TO SUBPROBLEM 1 A SOLUTION TO SUBPROBLEM 2 A SOLUTION TO THE ORIGINAL PROBLEM

5. Mergesort • Algorithm: • Split A[1..n] in half and put copy of each half into arrays B[1.. n/2 ] and C[1.. n/2 ] • Recursively MergeSort arrays B and C • Merge sorted arrays B and C into array A • Merging: • REPEAT until no elements remain in one of B or C • Compare 1st elements in the rest of B and C • Copy smaller into A, incrementing index of corresponding array • Once all elements in one of B or C are processed, copy the remaining unprocessed elements from the other array into A

6. Mergesort Example 7 2 1 6 4 7 2 1 6 4 7 2 1 6 4 7 2 4 6 2 7 1 2 7 1 2 4 6 7

7. Efficiency of Mergesort • Recurrence: • C(n) = 2 C(n/2) + Cmerge(n) for n > 1, C(1) = 0 • Cmerge(n) = n - 1 in the worst case • All cases have same efficiency: (n log n) • Number of comparisons is close to theoretical minimum for comparison-based sorting: • log n! ≈ n lg n - 1.44 n • Space requirement: ( n ) (NOT in-place) • Can be implemented without recursion (bottom-up)

8. Quicksort • Select a pivot (partitioning element) • Rearrange the list into two sublists: • All elements positioned before the pivot are ≤ the pivot • Those positioned after the pivot are > the pivot • Requires a pivoting algorithm • Exchange the pivot with the last element in the first sublist • The pivot is now in its final position • QuickSort the two sublists p A[i]≤p A[i]>p

9. The Partition Algorithm

10. Quicksort Example 5 3 1 9 8 2 7 2 3 1 8 9 7 i j i j i j 5 3 1 9 8 2 7 2 1 3 8 7 9 i j i j i j 5 3 1 2 8 9 7 2 1 3 8 7 9 i j j i j i 5 3 1 2 8 9 7 1 2 3 7 8 9 j i 2 3 1 5 8 9 7 Recursive Call Quicksort (7) & Quicksort (9) Recursive Call Quicksort (1) & Quicksort (3) Recursive Call Quicksort (2 3 1) & Quicksort (8 9 7)

11. Worst Case Efficiency of Quicksort • In the worst case all splits are completely skewed • For instance, an already sorted list! • One subarray is empty, other reduced by only one: • Make n+1 comparisons • Exchange pivot with itself • Quicksort left = Ø, right = A[1..n-1] • Cworst = (n+1) + n + … + 3 = (n+1)(n+2)/2 - 3 = (n2) p A[i]≤p A[i]>p

12. General Efficiency of Quicksort • Efficiency Cases: • Best: split in the middle — ( n log n) • Worst: sorted array! — ( n2) • Average: random arrays —( n log n) • Improvements (in combination 20-25% faster): • Better pivot selection: median of three partitioning avoids worst case in sorted files • Switch to Insertion Sort on small subfiles • Elimination of recursion • Considered the method of choice for internal sorting for large files (n ≥ 10000)

13. Objectives • To introduce the divide-and-conquer mind set • To show a variety of divide-and-conquer solutions: • Merge Sort • Quick Sort • Convex Hull by Divide-and-Conquer • Strassen’s Matrix Multiplication • To discuss the strengths and weaknesses of a divide-and-conquer strategy

14. QuickHull Algorithm • Sort points by increasing x-coordinate values • Identify leftmost and rightmost extreme points P1 and P2 (part of hull) • Compute upper hull: • Find point Pmax that is farthest away from line P1P2 • Quickhull the points to the left of line P1Pmax • Quickhull the points to the left of line PmaxP2 • Similarly compute lower hull Pmax L P2 L P1

15. Finding the Furthest Point • Given three points in the plane p1 , p2 , p3 • Area of Triangle =  p1 p2 p3 = 1/2 D • D = •  D = x1 y2 + x3 y1 + x2 y3 - x3 y2 - x2 y1 - x1 y3 • Properties of  D : • Positive iffp3 is to the left of p1p2 • Correlates with distance of p3 from p1p2

16. Efficiency of Quickhull • Finding point farthest away from line P1P2 is linear in the number of points • This gives same efficiency as quicksort: • Worst case: (n2) • Average case: (n log n) • If an initial sort is required, this can be accomplished in ( n log n) — no increase in asymptotic efficiency class • Alternative Divide-and-Conquer Convex Hull: • Graham’s scan and DCHull • Also ( n log n) but with lower coefficients

17. Strassen’s Matrix Multiplication • Strassen observed [1969] that the product of two matrices can be computed as follows: • Op Count: • Each of M1, …, M7 requires 1 mult and 1 or 2 add/sub • Total = 7 mul and 18 add/sub • Compared with brute force which requires 8 mults and 4 add/sub. But is asymptotic behaviour is NB • n/2  n/2 submatrices are computed recursively by the same method = * =

18. Efficiency of Strassen’s Algorithm • If n is not a power of 2, matrices can be padded with zeros • Number of multiplications: • M(n) = 7 M(n/2) for n > 1, M(1) = 1 • Set n = 2k, apply backward substitution • M(2k) = 7M(2k-1) = 7 [7 M(2k-2)] = 72 M(2k-2) = … = 7k • M(n) = 7log n = nlog 7 ≈ n2.807 • Number of additions: A(n)  (nlog 7) • Other algorithms closer to the lower limit of n2 multiplications, but are even more complicated

19. Strengths and Weaknesses of Divide-and-Conquer • Strengths: • Generally improves on Brute Force by one base efficiency class • Easy to analyse using the Recurrence Templates • Weaknesses: • Often requires recursion, which introduces overheads • Can be inapplicable and inferior to simpler algorithmic solutions