Divide-and-Conquer

1. Divide-and-Conquer

2. Steps in Divide and Conquer As its name implies divide-and-conquerinvolves dividing a problem into smaller problems that can be more easily solved. While the specifics vary from one application to another, divide-and-conquer always includes the following three steps in some form: Divide - Typically this steps involves splitting one problem into two problems of approximately 1/2 the size of the original problem. Conquer - The divide step is repeated (usually recursively) until individual problem sizes are small enough to be solved (conquered) directly. Recombine - The solution to the original problem is obtained by combining all the solutions to the sub-problems. Divide and Conquer is not applicable to every problem class. Even when D&C works it may not provide for an efficient solution.

3. Binary Search - Iterative Version procedure bin1(n:integer,x:keytype,S:list,loc:index)is lo,hi,mid : index; begin lo:=1; hi:=n; loc:=0; while lo<=hi and loc=0 loop mid:=(lo+hi)/2; if x=S(mid) then loc:=mid; elsif x<S(mid) then hi:=mid-1; else lo:=mid+1; endif; endloop; end bin1; x=42 lo=1 hi=9 mid=4 S(4)=19 lo=5 hi=9 mid=7 S(7)=45 lo=5 hi=6 mid=5 S(5)=24 lo=6 hi=6 mid=6 S(6)=39 lo=7 hi=6 S 0 10 1 12 2 15 3 19 4 19 5 24 6 39 7 45 8 53 9 77 T(n) = C + lg2n D -> O(lg2n)

4. x=42 lo=0 hi=9 mid=4 S(4)=19 lo=5 hi=9 mid=7 S(7)=45 lo=5 hi=6 mid=5 S(5)=24 lo=6 hi=6 mid=6 S(6)=39 lo=7 hi=6 return 0 Binary Search - Recursive Version function location(lo,hi : index) return index is begin if lo>hi then return 0; else mid:=(lo+hi)/2; if x=S(mid) then return mid; elsif x<S(mid) then return location(lo,mid-1); else return location(mid+1,hi); end if; end if; end location; S 0 10 1 12 2 15 3 19 4 19 5 24 6 39 7 45 8 53 9 77

5. Analysis of Function location( ) We need a closed form expression for T(n) that does not contain a term involving T. We assume that n>1 and note that we need to replace T(n/2) with an explicit function of n. We can use the recurrence relation to express the term T(n/2) in another form that can, in turn be substituted back into our expression for T(n). Eventually we can write a general expression for the kth substitution. Then we can determine a value for k (in terms of n) that sets the parameter of T equal to one. This lets us substitute T(1)=C in our recurrence and replace any other occurrences of k with terms involving n. We can then determine the order of complexity. C n<=1 T(n) = T(n/2) + D otherwise T(n/2)=T(n/4) + D T(n)=(T(n/4) + D) + D T(n)=T(n/4) + 2D T(n)=T(n/8) + 3D : T(n)=T(n/2k) + kD T(n)=T(1) + (lg2n)D T(n)=C + (lg2n)D T(n) -> O(lg2n)

6. Mergesort procedure msort(n:integer; S:list) is h: constant :=n/2; m: constant :=n-h; U,V : list; begin if n>1 then U(1..h):=S(1..h); V(1..m):=S(h+1..n); msort(h,U); msort(m,V); merge(h,m,U,V,S); endif; end msort; procedure merge(h,m:integer; U,V,S:list); i,j,k : integer; begin i:=1; j:=1; k:=1; while i<=h and j<=m loop if U(i)<V(j) then S(k):=U(i); i:=i+1; else S(k):=V(j); j:=j+1; endif; k:=k+1; end loop; if i>h then S(k..h+m):=V(j..m); else S(k..h+m):=U(i..h); end if; end merge; Write the recurrence relation for the complexity of msort and then solve it.

7. Quicksort procedure quicksort(lo,hi: integer) is pivot : integer; begin if hi>lo then partition(lo,hi,pivot); quicksort(lo,pivot-1); quicksort(pivot,+1,hi); end if; end quicksort; procedure partition(lo,hi,pivot)is j,item : integer; begin item := S(lo); j:= lo; for i in lo+1..hi loop if S(i)<item then j:=j+1; swap(S(i),S(j)); end if; end loop; pivot:=j swap(S(lo),S(pivot)); end partition; Where is the recombine step in the quicksort divide-and-conquer algorithm?

8. i j 1 2 3 4 5 6 7 8 9 10 11 12 13 14 2 1 6 9 8 5 7 4 6 3 5 4 7 6 0 5 4 2 6 5 8 9 7 4 6 3 5 4 7 6 0 5 6 3 6 5 4 9 7 8 6 3 5 4 7 6 0 5 8 4 6 5 4 3 7 8 6 9 5 4 7 6 0 5 9 5 6 5 4 3 5 8 6 9 7 4 7 6 0 5 10 6 6 5 4 3 5 4 6 9 7 8 7 6 0 5 13 7 6 5 4 3 5 4 0 9 7 8 7 6 6 5 14 8 6 5 4 3 5 4 0 5 7 8 7 6 6 9 8 5 5 4 3 5 4 0 6 7 8 7 6 6 6 pivot value items being swapped new sublists for next pass of quicksort Quicksort Example

9. Quicksort: Worst-Case PerformanceWhat does it mean? An analysis shows that the worst-case performance for quicksort( ) is order O(n2). However, quicksort performs much better than this in practice. Does this mean that algorithm analysis is a waste of effort? Not really. In addition to showing worst-case performance, algorithm analysis provides us with a better understanding of the conditions under which an algorithm performs well. We can see that the quicksort algorithm performs worst on a sorted list and performs best on a randomly distributed list (at least one in which the pivot value is always near the median value of the sublist being partitioned). Algorithm analysis helps us determine which algorithm is best for a particular application, as well as suggesting ways to modify an algorithm to improve its performance. Can you think of a simple way to improve quicksort( )?

10. Other Applications for D&C Searching and sorting problems are obvious candidates for D&C but the Divide-and-Conquerproblem-solving method gets used in other ways you might not expect. Game Strategy Computing numeric roots Fast Exponentiation

11. Modular Exponentiation Suppose that we need to compute the value of anfor some reasonably large n. Such problems occur in primality testing for cryptography. The simplest algorithm performs n − 1 multiplications, by computing a × a × . . . × a. However, we can do better by observing that n = ⌊n/2⌋ + ⌈n/2⌉. If n is even, then an = (an/2)2. If n is odd, then an = a(a⌊n/2⌋)2. In either case, we have halved the size of our exponent at the cost of at most two multiplications, so O(lg2n) multiplications suffice to compute the final value. function power(a, n) if (n = 0) return(1) x = power(a, ⌊n/2⌋) if (n is even) return(x2) else return(a * x2) Introduction to Algorithms - Steven Skiena

12. Square Root (and other roots) of a Number The square root of n is the number r such that r2 = n. Square root computations are performed inside every pocket calculator – but how? Observe that the square root of n ≥ 1 must be at least 1 and at most n. Let l = 1 and r = n. Consider the midpoint of this interval, m = (l + r)/2. How does m2 compare to n? If n ≥ m2, then the square root must be greater than m, so the algorithm repeats with l = m. If n < m2, then the square root must be less than m, so the algorithm repeats with r = m. Either way, we have halved the interval with only one comparison. Therefore, after only lg2n rounds we will have identified the square root to within ±1.

13. Twenty Questions Player One - Picks a word. Player Two - Asks a series of YES or NO questions, attempting to guess the word in <20 tries. alphabetical version of Twenty Questions

14. Computing the median of n numbers is easy: just sort them. The drawback is that this takes O(n log2n) time, whereas we would ideally like something linear. We have reason to be hopeful, because sorting is doing far more work than we really need. We just want the middle element and don't care about the relative ordering of the rest of them. When looking for a recursive solution, it is paradoxically often easier to work with a more general version of the problem for the simple reason that this gives a more powerful step to recurse upon. In our case, the generalization we will consider is selection. SELECTION Input: A list of numbers S; an integer k Output: The kth smallest element of S For instance, if k = 1, the minimum of S is sought, whereas if k = [|S|/2], it is the median. Computing the Median Divide-and-Conquer Algorithms - Vazirani

15. The K-Median Algorithm (Selection) Chapter 2 Divide-and-conquer algorithms – see readings http://www.cs.berkeley.edu/~vazirani/

16. Finding the Median 6 7 8 4 0 2 4 3 3 5 8 1 4 8 9 7 3 2 0 1 3 3 6 7 8 4 4 5 8 4 8 9 7 SL SM SR 6 7 8 4 4 5 5 8 4 8 9 7 4 4 4 5 6 7 8 8 8 9 7 SL SM SR 4 4 4 4

17. Summary Divide-and-Conquer (actually three steps) 1. divide 2. conquer 3. recombine Most Naturally Implemented using Recursion (although iteration possible) Analyze performance by solving Recurrence Relations Worst-Case Performance may not be typical (or even possible) performance. Divide-and-Conquer is embedded in Arithmetic Processors/Calculators Cryptography relies on D&C Methods Game Strategy can be based on D&C Using D&C Selection Algorithm to Find the Median