Lecture 7

1. Lecture 7

2. Solution by Substitution Method T(n) = 2 T(n/2) + n • Substitute n/2 into the main equation 2T(n/2) = 2(2(T(n/4)) + n/2) = 4T(n/4) + n And T(n) = 4T (n/4) + 2n • Again by substituting n/4, we can see 4T(n/4) = 4(2T(n/8)) + (n/4) = 8T(n/8) + n And T(n) = 8T(n/8) + 3n • Continuing in this manner, we obtain T(n) = 2k T(n/2k) + k.n Using k = lgn T(n) = nT(1) + nlgn = nlgn + n T(n) = O(nlgn)

3. Overview • Divide and Conquer • Merge Sort • Quick Sort

4. Quick Sort • Divide: • Pick any element p as the pivot, e.g, the first element • Partition the remaining elements into FirstPart,which contains all elements< p SecondPart, which contains all elements ≥ p • Recursively sort the FirstPart and SecondPart • Combine: no work is necessary since sorting is done in place

5. Partition Recursive call p p p p ≤ x p ≤ x Sorted FirstPart Sorted SecondPart x < p Quick Sort A: pivot FirstPart SecondPart x < p Sorted

6. Quick Sort Quick-Sort(A, left, right) ifleft ≥ right return else middle ← Partition(A, left, right) Quick-Sort(A, left, middle–1 ) Quick-Sort(A, middle+1, right) end if

7. p x < p p p ≤ x x < p p ≤ x p Partition A: A: A: p

8. Partition Example A: 4 8 6 3 5 1 7 2

9. Partition Example i=0 A: 4 8 6 3 5 1 7 2 j=1

10. Partition Example i=0 A: 4 8 6 3 5 1 7 2 8 j=1

11. Partition Example i=0 A: 4 8 6 6 3 5 1 7 2 j=2

12. i=1 3 8 Partition Example i=0 A: 4 6 3 5 1 7 2 8 3 j=3

13. i=1 Partition Example A: 4 3 6 8 5 1 7 2 5 j=4

14. i=1 Partition Example A: 4 3 6 8 5 1 1 7 2 j=5

15. i=2 1 6 Partition Example A: 4 3 6 8 5 1 7 2 j=5

16. i=2 Partition Example A: 7 4 3 1 8 5 6 7 2 j=6

17. i=2 i=3 Partition Example A: 8 2 2 4 3 1 8 5 6 7 2 j=7

18. i=3 Partition Example A: 4 3 1 2 5 6 7 8 j=8

19. i=3 Partition Example A: 4 4 3 1 2 5 6 7 8 2

20. x < 4 4 ≤ x Partition Example pivot in correct position A: 2 4 3 1 5 6 7 8

21. Partition(A, left, right) • x ← A[left] • i ← left • for j ← left+1 to right • if A[j] < x then • i ← i + 1 • swap(A[i], A[j]) • end if • end for j • swap(A[i], A[left]) • return i n = right – left +1 Time: cn for some constant c Space: constant

22. 2 3 1 4 5 6 7 8 Quick-Sort(A, 0, 7) Partition A: 4 8 6 3 5 1 7 2

23. 1 3 2 Quick-Sort(A, 0, 7) , partition Quick-Sort(A, 0, 2) A: 5 6 7 8 4 2 3 1

24. Quick-Sort(A, 0, 7) Quick-Sort(A, 0, 0) , base case , return 5 6 7 8 4 3 1 2 1

25. 3 3 Quick-Sort(A, 0, 7) Quick-Sort(A, 1, 1) , base case 5 6 7 8 4 1 2

26. 1 1 3 3 2 2 Quick-Sort(A, 0, 7) Quick-Sort(A, 0, 2), return Quick-Sort(A, 2, 2),return 5 6 7 8 4

27. 5 6 7 8 6 7 8 5 1 3 2 Quick-Sort(A, 0, 7) Quick-Sort(A, 4, 7) , partition Quick-Sort(A, 2, 2),return 4

28. 1 3 2 6 7 8 Quick-Sort(A, 0, 7) Quick-Sort(A, 5, 7) , partition 4 5 6 6 7 8

29. 1 3 2 Quick-Sort(A, 0, 7) Quick-Sort(A, 6, 7) , partition 4 5 6 7 7 8 8

30. 1 3 2 Quick-Sort(A, 0, 7) , return , base case Quick-Sort(A, 7, 7) 4 5 6 7 8 8

31. 1 3 2 Quick-Sort(A, 0, 7) , return Quick-Sort(A, 6, 7) 4 5 6 7 8

32. 1 3 2 6 7 8 Quick-Sort(A, 0, 7) , return Quick-Sort(A, 5, 7) 4 5

33. 5 1 3 2 6 7 8 Quick-Sort(A, 0, 7) , return Quick-Sort(A, 4, 7) 4

34. 5 1 3 2 6 7 8 Quick-Sort(A, 0, 7) , done! Quick-Sort(A, 0, 7) 4

35. cn n n/2 n/2 2 × cn/2 = cn log n levels n/4 n/4 n/4 4 × c/4 = cn n/4 n/3 × 3c = cn 3 3 3 Quick-Sort: Best Case • Even Partition Total time: (nlogn)

36. Quick-Sort: Worst Case • Unbalanced Partition cn n c(n-1) n-1 c(n-2) n-2 3c 3 Happens only if • input is sortd • input is reversely sorted 2c 2 Total time: (n2)

37. Quick-Sort: an Average Case Quick-Sort: an Average Case • Suppose the split is 1/10 : 9/10 cn n cn 0.1n 0.9n log10n 0.81n 0.09n 0.01n 0.09n cn log10/9n 2 ≤cn ≤cn 2 Total time: (nlogn)

38. Quick-Sort Summary • Time • Most of the work done in partitioning. • Average case takes (n log(n)) time. • Worst case takes (n2) time Space • Sorts in-place, i.e., does not require additional space

39. Recursion • Recursion is more than just a programming technique. It has two other uses in computer science and software engineering, namely: • as a way of describing, defining, or specifying things. • as a way of designing solutions to problems (divide and conquer).

41. In general, we can define the factorial function in the following way:

42. Iterative Definition • This is an iterativedefinition of the factorial function. • It is iterative because the definition only contains the algorithm parameters and not the algorithm itself. • This will be easier to see after defining the recursive implementation.

43. Recursive Definition • We can also define the factorial function in the following way:

44. Iterative vs. Recursive Function does NOTcall itself • Iterative 1 if n=0 factorial(n) = n x (n-1) x (n-2) x … x 2 x 1 if n>0 • Recursive 1 if n=0 factorial(n) = n x factorial(n-1) if n>0 Function calls itself

45. Recursion • To see how the recursion works, let’s break down the factorial function to solve factorial(3)

46. Breakdown • Here, we see that we start at the top level, factorial(3), and simplify the problem into 3 x factorial(2). • Now, we have a slightly less complicated problem in factorial(2), and we simplify this problem into 2 x factorial(1).

47. Breakdown • We continue this process until we are able to reach a problem that has a known solution. • In this case, that known solution is factorial(0) = 1. • The functions then return in reverse order to complete the solution.

48. Breakdown • This known solution is called the base case. • Every recursive algorithm must have a base case to simplify to. • Otherwise, the algorithm would run forever (or until the computer ran out of memory).

49. Breakdown • The other parts of the algorithm, excluding the base case, are known as the general case. • For example: 3 x factorial(2)  general case 2 x factorial(1)  general case etc …

50. Iterative Algorithm factorial(n) { i = 1 factN = 1 loop (i <= n) factN = factN * i i = i + 1 end loop return factN } The iterative solution is very straightforward. We simply loop through all the integers between 1 and n and multiply them together.