Data Structure and Algorithms Design

1. Data Structure and Algorithms Design Dr. MaheswariKarthikeyan Lecture2

2. Asymptotic Notations A way to describe behavior of functions in the limit • How we indicate running times of algorithms • Describe the running time of an algorithm as n grows to  O notation: asymptotic “less than”: f(n) “≤” g(n)  notation: asymptotic “greater than”: f(n) “≥” g(n)  notation: asymptotic “equality”: f(n) “=” g(n)

3. Asymptotic Notations - Examples For each of the following pairs of functions, either f(n) is O(g(n)), f(n) is Ω(g(n)), or f(n) is Θ(g(n)). Determine which relationship is correct. • f(n) = log n2; g(n) = log n + 5 • f(n) = n; g(n) = log n2 • f(n) = log log n; g(n) = log n • f(n) = n; g(n) = log2 n f(n) =  (g(n)) f(n) = (g(n)) f(n) = O(g(n)) f(n) = (g(n))

4. Asymptotic notations O-notation • Intuitively: O(g(n)) = the set of functions with a smaller or same order of growth as g(n)

5. Examples 3n + 2 = O(n) ; 3n + 2 <= 4n for all n >= 2 3n + 3 = O(n) ; 3n + 3 <= 4n for all n >= 3 100n + 6 <= 101n for all n >= 6 100n + 6 = O(n) ; 10 n 2 + 4n + 2 = O(n2) 10 n 2 + 4n + 2 < = 11 n2 for n >= 5

6. Asymptotic notations (cont.)  - notation • Intuitively: (g(n)) = the set of functions with a larger or same order of growth as g(n)

7. Examples 3n + 2 >= 3n for all n >= 1 3n + 2 = ? 3n + 3 = ? 100n + 6 = ? 3n + 3 >= 3n for all n >= 1 100n + 6 >= 100n for all n >= 1

8. Asymptotic notations (cont.) -notation • Intuitively(g(n)) = the set of functions with the same order of growth as g(n)

9. Sorting Iterative methods: Insertion sort Bubble sort Selection sort • Non-comparison methods • Counting sort • Radix sort • Bucket sort • Divide and conquer • Merge sort • Quicksort

11. 24 10 6 Insertion Sort To insert 12, we need to make room for it by moving first 36 and then 24. 36 12

12. 24 10 6 Insertion Sort 36 12

13. 24 36 Insertion Sort 10 6 12

14. Insertion Sort input array 5 2 4 6 1 3 at each iteration, the array is divided in two sub-arrays: left sub-array right sub-array unsorted sorted

15. Insertion Sort

16. 1 2 3 4 5 6 7 8 a1 a2 a3 a4 a5 a6 a7 a8 key Insertion Sort Alg.:INSERTION-SORT(A) for j ← 2to n do key ← A[ j ] Insert A[ j ] into the sorted sequence A[1 . . j -1] i ← j - 1 while i > 0 and A[i] > key do A[i + 1] ← A[i] i ← i – 1 A[i + 1] ← key Insertion sort – sorts the elements in place

17. Analysis of Insertion Sort cost times c1 n c2 n-1 0 n-1 c4 n-1 c5 c6 c7 c8 n-1 INSERTION-SORT(A) for j ← 2 to n do key ← A[ j ] Insert A[ j ] into the sorted sequence A[1 . . j -1] i ← j - 1 while i > 0 and A[i] > key do A[i + 1] ← A[i] i ← i – 1 A[i + 1] ← key tj: # of times the while statement is executed at iteration j

18. Best Case Analysis The array is already sorted • A[i] ≤ key upon the first time the while loop test is run (when i= j -1) • tj= 1 T(n) = c1n + c2(n -1) + c4(n -1) + c5(n -1) + c8(n-1) = (c1 + c2 + c4 + c5 + c8)n + (c2 + c4 + c5 + c8) = an + b = (n) “while i > 0 and A[i] > key”

19. Worst Case Analysis The array is in reverse sorted order • Always A[i] > key in while loop test • Have to compare keywith all elements to the left of the j-th position  compare with j-1 elements  tj = j using we have: a quadratic function of n •  T(n) = (n2) order of growth in n2

20. 8 4 6 9 2 3 1 Selection Sort Alg.: SELECTION-SORT(A) n ← length[A] for j ← 1to n - 1 do smallest ← j for i ← j + 1to n do if A[i] < A[smallest] then smallest ← i exchange A[j] ↔ A[smallest]

21. n2/2 comparisons • n exchanges Analysis of Selection Sort cost times c1 1 c2 n c3 n-1 c4 c5 c6 c7 n-1 Alg.:SELECTION-SORT(A) n ← length[A] for j ← 1to n - 1 do smallest ← j for i ← j + 1to n do if A[i] < A[smallest] then smallest ← i exchange A[j] ↔ A[smallest]

22. Divide-and-Conquer Divide the problem into a number of sub-problems • Similar sub-problems of smaller size Conquer the sub-problems • Solve the sub-problems recursively • Sub-problem size small enough  solve the problems in straightforward manner Combine the solutions to the sub-problems • Obtain the solution for the original problem

23. Merge Sort Approach To sort an array A[p . . r]: Divide • Divide the n-element sequence to be sorted into two subsequences of n/2 elements each Conquer • Sort the subsequences recursively using merge sort • When the size of the sequences is 1 there is nothing more to do Combine • Merge the two sorted subsequences

24. Merge Sort r p q Alg.: MERGE-SORT(A, p, r) if p < rCheck for base case then q ← (p + r)/2Divide MERGE-SORT(A, p, q)Conquer MERGE-SORT(A, q + 1, r) Conquer MERGE(A, p, q, r)Combine Initial call:MERGE-SORT(A, 1, n) 1 2 3 4 5 6 7 8 5 4 7 1 3 2 6 2

25. 1 2 3 4 5 6 7 8 q = 4 5 2 4 7 1 3 2 6 1 2 3 4 5 6 7 8 5 2 4 7 1 3 2 6 1 2 3 4 5 6 7 8 5 2 4 7 1 3 2 6 5 1 2 3 4 6 7 8 5 2 4 7 1 3 2 6 Example – n Power of 2 Example

26. 1 2 3 4 5 6 7 8 1 2 2 3 4 5 6 7 1 2 3 4 5 6 7 8 2 4 5 7 1 2 3 6 1 2 3 4 5 6 7 8 2 5 4 7 1 3 2 6 5 1 2 3 4 6 7 8 5 2 4 7 1 3 2 6 Example – n Power of 2

27. 1 2 3 4 5 6 7 8 9 10 11 q = 6 4 7 2 6 1 4 7 3 5 2 6 1 2 3 4 5 6 7 8 9 10 11 q = 3 4 7 2 6 1 4 7 3 5 2 6 q = 9 1 2 3 4 5 6 7 8 9 10 11 4 7 2 6 1 4 7 3 5 2 6 1 2 3 4 5 6 7 8 9 10 11 4 7 2 6 1 4 7 3 5 2 6 1 2 4 5 7 8 4 7 6 1 7 3 Example – n Not a Power of 2

28. 1 2 3 4 5 6 7 8 9 10 11 1 2 2 3 4 4 5 6 6 7 7 1 2 3 4 5 6 7 8 9 10 11 1 2 4 4 6 7 2 3 5 6 7 1 2 3 4 5 6 7 8 9 10 11 2 4 7 1 4 6 3 5 7 2 6 1 2 3 4 5 6 7 8 9 10 11 4 7 2 1 6 4 3 7 5 2 6 1 2 4 5 7 8 4 7 6 1 7 3 Example – n Not a Power of 2

29. r p q 1 2 3 4 5 6 7 8 2 4 5 7 1 2 3 6 Merging Input: Array Aand indices p, q, rsuch that p ≤ q < r • Subarrays A[p . . q] and A[q + 1 . . r] are sorted Output: One single sorted subarray A[p . . r]

30. Merging Idea for merging: • Two piles of sorted cards • Choose the smaller of the two top cards • Remove it and place it in the output pile • Repeat the process until one pile is empty • Take the remaining input pile and place it face-down onto the output pile

31. p q r MERGE(A, 9, 12, 16)

32. Example: MERGE(A, 9, 12, 16)

33. Example (cont.)

34. Example (cont.)

35. Example (cont.) Done!

36. p q 2 4 5 7  L q + 1 r r p q 1 2 3 6  R 1 2 3 4 5 6 7 8 2 4 5 7 1 2 3 6 n2 n1 Merge - Pseudocode Alg.:MERGE(A, p, q, r) Compute n1and n2 Copy the first n1 elements into L[1 . . n1 + 1] and the next n2 elements into R[1 . . n2 + 1] L[n1 + 1] ← ;R[n2 + 1] ←  i ← 1; j ← 1 for k ← pto r do if L[ i ] ≤ R[ j ] then A[k] ← L[ i ] i ←i + 1 else A[k] ← R[ j ] j ← j + 1

37. Running Time of Merge Initialization (copying into temporary arrays): • (n1 + n2) = (n) Adding the elements to the final array (the last for loop): • n iterations, each taking constant time  (n) Total time for Merge: • (n)

38. Analyzing Divide-and Conquer Algorithms The recurrence is based on the three steps of the paradigm: • T(n) – running time on a problem of size n • Divide the problem into a subproblems, each of size n/b: takes D(n) • Conquer (solve) the subproblems aT(n/b) • Combine the solutions C(n) (1) if n ≤ c T(n) = aT(n/b) + D(n) + C(n) otherwise

39. MERGE-SORT Running Time Divide: • compute qas the average of pand r:D(n) = (1) Conquer: • recursively solve 2 subproblems, each of size n/2  2T (n/2) Combine: • MERGE on an n-element subarray takes (n) time  C(n) = (n) (1) if n =1 T(n) = 2T(n/2) + (n) if n > 1

40. Solve the Recurrence T(n) = c if n = 1 2T(n/2) + cn if n > 1 Use Master’s Theorem: Compare n with f(n) = cn Case 2: T(n) = Θ(nlgn)