Comprehensive Guide on Searching and Sorting Algorithms

1. Searching and Sorting Gary Wong

2. Prerequisite • Time complexity • Pseudocode • (Recursion)

3. Searching Linear (Sequential) Search Binary Search Sorting Bubble Sort Merge Sort Quick Sort Counting Sort Agenda

4. Linear Search One by one...

5. Linear Search • Check every element in the list, until the target is found • For example, our target is 38: Not found! Found!

6. Linear Search • Initilize an index variable i • Compare a[i] with target • If a[i]==target, found • If a[i]!=target, • If all have checked already, not found • Otherwise, change i into next index and go to step 2

7. Linear Search • Time complexity in worst case? • If N is number of elements, • Time complexity = O(N) • Advantage? • Disadvantage?

8. Binary Search Chop by half...

9. L R Binary Search • Given a SORTED list: • (Again, our target is 38) Smaller! Found! Larger!

10. Binary Search • Why always in the middle, but not other positions, say one-third of list? 1) Initialize boundaries L and R 2) While L is still on the left of R • mid = (L+R)/2 • If a[mid]>Target, set R be m-1 and go to step 2 • If a[mid]<Target, set L be m+1 and go to step 2 • If a[mid]==Target, found

11. Binary Search • Time complexity in the worst case? • If N is the number of elements, • Time complexity = O(lg N) • Why? • Advantage? • Disadvantage?

12. Example: Three Little Pigs • HKOI 2006 Final Senior Q1 • Given three lists, each with M numbers, choose one number from each list such that their sum is maximum, but not greater than N. • Constraints: • M ≤ 3000 • Time limit: 1s

13. Example: Three Little Pigs • How many possible triples are there? • Why not check them all? • Time complexity? • Expected score = 50

14. Example: Three Little Pigs • A simpler question: How would you search for the maximum number in ONE SORTED list such that it is not greater than N? • Binary search! • With slight modification though • How?

15. L R Example: Three Little Pigs • Say, N=37

16. Example: Three Little Pigs • Let’s go back to original problem • If you have chosen two numbers a1[i] and a2[j] already, how would you search for the third number? • Recall: How would you search for the maximum number in ONE SORTED list such that it is not greater than N-a1[i]-a2[j]?

17. Example: Three Little Pigs • Overall time complexity? • Expected score = 90 • 

18. Example: Three Little Pigs • Slightly harder: Given TWO lists, each with M numbers, choose one number from each list such that their sum is maximum, but not greater than N. • Linear search? • Sort one list, then binary search! • Time complexity = O(M lg M) • O(M2) if less efficient sorting algorithm is used • But, can it be better?

19. Example: Three Little Pigs • Why is it so slow if we use linear search? • If a1[i] and a2[j] are chosen, and their sum is smaller than N: • Will you consider any number in a1 that is smaller than or equal a1[i]? • If a1[i] and a2[j] are chosen, and their sum is greater than N: • Will you consider any number in a2 that is greater than or equal to a2[j]?

20. Example: Three Little Pigs • Recall: Why is it so slow if we use linear search? • Because you use it for too many times! • At which number in each list should you begin the linear search? • Never look back at those we do not consider! • Time complexity? • Expected score = 100 

21. What can you learn? • Improve one ‘dimension’ using binary search • Linear search for a few times can be more efficient than binary search for many times! • DO NOT underestimate linear search!!!

22. Points to note • To use binary search, the list MUST BE SORTED (either ascending or decending) • NEVER make assumptions yourself • Problem setters usually do not sort for you • Sorting is the bottleneck of efficiency • But... how to sort?

23. How to sort? • For C++: sort() • Time complexity for sort() is O(N lg N) • which is considered as efficient • HOWEVER... • Problem setters SELDOM test contestants on pure usage of efficient sorting algorithms • Special properties of sorting algorithms are essential in problem-solving • So, pay attention! 

24. Bubble Sort Smaller? Float! Larger? Sink!

25. Bubble Sort • Suppose we need to sort in ascending order... • Repeatedly check adjacent pairs sequentially, swap if not in correct order • Example: • The last number is always the largest 18 9 20 11 77 45 Incorrect order, swap! Correct order, pass!

26. Bubble Sort • Fix the last number, do the same procedures for first N-1 numbers again... • Fix the last 2 numbers, do the same procedures for first N-2 numbers again... • ... • Fix the last N-2 numbers, do the same procedures for first 2 numbers again...

27. Bubble Sort for i -> 1 to n-1 for j -> 1 to n-i if a[j]>a[j+1], swap them • How to swap?

28. Merge Sort & Quick Sort Many a little makes a mickle...

29. Merge Sort • Now given two SORTED list, how would you ‘merge’ them to form ONE SORTED list? List 1: 8 14 22 List 2: 10 13 29 65 Temporary list: 8 8 10 10 13 13 14 14 22 22 29 29 65 65 Combined list:

30. Merge Sort Merge • While both lists have numbers still not yet considered • Compare the current first number in two lists • Put the smaller number into temporary list, then discard it • If list 1 is not empty, add them into temporary list • If list 2 is not empty, add them into temporary list • Put the numbers in temporary list back to the desired list

31. Merge Sort • Suppose you are given a ‘function’ called ‘mergesort(L,R)’, which can sort the left half and right half of list from L to R: • How to sort the whole list? • Merge them! • How can we sort the left and right half? • Why not making use of ‘mergesort(L,R)’? 10 13 29 65 8 14 22 L (L+R)/2 (L+R)/2+1 R

32. Merge Sort mergesort(L,R){ If L is equal to R, done; Otherwise, m=(L+R)/2; mergesort(L,M); mergesort(M+1,R); Merge the lists [L,M] and [M+1,R]; }

33. Merge Sort mergesort(0,6) mergesort(0,1) mergesort(2,3) mergesort(4,5) mergesort(6,6) 65 10 29 13 14 8 22 mergesort(0,0) mergesort(1,1) mergesort(2,2) mergesort(3,3) mergesort(4,4) mergesort(5,5) mergesort(0,3) mergesort(4,6) 8 10 10 65 13 10 13 29 13 14 29 65 22 8 8 14 29 14 65 22

34. Merge Sort • Time complexity? • O(N lg N) • Why?

35. Quick Sort • Choose a number as a ‘pivot’ • Put all numbers smaller than ‘pivot’ on its left side • Put all numbers greater than (or equal to) ‘pivot’ on its right side 10 13 29 65 8 14 22 10 13 8 14 22 65 29

36. x y Quick Sort a[y] < pivot! shift x, swap! • How? • y shifts to right by 1 unit in every round • Check if a[y] < pivot • If yes, shift x to right by 1 unit and swap a[x] and a[y] • If y is at 2nd last position, swap a[x+1] and pivot • Time complexity? 10 13 29 65 8 14 22

37. Quick Sort • Use similar idea as in merge sort • If we have a function called ‘quicksort(L,R)’... • Make use of ‘quicksort(L,R)’ to sort the two halves! 10 13 8 14 22 65 29

38. Quick Sort quicksort(L,R){ If L is equal to R, done; Otherwise, Choose a number as a pivot, say a[p]; Perform pivoting action; quicksort(L,p-1); quicksort(p+1,R); }

39. Quick Sort • Time complexity in worst case? • O(N2)  • What is the worst case? • Preventions: • Choose a random number as pivot • Pick 3 numbers, choose their median • Under randomization, expected time complexity is O(N lg N)

40. Counting Sort No more comparison...

41. Counting Sort • Create a list, then tally! • Count the occurences of elements • Example: • Sort {2,6,1,4,2,1,9,6,4,1} in ascending order • A list from 1 to 9 (or upper bound of input no.) • Three ‘1’s, two ‘2’s, two ‘4’s, two ‘6’s and one ‘9’ • List them all!

42. Counting Sort • Time complexity? • O(range * N) • Good for numbers with small range

43. More... If we have time...

44. More... • Lower bound of sorting efficiency?! • Guess the answer by binary search • Count the number of ‘inversions’ • Find the kth largest number • Other sorting algorithms

45. Time for lunch Yummy!