Dr.Surasak Mungsing E-mail: Surasak.mu@spu.ac.th

1. CSC 201Analysis and Design of AlgorithmsLecture 05:Analysis of time Complexity of Sorting Algorithms Dr.SurasakMungsing E-mail: Surasak.mu@spu.ac.th

2. Sorting Algorithms • Bin Sort • Radix Sort • Insertion Sort • Shell Sort • Selection Sort • Heap Sort • Bubble Sort • Quick Sort • Merge Sort

4. D H E J B C I G D E A B J H I G C A F F 4 4 3 3 4 4 0 4 3 3 4 4 5 5 3 2 2 3 4 0 I J G H D C F A E B Bin 0 Bin 1 Bin 2 Bin 3 Bin 4 Bin 5 Bin Sort (a) Input chain (b) Nodes in bins (c) Sorted chain

5. 216 521 425 116 91 515 124 34 96 24 24 521 515 216 91 34 116 124 91 34 96 521 124 24 116 425 124 24 425 216 515 216 34 425 116 91 515 521 96 96 (a) Input chain (b) Chain after sorting on least significant digit (c) Chain after sorting on second-least significant digit (d) Chain after sorting on most significant digit Radix Sort with r=10 and d=3

6. Insertion Sort Concept

8. Shell Sort Algorithm

9. Shell Sort Algorithm

10. Shell Sort Algorithm

12. Shell Sort Demohttp://e-learning.mfu.ac.th/mflu/1302251/demo/Chap03/ShellSort/ShellSort.html

13. Selection Sort Concept

15. Heap Sort Algorithm

17. Bubble Sort Concept

19. 31 75 81 57 43 0 13 26 65 92 Select pivot 31 75 81 57 43 0 13 26 65 92 Partition 31 75 92 57 13 65 43 81 0 26 Quick sort The fastest known sorting algorithm in practice.

20. 31 75 92 57 13 65 43 81 0 26 Quick sort small Quick sort large 13 43 26 31 0 57 75 65 81 92 26 31 13 0 43 75 57 65 81 92 Quick sort

21. Quick sort

22. Quick Sort Partitions

23. Quick sort

24. External Sort: A simple merge

25. Merge Sort

26. Merge Sort

28. Sort Algorithm Analysis

29. Analysis of Insertion Sort for (int i = 1; i < a.length; i++) {// insert a[i] into a[0:i-1] int t = a[i]; int j; for (j = i - 1; j >= 0 && t < a[j]; j--) a[j + 1] = a[j]; a[j + 1] = t; }

30. Complexity • Space/Memory • Time • Count a particular operation • Count number of steps • Asymptotic complexity

31. Comparison Count for (int i = 1; i < a.length; i++) {// insert a[i] into a[0:i-1] int t = a[i]; int j; for (j = i - 1; j >= 0 && t < a[j]; j--) a[j + 1] = a[j]; a[j + 1] = t; }

32. Comparison Count for (j = i - 1; j >= 0 && t < a[j]; j--) a[j + 1] = a[j]; How many comparisons are made?

33. Comparison Count for (j = i - 1; j >= 0 && t < a[j]; j--) a[j + 1] = a[j]; number of compares depends on a[]s and t as well as on i

34. Comparison Count • Worst-case count = maximum count • Best-case count = minimum count • Average count

35. Worst-Case Comparison Count for (j = i - 1; j >= 0 && t < a[j]; j--) a[j + 1] = a[j]; a = [1, 2, 3, 4] and t = 0  4 compares a = [1,2,3,…,i] and t = 0  i compares

36. n  T(n) (i-1) = i=2 Worst-Case Comparison Count for (int i = 1; i < n; i++) for (j = i - 1; j >= 0 && t < a[j]; j--) a[j + 1] = a[j]; total compares = 1 + 2 + 3 + … + (n-1) = (n-1)n/2 = O(n2)

37. i-1 2 n  = T i=2 Average-Case Comparison Count for (int i = 1; i < n; i++) for (j = i - 1; j >= 0 && t < a[j]; j--) a[j + 1] = a[j]; = O(n2)

38. 31 75 81 57 43 0 13 26 65 92 Select pivot 31 75 81 57 43 0 13 26 65 92 Partition 31 75 92 57 13 65 43 81 0 26 Analysis of Quick Sort

39. private static void quicksort( Comparable [ ] a, int left, int right ) { /* 1*/ if( left + CUTOFF <= right ) { /* 2*/ Comparable pivot = median3( a, left, right ); // Begin partitioning /* 3*/ int i = left, j = right - 1; /* 4*/ for( ; ; ) { /* 5*/ while( a[ ++i ].compareTo( pivot ) < 0 ) { } /* 6*/ while( a[ --j ].compareTo( pivot ) > 0 ) { } /* 7*/ if( i < j ) /* 8*/ swapReferences( a, i, j ); else /* 9*/ break; } /*10*/ swapReferences( a, i, right - 1 ); // Restore pivot /*11*/ quicksort( a, left, i - 1 ); // Sort small elements /*12*/ quicksort( a, i + 1, right ); // Sort large elements } else // Do an insertion sort on the subarray /*13*/ insertionSort( a, left, right ); } MainQuick SortRoutine

40. n  i i=2 Worst-Case Analysis เวลาที่ใช้ในการ runquick sortเท่ากับเวลาที่ใช้ในการทำrecursive call2 ครั้ง + linear timeที่ใช้ในการเลือกpivot ซึ่งทำให้basic quicksort relation เท่ากับT(n) = T(i) + T(n-i-1) + cn ในกรณีWorst- caseเช่น การที่pivotมีค่าน้อยที่สุดเสมอ เวลาที่ใช้ในการทำrecursion คือT(n) = T(n-1) + cn n>1 T(n-1)= T(n-2)+c(n-1) T(n-2)= T(n-3)+c(n-2) … T(2) = T(1)+c(2) รวมเวลาทั้งหมดT(n) = T(1) + c = O(n2)

41. = + c = + c T(n/2) T(1) T(n/4) T(n/4) T(n/8) T(2) T(n) T(n/2) T(n) T(1) = + c n/2 n/4 n/4 n/8 n/2 1 2 n n 1 + c log n + c = = Best-Case Analysis The pivot is in the middle; T(n) = 2 T(n/2) + cn Divide both sides by n; Add all equations; … T(n) = c n logn + n = O(n log n)

42. + cN T(N) = Average timeofT(i)andT(N-i-1)is N -1 N -1 N -1    T( j ) T( j ) T( j ) j=0 j=0 j=0 + cN2 2 NT(N) = 2 (N-1) T(N-1) = + c(N – 1)2 1 2 T( j ) N N N -2  j=0 Average-Case Analysis (1/4) Total time; T(N) = T(i) + T(N-i-1) + c N…………..(1) …………..(2) Therefore …………..(3) …………..(4) …………..(5)

43. Rearrange terms in equation and ignore c onthe right-hand side; NT(N) = (N+1) T(N-1) +2cN …………..(7) …………..(8) (7) divides by N(N+1); T(N-1) 2c T(N) + = N N+1 N+1 Average-Case Analysis(2/4) …………..(6) NT(N) – (N-1) T(N-1) = 2T(N-1) +2cN - c (5)– (4);

44. …………..(8) …………..(9) …………..(10) . . . …………..(11) T(N-1) T(N-1) T(N-2) T(N-3) T(1) 2c 2c 2c 2c T(N-2) T(N) T(2) T(1) T(N) + + + + = = = = + = 2c N-1 N-2 N 2 N+1 N-1 N 3 N+1 N-1 N 3 2 N+1 N +1  i=3 Average-Case Analysis(3/4) Sun equations (8)to(11); …………..(12)

45. …………..(12) Sum in equation (12)ia approximately logC(N+1)+  - 3/2, whichisEuler’s constant  0.577 …………..(13) therefore T(1) T(N) T(N) + = 2c and =O(Nlog N) T(N) …………..(14) =O(log N) 2 N+1 N+1 N +1  i=3 Average-Case Analysis(4/4) In summary, time complexity ofQuick sort algorithmforAverage-Case is T(n) = O(n log n)