1 / 42

LECTURE 6: Analysis of sorting methods

LECTURE 6: Analysis of sorting methods. Outline. The problem of sorting Insertion sort – correctness and efficiency analysis Selection sort – correctness and efficiency analysis Bubble sort – correctness and efficiency analysis. The problem of sorting. Let us consider:

brian-blevins
Download Presentation

LECTURE 6: Analysis of sorting methods

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. LECTURE 6: Analysis of sorting methods Algorithmics - Lecture 6

  2. Outline • The problem of sorting • Insertion sort – correctness and efficiency analysis • Selection sort – correctness and efficiency analysis • Bubble sort – correctness and efficiency analysis Algorithmics - Lecture 6

  3. The problem of sorting Let us consider: • A sequence of items • Each item has a characteristic called sorting key. The values of the sorting key belong to a set on which a total order relationship exists • Sorting the sequence = arrange its elements such that the sorting keys are in increasing (or decreasing) order Algorithmics - Lecture 6

  4. The problem of sorting Examples: • Sequence of numbers: (5,8,3,1,6) The sorting key is the entire entity (an integer value) The result of the sorting process is (1,3,5,6,8) 2. Sequence of records containing two fields (name and mark): ((Peter,9), (John, 10), (Victor,8), (Adam,9)) There are two possible criteria of sorting here: name and mark Increasing sorting by name: ((Adam,9),(John,10),(Peter,9),(Victor,8)) Decreasing sorting by mark: ((John,10),(Peter,9),(Adam,9),(Victor,8)) Algorithmics - Lecture 6

  5. The problem of sorting More formally … Ordering (increasingly) a sequence (x1,x2,…,xn) is equivalent to finding a permutation (p(1),p(2),…,p(n))of the indices such that key(xp(1))<=key(xp(2)) <= … <=key(xp(n)) In the following we shall consider that the sorting key is the element itself, thus the following condition is satisfied: xp(1)<=xp(2) <= … <=xp(n) Algorithmics - Lecture 6

  6. The problem of sorting Other assumptions: • We shall consider that the sequence is stored in a random access memory (e.g. in an array) This means that we will discuss about internal sorting • We shall analyze only sorting methods which are in place (the additional space needed for sorting has at most the size of an element). Algorithmics - Lecture 6

  7. Properties of sorting methods: • Simplicity The method is simple if it is easy to understand and to implement • Efficiency The method is efficient if it doesn’t use a large amount of resources (running time) • Naturalness The method is natural if the number of operations depends on how out of order is the initial sequence • Stability The method is stable if it preserves the initial ordering of any two elements having identical keys Algorithmics - Lecture 6

  8. Stability Example: Initial configuration: ((Adam,9), (John, 10), (Peter,9), (Victor,8)) Stable sorting : ((John,10),(Adam,9),(Peter,9),(Victor,8)) Unstable sorting : ((John,10), (Peter,9),(Adam,9), (Victor,8)) Algorithmics - Lecture 6

  9. Basic sorting methods Are simple, intuitive but not very efficient methods… However, they are the starting point for advanced methods. Examples: • Insertion sort • Selection sort • Bubble sort Algorithmics - Lecture 6

  10. Insertion sort Basic idea: Starting with the second element of the array, each element is inserted in the subsequence which precedes it such that this subsequence is kept sorted (x1,x2,…,xi-1,xi,xi+1,…xn) Already sorted subsequence (destination subsequence) Element to be inserted Algorithmics - Lecture 6

  11. Insertion sort - example WHILE (j>=1) AND (aux<x[j]) j i-1 i-2 i-3 i-4 i aux=5 2 8 5 9 8 1 88 9 8 1 5 8 9 8 1 FOR i:=2,n aux=9 3 5 89 8 1 5 89 8 1 aux=8 4 5 898 1 5 899 1 5 889 1 aux=1 5 5 889 1 5 889 9 5 888 9 5 888 9 5 5 88 9 1 5 88 9 Algorithmics - Lecture 6

  12. Insertion sort – algorithm Algorithm Insertion(x[1..n]) FOR i:=2,n DO aux:=x[i] j:=i-1 WHILE (j>=1) AND (aux<x[j)) DO x[j+1]:=x[j] j:=j-1 ENDWHILE x[j+1]:=aux ENDFOR RETURN x[1..n] General structure FOR i:=2,n DO <insert x[i] in the already sorted subarray x[1..i-1] such that x[1..i] is also sorted> ENDFOR Algorithmics - Lecture 6

  13. Insertion sort – a variant Insertion(x[1..n]) FOR i:=2,n DO aux:=x[i] j:=i-1 WHILE (j>=1) AND (aux<x[j]) DO x[j+1]:=x[j] j:=j-1 ENDWHILE x[j+1]:=aux ENDFOR RETURN x[1..n] Insertion(x[1..n]) FOR i:=2,n DO x[0]:=x[i] j:=i-1 WHILE (x[0]<x[j]) DO x[j+1]:=x[j] j:=j-1 ENDWHILE x[j+1]:=x[0] ENDFOR RETURN x[1..n] Algorithmics - Lecture 6

  14. Insertion sort – correctness analysis Insertion(x[1..n]) i:=2 {x[1..i-1] sorted} WHILE i<=n aux:=x[i] j:=i-1 {x[1..j] sorted, aux<=x[j+1]<=..<=x[i]} WHILE (j>=1) AND (aux<x[j]) DO x[j+1]:=x[j] {x[1..j-1] sorted, aux<x[j]=x[j+1]<= … <=x[i]} j:=j-1 {x[1..j] sorted, aux<x[j+1]=x[j+2]<= … <=x[i]} Either {aux>=x[j] and x[1]<=…x[j]<=aux<x[j+1]=x[j+2]<=…<=x[i]} or {(j=0) and aux<x[1]=x[2]<=…<=x[i]} ENDWHILE x[j+1]:=aux {x[1]<=x[2]<=…x[j+1]<=x[j+2]<=…<=x[i]} (x[1..i] sorted) i:=i+1 {x[1..i-1] sorted} ENDWHILE RETURN x[1..n] Algorithmics - Lecture 6

  15. Insertion sort – correctness analysis Thus we obtained … An invariant for the outer loop: {x[1..i-1] sorted} An invariant for the inner loop: {x[1..j] sorted, aux<=x[j+1]<=x[j+2]<=..<=x[i]} Both loops are finite: termination function for the outer loop: t(p)=n+1-ip termination function for the inner loop: jp if aux <x[jp] t(p)= 0 if aux>=x[jp] Algorithmics - Lecture 6

  16. Insertion sort – efficiency analysis • Input size: n • Operations: • Comparisons (TC(n)) • Movements (of the elements) (TM(n)) • For each i (2 <=i <= n): • 1 <=TC(n) <=i • For all i: • (n-1) <=TC(n) <=(n+2)(n-1)/2 Insertion(x[1..n]) FOR i:=2,n DO x[0]:=x[i] j:=i-1 WHILE x[0]<x[j] DO x[j+1]:=x[j] j:=j-1 ENDWHILE x[j+1]:=x[0] ENDFOR RETURN x[1..n] Algorithmics - Lecture 6

  17. Insertion sort – efficiency analysis • Input size: n • Operations: • Comparisons (TC(n)) • Movements (of the elements) (TM(n)) • For each i (2 <=i <= n): • 0+2 <=TM(n) <=(i-1)+2=i+1 • For all i: • 2 (n-1) <=TM(n) <=(n+1)(n+2)/2-3 Insertion(x[1..n]) FOR i:=2,n DO x[0]:=x[i] j:=i-1 WHILE x[0]<x[j] DO x[j+1]:=x[j] j:=j-1 ENDWHILE x[j+1]:=x[0] ENDFOR RETURN x[1..n] Algorithmics - Lecture 6

  18. Insertion sort – efficiency analysis Comparisons: (n-1) <=TC(n) <=(n+2)(n-1)/2 Movements: 2 (n-1) <=TM(n) <=(n+1)(n+2)/2-3 Total: 3(n-1) <=T(n) <=n2+2n-3 Efficiency classes: T(n) Ω(n) T(n)  O(n2) Algorithmics - Lecture 6

  19. Insertion sort – stability 8 5 9 8’ 1 5 8 9 8’ 1 5 89 8’ 1 5 89 8’ 1 5 898’ 1 5 88’9 1 1 5 88’ 9 5 88’9 1 The insertion method is stable Algorithmics - Lecture 6

  20. Outline • The problem of sorting • Insertion sort • Selection sort • Bubble sort Algorithmics - Lecture 6

  21. Selection sort Basic idea: For each position, i (starting with the first one) the subsequence x[i..n] is scanned in order to identify the minimum. This minimum is swapped with the i-th element. (x1,x2,…,xi-1,xi,xi+1,…xn) Already sorted subsequence Subsequence searched for the minimum The position where the minimum will be placed Algorithmics - Lecture 6

  22. Selection sort FOR j:=i+1,n j i i+1 5 … i 1 85 9 8 1 859 8 1 8 5 9 8 1 8 5 9 8 1 8 5 9 8 1 FOR i:=1,n-1 2 1 5 9 8 8 1 59 8 8 1 5 9 8 8 1 5 9 8 8 3 1 59 8 8 1 5 9 8 8 15 9 8 8 4 1 589 8 1 5 8 9 8 158 89 Algorithmics - Lecture 6

  23. Selection sort Algorithm Selection (x[1..n]) FOR i:=1,n-1 DO k:=i; FOR j:=i+1,n DO IF x[k]>x[j] THEN k:=j ENDIF ENDFOR IF k<>i THEN x[i]<->x[k] ENDIF ENDFOR RETURN x[1..n] General structure FOR i:=1,n-1 DO<find the minimum of x[i..n] and swap it with x[i]> ENDFOR Algorithmics - Lecture 6

  24. Selection sort – correctness analysis Algorithm Selection (x[1..n]) i:=1 {x[1..i-1] sorted and x[i-1]<=x[j], j=i..n} WHILE i<=n-1 DO k:=i j:=i+1 {x[k]<=x[r] for all r=i+1..j-1} WHILE j<=n DO IF x[k]>x[j] THEN k:=j ENDIF j:=j+1 ENDWHILE {x[k]<=x[r] for all r=i+1..n} IF k<>i THEN x[i]<->x[k] {x[1..i] sorted and x[i]<=x[j], j=i+1..n} i:=i+1 ENDWHILE {x[1..i-1] sorted and x[i-1]<=x[j], j=i..n} RETURN x[1..n] Algorithmics - Lecture 6

  25. Selection sort – correctness analysis Thus we obtained … An invariant for the outer loop: {x[1..i-1] sorted and x[i-1]<=x[j], j=i..n} An invariant for the inner loop: {x[k]<=x[r] for all r=i+1..j-1} Both loops are finite: termination function for the outer loop: t(p)=n-ip termination function for the inner loop: t(p)=n+1-jp Algorithmics - Lecture 6

  26. Selection sort – efficiency analysis Selection (x[1..n]) FOR i:=1,n-1 DO k:=i FOR j:=i+1,n DO IF x[k]>x[j] THEN k:=j ENDFOR IF k<>i THEN x[i]<->x[k] ENDFOR RETURN x[1..n] Input size: n Operations: • Comparisons (TC(n)) • Movements (of the elements) (TM(n)) For each i (1 <=i <= n-1): TC(n,i) =n-i For all i: TC(n) =n(n-1)/2 Algorithmics - Lecture 6

  27. Selection sort – efficiency analysis Selection (x[1..n]) FOR i:=1,n-1 DO k:=i FOR j:=i+1,n DO IF x[k]>x[j] THEN k:=j ENDFOR IF k<>i THEN x[i]<->x[k] ENDFOR RETURN x[1..n] Input size: n Operations: • Comparisons (TC(n)) • Movements (of the elements) (TM(n)) For each i (2 <=i <= n): 0 <= TM(n,i) <=3 For all i: 0 <= TM(n) <= 3(n-1) Algorithmics - Lecture 6

  28. Selection sort – efficiency analysis Comparisons: TC(n) =n(n-1)/2 Movements: 0 <=TM(n) <= 3(n-1) Total: n(n-1)/2 <=T(n) <=n(n-1)/2+3(n-1) Efficiency class: T(n) (n2) Algorithmics - Lecture 6

  29. Selection sort - stability 85 9 8’1 859 8’1 8 5 9 8’1 8 5 9 8’1 8 5 9 8’1 1 5 9 8’8 1 59 8’8 1 5 9 8’8 1 5 9 8’8 1 59 8’8 1 5 9 8’8 15 9 8’8 1 58’9 8 1 5 8’9 8 158’89 The method is not stable Algorithmics - Lecture 6

  30. Outline • The problem of sorting • Insertion sort • Selection sort • Bubble sort Algorithmics - Lecture 6

  31. Bubble sort Basic idea: The array is scanned from left to right and the adjacent elements are compared. If they are out of order then they are swapped. The scanning process is repeated until the array is sorted. (x1,x2,…,xm-1,xm,xm+1,…xn) Already sorted subsequence Algorithmics - Lecture 6

  32. Bubble sort FOR j:=1,i-1 j 1 2 … 4 i 5 85 9 8 1 589 8 1 5 8 98 1 5 8 8 91 5 8 8 19 FOR i:=n,2,-1 4 58 8 1 9 5 8 8 1 9 5 8 8 19 5 818 9 3 5 8 18 9 5 8 1 8 9 5188 9 2 5 1 88 9 1 5 8 9 8 Algorithmics - Lecture 6

  33. Bubble sort Algorithm Bubblesort(x[1..n]) FOR i:=n,2,-1 DO FOR j:=1,i-1 DO IF x[j]>x[j+1] THEN x[j]<->x[j+1] ENDIF ENDFOR ENDFOR RETURN x[1..n] General structure FOR i:=n,2,-1 DO < scan x[1..i-1], compare the adjacent elements and swap them if they are out of order> ENDFOR Algorithmics - Lecture 6

  34. Bubble sort – correctness analysis Bubble(x[1..n]) i:=n {x[i..n] is sorted and x[i]>=x[j], j=1..i} WHILE i>=2 DO j:=1 {x[j]>=x[k], k=1..j-1} WHILE j<=i-1 DO IF x[j]>x[j+1] THEN x[j]<->x[j+1] {x[j+1]>=x[k], k=1..j} j:=j+1 {x[j]>=x[k], k=1..j-1} ENDWHILE {x[i-1]>=x[j], j=1..i-1} {x[i-1..n] is sorted and x[i-1]>=x[j], j=1..i-1} i:=i-1 ENDWHILE {x[i..n] is sorted and x[i]>=x[j], j=1..i} RETURN x[1..n] Algorithmics - Lecture 6

  35. Bubble sort – correctness analysis Thus we obtained … An invariant for the outer loop: {x[i..n] is sorted and x[i]>=x[j], j=1..i} An invariant for the inner loop: {x[j]>=x[k], k=1..j-1} Both loops are finite: termination function for the outer loop: t(p)=ip-1 termination function for the inner loop: t(p)=ip-jp Algorithmics - Lecture 6

  36. Bubble sort – efficiency analysis Bubblesort(x[1..n]) FOR i:=n,2,-1 DO FOR j:=1,i-1 DO IF x[j]>x[j+1] THEN x[j]<->x[j+1] ENDIF ENDFOR ENDFOR RETURN x[1..n] Input size: n Operations: Comparisons (TC(n)) Movements (of the elements) (TM(n)) For each i (1 <=i <= n-1): TC(n,i) =i-1 For all i: TC(n) =n(n-1)/2 Algorithmics - Lecture 6

  37. Bubble sort – efficiency analysis Bubblesort(x[1..n]) FOR i:=n,2,-1 DO FOR j:=1,i-1 DO IF x[j]>x[j+1] THEN x[j]<->x[j+1] ENDIF ENDFOR ENDFOR RETURN x[1..n] Input size: n Operations: Comparisons (TC(n)) Movements (of the elements) (TM(n)) For each i (2 <= i <= n): 0 <=TM(n,i) <= 3(i-1) For all i: 0 <=TM(n) <= 3n(n-1)/2 Algorithmics - Lecture 6

  38. Bubble sort – efficiency analysis Comparisons: TC(n) =n(n-1)/2 Movements: 0 <=TM(n) <= 3n(n-1)/2 Total: n(n-1)/2 <= T(n) <= 2n(n-1) Efficiency classes: T(n) (n2) Algorithmics - Lecture 6

  39. Bubble sort – other variants Idea: scan the array only up to the index of the last swap Bubblesort(x[1..n]) t:=n WHILE t>1 m:=t; t:=0; FOR j:=1,m-1 DO IF x[j]>x[j+1] THEN x[j]<->x[j+1]; t:=j ENDIF ENDFOR ENDWHILE RETURN x[1..n] n-1<=TC(n)<=n(n-1)/2 0<=TM(n)<=3n(n-1)/2 Idea: repeat the array scanning only if at the last scan at least one swap has been made Bubblesort(x[1..n]) sw:=TRUE WHILE sw=True DO sw:=FALSE FOR j:=1,n-1 DO IF x[j]>x[j+1] THEN x[j]<->x[j+1] sw:=TRUE ENDIF ENDFOR ENDWHILE RETURN x[1..n] n-1<=TC(n)<=n(n-1) 0<=TM(n)<=3n(n-1)/2 Algorithmics - Lecture 6

  40. Bubble sort - stability 85 9 8’ 1 589 8’ 1 5 8 98’ 1 5 8 8’91 5 8 8’ 19 58 8’1 9 5 8 8’1 9 5 8 8’19 5 818’ 9 5 8 18’ 9 5 8 1 8’ 9 5188’ 9 5 1 88’ 9 1 5 8 9 8’ Bubble sort is stable Algorithmics - Lecture 6

  41. Some useful (or just interesting) links… http://www.sorting-algorithms.com/ http://www.softpanorama.org/Algorithms/sorting.shtml http://www.brian-borowski.com/Software/Sorting/ http://www.youtube.com/watch?v=INHF_5RIxTE ☺ http://boingboing.net/2010/08/19/what-does-a-bubble-s.html ☺ Algorithmics - Lecture 6

  42. Next lecture will be on … … algorithm design techniques … recursive algorithms … decrease and conquer Algorithmics - Lecture 6

More Related