Algorithms

1. Algorithms

2. Algorithms An algorithm is a computable (i.e. finite) set of steps to achieve a desired result. To analyse an algorithm, we have to consider its: effectiveness (whether it can be rendered as a computer program), correctness (whether it can produce a correct result), termination (whether it can arrive at an answer), efficiency (how much resource is consumed to produce a correct result), program complexity (whether it is easy to understand). In our context, we mainly concern the run-time efficiency of an algorithm.

3. Run-time efficiency of an algorithm If the run-time of a certain searching algorithm is approximately proportional to the number of items in the list to be searched, we say that the run-time efficiency isin the order of n. (e.g. sequential search) If it is approximately proportional to the square of the number of items in the list to be searched, we say that the run-time efficiency is in the order of n2. (e.g. bubble sort) … and so on.

4. Searching • Searching means to scan through lists of data until one or more items of data that match some specific criteria is found. • Efficiency of searching techniques are compared by considering the average number of comparisons that must be made to get the desired item (i.e. the average search length). • In addition, the search time(the average time taken to search for a given item) is also an important factor which determines the efficiency of an algorithm.

5. Sequential search Sequential searchexamines the data items in the order they appear in the list until the desired value is located or the end of the list is reached. Average search length for a list of n data items =(1+2+ … +n)/n =(n+1)/2 run-time efficiency is in the order of n.

6. Sequential search function SequentialSearch(list: ArrayType; size: integer;  target: ElementType): integer; var i: integer; begin   i := 1;    while (i < size) and (target <> list[i]) do      i := i + 1;    if target = list[i] then      SequentialSearch := i    else      SequentialSearch := 0 end; If the list is already sorted in ascending order, the second condition in the while statement can be changed to (target > list[i]).

7. Binary search Binary search cuts the list of data items (which must be sorted) in half repeatedly until the search value is found or the list is fully searched. e.g. Run-time efficiency: inthe order of (log2n)

8. Binary search function BinarySearch(list: ArrayType; size: integer;            target: ElementType): integer; var first, last, mid, location: integer; begin    first := 1;    last := size;    location := 0;    while (first <= last) and (location = 0) do begin      mid := (first + last) div 2;      if target = list[mid] then        location := mid      else if target < list[mid] then        last := mid - 1      else        first := mid + 1    end;    BinarySearch := location end;

9. Hashing Hashing is a scheme for providing rapid access to data items which are distinguished by some key.Each data item to be stored is associated with a key, e.g. the name of a person. A hash function is applied to the item’s key and the resulting hash value is used as an index to select one of a number of ‘hash buckets’ (blocks of storage) in a hash table. The table contains pointers to the original items.

10. Common hash functions - Mid-square Mid-square: This function is computed by squaring the key and then using an appropriate number of bits from the middle of the square to obtain the bucket address (the key is assumed to fit into one computer word). e.g. 19762=3904576

11. Common hash functions - Division Division: The key X is divided by some number M and the remainder is used as the hash address for X. e.g. 1976 div 97 = 36

12. Common hash functions - Folding Folding: The key X is partitioned into several parts, all but the last being of the same length. These parts are then added together to obtain the hash address for X. e.g. if X = 1232032411220, and the length of each part is 3 digits, then the hash address = 123 + 203 + 241 + 112 + 20 = 699.

13. Hashing If two items’ keys hash to the same value (a ‘hash collision’) then some alternative location is used. (e.g. linear probing: the next free location cyclically following the indicated one). For best performance, the table size and hash function must be tailored to the number of entries and range of keys to be used. The hash function usually depends on the table size so if the table needs to be enlarged it must usually be completely rebuilt.

14. Run-time efficiency of hashing Run-time efficiency: in the order of 1 (best case). However, a collision lead to extra searching time. The memory requirement for implementing hashing (search) is relatively high.

15. Sorting algorithms Sorting data requires putting the raw data in some predetermined order which might be alphabetical if the data is a set of words, or it might be ascending or descending order if the data is numerical.

16. Sorting algorithms Sorting algorithms depend very much on the way in which the data is stored, and on particular aspects of hardware. The following factors, together with considerations such as speed and amount of memory used, plays a part in deciding the most suitable sorting algorithm to use. • Is the data in the main memory? • Is it on disk or on tape? • Can all the data fit into the main memory at a time?

17. Sorting algorithms Sorting methods can be characterised into two broad categories: • Internal sorting methods, which are to be used when the file to be sorted is small enough so that the entire sort can be carried out in main memory. • External sorting methods, which are to be used upon large files, where data are directly stored on secondary storage (such as magnetic disk and magnetic tape).

18. Bubble sort procedure BubbleSort(var a: list; n: integer); var   i, pass: integer; begin    for pass := 1 to n - 1 do      for i := 1 to n - pass do        if a[i] > a[i + 1] then          swap(a[i], a[i + 1]) end; Run-time efficiency: in the order of n2

19. Insertion sort This algorithm sorts all data into sequence by inserting elements to suitable positions. In the nth pass, we investigate the sublist consisting of the first (n+1) elements. The (n+1)th element is inserted to its appropriate position in the sublist.

20. Insertion sort - Example

21. Insertion sort - Algorithm procedure InsertionSort(var a: list; n: integer); var    i, j, x: integer; begin    for i := 2 to n do begin      x := a[i];      j := i - 1;      while (j > 0) and (x < a[j]) do begin        a[j + 1] := a[j];        j := j - 1      end;      a[j + 1] := x    end end; Run-time efficiency: in the order of n2

22. Quick sort This algorithm sorts by splitting the list of data to two sublists and then sorts the two sublists recursively. A value called the pivot is used to split the list into two sublists. • The pivot can be any value of the same type as that of the elements of the list (not necessary chosen from the list). However, the efficiency is the highest if the pivot is the median of the elements to be sorted. Run-time efficiency: in the order of (nlogn)

23. Quick sort Example

24. Quick sort Example

25. Quick sort Example

26. Quick sort Example

27. Quick sort Example

28. Quick sort Example

29. Quick sort Example

30. Quick sort Example

31. Quick sort Algorithm procedure QuickSort(var a: list; n: integer);     procedure QSort(L, R: integer);    var  i, j, k, x: integer;    begin      i := L;  j := R;      x := Pivot(L, R);    { Find a pivot for the sublist a[L..R] }      repeat        while a[i] < x do i := i + 1;        while a[j] > x do j := j - 1;        if i <= j then begin          swap(a[i], a[j]); i := i + 1;  j := j - 1        end      until i > j;       if j - L >= 1 then QSort(L, j); { Recursively sort the left sublist }     if R - i >= 1 then QSort(i, R); { Recursively sort the right sublist }    end; begin                        { QuickSort }   QSort(1, n); end;

32. Merge sort This algorithm sorts all data into sequence by merging each pair of two sorted sublists into one recursively. In merge sort, extra space has to be allocated to temporarily store the merged list (note that in other sorting algorithms, there is no need to allocate extra blocks of memory for temporary list storage), thus merge sort is regarded as external sorting method. Run-time efficiency: in the order of (nlogn)

33. Merge sort - Example

34. Merge sort - Example

35. Merge sort - Example

36. Merge sort - Example

37. Merge sort - Algorithm procedure MergeSort(var a: list; n: integer);   procedure MSort(L, R: integer);    var mid: integer;    begin      if L < R then begin        mid := (L + R) div 2;        MSort(L, mid); { Recursively sort the left sublist }        MSort(mid + 1, R);{ Recursively sort the right sublist }        Merge(L, mid, mid + 1, R)  { Merge the two sublists }     end    end; begin                            { MergeSort }    MergeSort(1, n); end;

38. Selection sort This algorithm sorts all data into sequence by exchanging the largest and the last element at each pass (assuming sorting to ascending order). Run-time efficiency: in the order of n2

39. Selection sort – Example

40. Selection sort – Example

41. Selection sort – Algorithm procedure SelectionSort(var a: list; n: integer); var    i, j, pos, largest: integer; begin    for i := n downto 2 do begin      largest := a[1];      pos := 1;      for j := 1 to i do        if a[j] > largest then begin          largest := a[j];          pos := j        end;      swap(a[i], a[pos])    end end;

42. Shell sort This algorithm divides the list of data into several interlaced sublists and sorts them (usually by insertion sort). • The gap between two adjacent elements in the interlaced sublists is gradually decreased until the gap is equal to 1. • Using this method, the time used for sorting the whole list using insertion sort is shorter. run-time efficiency : in the order of (nlogn)

43. Shell sort - Example

44. Shell sort - Example

45. Shell sort - Algorithm procedure ShellSort(var a: list; n: integer); var    i, j, x, gap: integer; begin    gap := InitialGap(n);            { The initial gap }    while gap > 0 do begin      for i := gap + 1 to n do begin      { i begins at the 2nd key of the first sublist }        x := a[i];                { The item to be inserted }        j := i - gap;        while (j > 0) and (x < a[j]) do begin          a[j + gap] := a[j];          j := j - gap        end;        a[j + gap] := x      end;      gap := NextGap(gap, n)       { The next gap (should be smaller) }   end end;

46. Merging Merging is a task of combining two or more groups of data to form a new one that preserves the order of the original groups of data. • Merging is usually operated on files and lists. The files or the lists to be merged are already sorted in a particular order (e.g. in ascending order of the key field), and the merged file or the merged list is sorted in the same order as that of the original files or lists.

47. Merging Example

48. Merging Algorithm procedure merge(A, B: list; var C: list); var  iA, iB, iC: integer; begin   iA := 1; iB := 1; iC := 1;   while (iA <= NumberOfEntries(A)) and (iB <= NumberOfEntries(B)) do begin      if A[iA].key < B[iB].key then begin        C[iC] := A[iA]; iC := iC + 1; iA := iA + 1      end  else if A[iA].key > B[iB].key then begin        C[iC] := B[iB]; iC := iC + 1; iB := iB + 1      end      else begin        { Here A[iA].key = B[iB].key and we should combine the two entries         according to the need of actual application }        C[iC] := combine(A[iA], B[iB]);        iC := iC + 1; iA := iA + 1; iB := iB + 1     end   end;

49. Merging Algorithm (cont’d)   while iA <= NumberOfEntries(A) do begin      C[iC] := A[iA];      iC := iC + 1; iA := iA + 1   end;   while iB <= NumberOfEntries(B) do begin      C[iC] := B[iB];      iC := iC + 1; iB := iB + 1   end; end; run-time efficiency: in the order of n.

50. Merging and external sorting Sometimes we want to sort a file which contains huge amount of records stored on a hard disk, but the file is so huge that it cannot fit into the main memory. In this case, external sorting is required. The basic idea of sorting a huge file stored in a secondary storage is: • breaking the huge file into smaller files, each of which is small enough to fit into the main memory; • for all these small files, loading each one into the main memory, sorting it and saving it into the secondary storage; • merging the smaller files to give a sorted single file.