1 / 93

COMP108 Algorithmic Foundations Algorithm efficiency + Searching/Sorting

COMP108 Algorithmic Foundations Algorithm efficiency + Searching/Sorting. Prudence Wong http://www.csc.liv.ac.uk/~pwong/teaching/comp108. Learning outcomes. Able to carry out simple asymptotic analysis of algorithms See some examples of polynomial time and exponential time algorithms

Gabriel
Download Presentation

COMP108 Algorithmic Foundations Algorithm efficiency + Searching/Sorting

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. COMP108Algorithmic FoundationsAlgorithm efficiency + Searching/Sorting Prudence Wong http://www.csc.liv.ac.uk/~pwong/teaching/comp108

  2. Learning outcomes • Able to carry out simple asymptotic analysis of algorithms • See some examples of polynomial time and exponential time algorithms • Able to apply searching/sorting algorithms and derive their time complexities

  3. Time Complexity Analysis • How fast is the algorithm? Code the algorithm and run the program, then measure the running time • Depend on the speed of the computer • Waste time coding and testing if the algorithm is slow Identify some important operations/steps and count how many times these operations/steps needed to executed

  4. Time Complexity Analysis • How to measure efficiency? Number of operations usually expressed in terms of input size • If we doubled the input size, how much longer would the algorithm take?If we trebled the input size, how much longer would it take?

  5. Why efficiency matters? • speed of computation by hardware has been improved • efficiency still matters • ambition for computer applications grow with computer power • demand a great increase in speed of computation

  6. Amount of data handled matches speed increase? • When computation speed vastly increased, can we handle much more data? • Suppose • an algorithm takes n2comparisons to sortn numbers • we need 1 sec to sort 5 numbers (25 comparisons) • computing speed increases by factor of100 • Using 1 sec, we can now perform100x25comparisons,i.e., to sort50 numbers • With 100 times speedup, only sort 10 times more numbers!

  7. Time Complexity Analysis input n sum = 0 for i = 1 to n do begin sum = sum + i end output sum • Important operation of summation: addition • How many additions this algorithm requires? We need n additions (depend on the input size n)

  8. Space Complexity Analysis input n sum = 0 for i = 1 to n do begin sum = sum + i end output sum We need to store the number n, the variable sum, and the index i => needs 3 memory space In this example, the memory space does not depend on the input size n In other case, it may depend on the input size n

  9. Look for improvement input n sum = n*(n+1)/2 output sum Mathematical formula gives us an alternative to find the sum of first n numbers: 1 + 2 + ... + n = n(n+1)/2 We only need 3 operations: 1 addition, 1 multiplication, and 1 division (no matter what the input size n is)

  10. Finding the minimum...

  11. Finding min among 3 numbers Input: 3 numbers a, b, c Output: the min value of these 3 numbers Algorithm: Input a, b, c a  b ? N Y a  c ? Y b  c ? N Y N Output a Output b Output c Output c

  12. Finding min among 3 numbers Example 1 Input a, b, c a=3, b=5, c=2 Y N a  b ? a  c ? Y b  c ? N Y N Output a Output b Output c Output c 2 is output

  13. Finding min among 3 numbers Example 2 Input a, b, c a=6, b=5, c=7 Y N a  b ? a  c ? Y b  c ? N Y N Output a Output b Output c Output c 5 is output

  14. Time Complexity Analysis • Important operation: comparison • How many comparison this algorithm requires? input a, b, c if (a  b) then if (a  c) then output a else output c else if (b  c) then output b else output c It takes 2 comparisons

  15. Finding min among n numbers Input a0, a1, ... Any systematic approach? a0  a1 ? N Y a0  a2 ? Y a1  a2 ? Y N N a0  a3 ? a1  a3 ? a2  a3 ? a2  a3 ?

  16. Finding min among n numbers Input a[0], a[1], ..., a[n-1] min = 0 i = 1 i = i+1 i < n? N Y Output a[min] a[i] < a[min]? Y min = i N

  17. Finding min among n numbers input a[0], a[1], ..., a[n-1] min = 0 i = 1 while (i <n) do begin if (a[i] < a[min]) then min = i i = i + 1 end output a[min] Example a[]={50,30,40,20,10} 0 50 1 30 1 30 3 20 4 10

  18. Finding min among n numbers n-1 • Time complexity: ?? comparisons input a[0], a[1], ..., a[n-1] min = 0 i = 1 while (i <n) do begin if (a[i] < a[min]) then min = i i = i + 1 end output a[min]

  19. Finding min using for-loop • Rewrite the above while-loop into a for-loop input a[0], a[1], ..., a[n-1] min = 0 for i = 1 to n-1 do begin if (a[i] < a[min]) then min = i end output a[min]

  20. Time complexity- Big O notation …

  21. Which algorithm is the fastest? • Consider a problem that can be solved by 5 algorithms A1, A2, A3, A4, A5 using different number of operations (time complexity). f1(n) = 50n + 20 f2(n) = 10 n log2 n + 100 f3(n) = n2 - 3n + 6 f4(n) = 2n2 f5(n) = 2n/8 - n/4 + 2 f5(n) f3(n) f1(n) Depends on the size of the problem!

  22. f1(n) = 50n + 20 f2(n) = 10 n log2n + 100 f3(n) = n2 - 3n + 6 f4(n) = 2n2 f5(n) = 2n/8 - n/4 + 2

  23. What do we observe? • There is huge difference between • functions involving powers ofn (e.g., n, n log n, n2, called polynomial functions) and • functions involving powering by n (e.g., 2n, called exponential functions) • Among polynomial functions, those with same order of power are more comparable • e.g., f3(n) = n2 - 3n + 6 and f4(n) = 2n2

  24. Growth of functions

  25. Relative growth rate 2n n2 n log n c n

  26. Hierarchy of functions • We can define a hierarchy of functions each having a greater order of magnitude than its predecessor: 1 log n n n2 n3 ... nk ... 2n constant logarithmic polynomial exponential • We can further refine the hierarchy by inserting n log n between n and n2,n2 log n between n2 and n3, and so on.

  27. Hierarchy of functions (2) 1 log n n n2 n3 ... nk ... 2n • Important: as we move from left to right, successive functions have greater order of magnitude than the previous ones. • As n increases, the values of the later functions increase more rapidly than the values of the earlier ones. • Relative growth rates increase constant logarithmic polynomial exponential

  28. Hierarchy of functions (3) 1 log n n n2 n3 ... nk ... 2n • Now, when we have a function, we can assign the function to some function in the hierarchy: • For example, f(n) = 2n3 + 5n2 + 4n + 7The term with the highest power is 2n3.The growth rate of f(n) is dominated by n3. • This concept is captured by Big-O notation constant logarithmic polynomial exponential

  29. Big-O notation • f(n) = O(g(n)) [read as f(n) is of order g(n)] • Roughly speaking, this means f(n) is at most a constant times g(n) for all large n • Examples • 2n3 = O(n3) • 3n2 = O(n2) • 2n log n = O(n log n) • n3 + n2 = O(n3) • function on L.H.S and function on R.H.S are said to have the same order of magnitude

  30. Big-O notation - formal definition • f(n) = O(g(n)) • There exists a constant c and no such that f(n) cg(n) for all n > no • cnon>no then f(n)  cg(n)

  31. f(n) = O(g(n) – Graphical illustration constant c & no such thatn>no, f(n)  c g(n) c g(n) f(n) n0

  32. Example – n+60 = O(n) 2n n + 60 n c=2, n0=60 n0 constant c & no such that n>no, f(n)  c g(n)

  33. Example – 2nlog2n+100n+1000 = O(n log n) 3 n log n c=3, n0=600 f(n) n log n n0 constant c & no such that n>no, f(n)  c g(n)

  34. f(n) = O(g(n))? c g(n) f(n) value of f(n) for n<n0does NOT matter n0 constant c & no such that n>no, f(n)  c g(n)

  35. Which one is the fastest? • Usually we are only interested in the asymptotic time complexity, i.e., when n is large • O(log n) < O(n) < O(n) < O(n log n) < O(n2) < O(2n)

  36. Proof of order of magnitude • Show that 2n2 + 4n is O(n2) • Since n  n2 for all n>1,we have 2n2 + 4n  2n2 + 4n2 6n2for all n>1. • Therefore, by definition, 2n2 + 4n is O(n2). • Show that n3+n2logn+n is O(n3) • Since n n3 and log n  n for all n>1,we have n2 log n  n3, andn3 + n2log n + n 3n3for all n>1. • Therefore, by definition, n3+n2logn+n is O(n3).

  37. Exercise • Determine the order of magnitude of the following functions. • n3 + 3n2 + 3 • 4n2 log n + n3 + 5n2 + n • 2n2 + n2 log n • 6n2 + 2n O(n3) O(n3) O(n2 log n) O(2n)

  38. Exercise (2) • n3 = n3 n • 3n2  3n3 n1 • 3  3n3n1 •  n3+3n2+3  7n3 n1 • Prove the order of magnitude • n3 + 3n2 + 3 • n3 = n3 n • 3n2  n3 n3 • 3  n3n2  n3 + 3n2 + 3  3n3 n3 • 4n2 log n + n3 + 5n2 + n • 4n2 log n  4n3 n1 • n3 = n3 n • 5n2  n3 n5 • nn3 n1  4n2 log n + n3 + 5n2 + n  7n3 n5 • 4n2 log n  4n3 n1 • n3 = n3 n • 5n2  5n3 n1 • nn3 n1  4n2log n+n3+5n2+n  11n3 n1

  39. Exercise (2) cont’d • 2n2  2n2log n n2 • n2 log n = n2 log n n  2n2+n2log n  3n2log n n2 • Prove the order of magnitude • 2n2 + n2 log n • 2n2  n2 log n n4 • n2 log n = n2 log n n  2n2 + n2 log n  2n2 log n n4 • 6n2 + 2n • 6n2  6*2n n1 • 2n = 2n n  6n2 + 2n  7*2n n1

  40. More Exercise • Are the followings correct? • n2log n + n3 + 3n2 + 3 O(n2)? • n + 1000 O(n)? • 6n20 + 2n O(n20)? • n3 + 5n2 logn + n O(n2 log n) ? O(n3) YES O(2n) O(n3)

  41. Some algorithms we learnt input n sum = 0 for i = 1 to n do begin sum = sum + i end output sum • Computing sum of the first n numbers O(n) O(1) input n sum = n*(n+1)/2 output sum O(?) O(?) Finding the min value among n numbers min = 0 for i = 1 to n-1 do if (a[i] < a[min]) then min = i output a[min] O(n) O(?)

  42. More exercise • What is the time complexity of the following pseudo code? for i = 1 to 2n do for j = 1 to n do x = x + 1 O(n2) O(?) The outer loop iterates for 2n times. The inner loop iterates for n times for each i. Total: 2n * n = 2n2.

  43. Learning outcomes • Able to carry out simple asymptotic analysis of algorithms • See some examples of polynomial time and exponential time algorithms • Able to apply searching/sorting algorithms and derive their time complexities

  44. More polynomial time algorithms - searching …

  45. Searching • Input: a sequence of n numbers a0, a1, …, an-1; and a number X • Output: determine whether X is in the sequence or not • Algorithm (Linear search): • Starting from i=0, compare X with ai one by one as long as i < n. • Stop and report "Found!" when X = ai . • Repeat and report "Not Found!" when i >= n.

  46. Linear Search To find 7 • 12 34 2 9 7 5 six numbers7number X • 12 34 2 9 7 57 • 12 34 2 9 7 57 • 12 34 2 9 7 57 • 12 34 2 9 7 57found!

  47. Linear Search (2) To find 10 • 12 34 2 9 7 510 • 12 34 2 9 7 510 • 12 34 2 9 7 510 • 12 34 2 9 7 510 • 12 34 2 9 7 510 • 12 34 2 9 7 510not found!

  48. Linear Search (3) • i = 0 • while i < n do • begin • if X == a[i] then • report "Found!" and stop • else • i = i+1 • end • report "Not Found!"

  49. Time Complexity i = 0 while i < n do begin if X == a[i] then report "Found!" & stop else i = i+1 end report "Not Found!" • Important operation of searching: comparison • How many comparisons this algorithm requires? Best case: X is the 1st no., 1 comparison, O(1) Worst case: X is the last no. OR X is not found, n comparisons, O(n)

  50. Improve Time Complexity? If the numbers are pre-sorted, then we can improve the time complexity of searching by binary search.

More Related