Algorithm Representation & Analysis

1. Algorithm Representation & Analysis Lecture 02 ITS033 – Programming & Algorithms Asst. Prof. Dr. Bunyarit Uyyanonvara IT Program, Image and Vision Computing Lab. School of Information, Computer and Communication Technology (ICT) Sirindhorn International Institute of Technology (SIIT) Thammasat University http://www.siit.tu.ac.th/bunyaritbunyarit@siit.tu.ac.th02 5013505 X 2005

2. Overview • Conceptual Flow • Pseudo code • Asymptotic Notation • Growth Rate • Common Running Times

3. Algorithm Representation: Conceptual Flow Lecture 02.1 ITS033 – Programming & Algorithms Asst. Prof. Dr. Bunyarit Uyyanonvara IT Program, Image and Vision Computing Lab. School of Information, Computer and Communication Technology (ICT) Sirindhorn International Institute of Technology (SIIT) Thammasat University http://www.siit.tu.ac.th/bunyaritbunyarit@siit.tu.ac.th02 5013505 X 2005

4. Conceptual Flow Conceptual flow is a casual diagram showing a flow of thinking process. It can be written using informal diagram (which is not the programming-like Flow Chart.) Or written in a more formal Pseudo code.

5. Conceptual Flow • Example: • write a conceptual flow for Nag Screen • Random a number from 0 – 100, if the number is more than 50 then random a location on the screen (25 rows x 80 columns) and print ‘X’, but if the random number is less than or equal to 50 then random a location on the screen and print ‘O’. • Repeat the first step until the screen is fully covered by letters.

6. See example

7. Heuristics Heuristics is an loosely defined algorithm which is likely to perform faster or better. However, the output is not guaranteed. For example, in Nag Screen Problem, we random the position only 4000 times before we stop. More of Heuristic in Approximation algorithm and AI course in 3rd year.

8. Problem 1: New world puzzle - revisited • There are N people who want to cross a bridge; they all begin on the same side. • It is night, and they have one flashlight. A maximum of two people can cross the bridge at one time and have to have the flashlight with them. • Mr. A1 takes T1 minutes to cross the bridge, • Mr. A2 takes T2 minutes to cross the bridge, • Mr. A3 takes T3 minutes to cross the bridge, • Mr. An takes Tn minutes to cross the bridge, • T1 < T2 < T3 < T4 < … < Tn • A pair must walk together at the rate of the slower person’s pace. For example, Mr.A2 and Mr.A8 have to take T8 minutes to get across the bridge. • Design an Algorithm to move them all to the other side of the bridge with minimum total time.

9. See example

10. Problem 2: Traveling Salesman • A salesman wants to travel from city E to all the cities and come back to E again. Design an algorithm to find the best path (smallest summation of the cost, as indicated on the route) • A salesman wants to travel from city E to all the cities and come back to E again by visiting each city only once. Design an algorithm to find the best path (smallest summation of the cost, as indicated on

11. See example

12. Conceptual Flow Weakness Conceptual flow is probably too casual and can sometime only be understood by the person who created it. For communication and analysis purposes, a more formal Pseudo code may be used instead.

13. Algorithm Representation: Psudo Code Lecture 02.2 ITS033 – Programming & Algorithms Asst. Prof. Dr. Bunyarit Uyyanonvara IT Program, Image and Vision Computing Lab. School of Information, Computer and Communication Technology (ICT) Sirindhorn International Institute of Technology (SIIT) Thammasat University http://www.siit.tu.ac.th/bunyaritbunyarit@siit.tu.ac.th02 5013505 X 2005

14. Pseudo code • High-level description of an algorithm • More structured than English paragraph • Less detailed than a program • Preferred notation for describing algorithms • Hides program design issues

15. Example of Pseudo code AlgorithmarrayMax1(A, n) Input array A of n integers Output maximum element of A fori 1 ton  1 do max  true; forr i+1 tondo ifA[i] < A[r]then max  false exit; if (max = true) return (A[i]);

16. Algorithm header Algorithm method (arg [, arg…]) Input… Output… Control flow if…then… [else…] while…do… repeat…until… for…do… Indentation replaces braces Pseudo code

17. Method call var.method (arg [, arg…]) Return value returnexpression Expressions Assignment Equality testing n2 Superscripts and other mathematical formatting allowed Pseudo code

18. Pseudo code A structure of an algorithm using Pseudo code Algorithmwhat is the name of your algorithm ? InputWhat is your input for this ? Outputwhat will you get after process the input ? the algorithm itself.

19. Design an algorithm with Pseudo code to find a maximum value in an array with n elements. ? Pseudo code

20. AlgorithmarrayMax2(A, n) This algorithm is better than the previous one AlgorithmarrayMax2(A, n) Input array A of n integers Output maximum element of A currentMax A[0] fori 1 ton  1 do ifA[i]  currentMaxthen currentMax  A[i] returncurrentMax AlgorithmarrayMax2(A, n) Input array A of n integers Output maximum element of A setcurrentMaxtoA[0] repeat n-1 times starting from i= 1 if element no. i  currentMaxthen set currentMaxto this element returncurrentMax

21. Example of Pseudo code AlgorithmarrayMax1(A, n) Input array A of n integers Output maximum element of A fori 1 ton  1 do max  true; forr i+1 tondo ifA[i] < A[r]then max  false exit; if (max = true) return (A[i]); AlgorithmarrayMax1(A, n) Input array A of n integers Output maximum element of A repeat n-1 times starting from i – 1 assume that element no. i is a max repeat starting from r = i+1 until r=n if element no. i < element no. r then element no. i is not a max if (max = true) return (A[i]);

22. Design an algorithm with Pseudo code to find an average value of all the data store in an n-element array Pseudo code

23. Algorithm Complexity Analysis ITS033 – Programming & Algorithms Lecture 02.3 Asst. Prof. Dr. Bunyarit Uyyanonvara IT Program, Image and Vision Computing Lab. School of Information, Computer and Communication Technology (ICT) Sirindhorn International Institute of Technology (SIIT) Thammasat University http://www.siit.tu.ac.th/bunyaritbunyarit@siit.tu.ac.th02 5013505 X 2005

24. Complexity Analysis: How can we measure the efficiency of an algorithm ? the fundamental importance to the design of efficient algorithm is its “Running Time”. After all, time is the most precious measure of an algorithm’s efficiency.

25. Complexity Analysis: experiment • Write a program implementing the algorithm • Run the program with inputs of varying size and composition • get an accurate measure of the actual running time • Plot the results

26. Complexity Analysis: Limitations of Experiments • It is necessary to implement the algorithm, which may be difficult • Results may not be indicative of the running time on other inputs not included in the experiment. • In order to compare two algorithms, the same hardware and software environments must be used

27. Complexity Analysis: Theoretical Analysis • Uses a high-level description of the algorithm instead of an implementation • Characterizes running time as a function of the input size, n. • Takes into account all possible inputs • Allows us to evaluate the speed of an algorithm independent of the hardware/software environment

28. Complexity Analysis: bound • The bound of the running time could be estimated by computing the number of times which the basic operation is executed

29. Basic Operation ‘Basic Operations’ are the most important operation of the algorithm or the operation contributing the most to the total running time. Normally, it is not difficult to identify the basic operation of an algorithm. It is usually the most time-consuming operation in the algorithm’s inner-most loop.

30. Basic computations performed by an algorithm Identifiable in pseudocode Largely independent from the programming language Exact definition not important (we will see why later) Assumed to take a constant amount of time Basic Operations

31. By inspecting the pseudocode, we can determine the maximum number of basic operations executed by an algorithm, as a function of the input size, n Bound is a function of input size, N

32. For each of the following algorithms, indicate A natural size for its input Its basic operation Computing the sum of n numbers Computing n! Finding the largest element in a list of n numbers Example

33. Consider the algorithm for adding two n-by-n matrices. What is its basic operation? How many times is it performed as a function of the matrix order n? How many times is it performed as a function of the total number of element, N? Example

34. AlgorithmarrayMax(A, n)number of operations for this line currentMaxA[0] 2 fori1ton 1do 2+n ifA[i]  currentMaxthen 2(n 1) currentMaxA[i] 2(n 1) returncurrentMax1 Total 5n+ 1 Counting Basic Operations Student A’s counting AlgorithmarrayMax(A, n) number of operations for this line currentMaxA[0] 2 fori1ton 1do n - 1 ifA[i]  currentMaxthen 2(n 1) currentMaxA[i] 2(n 1) returncurrentMax1 Total 5n- 1 Student B’s counting

35. Extra example

36. Algorithm Efficiency

37. Order of Growth ITS033 – Programming & Algorithms Lecture 02.4 Asst. Prof. Dr. Bunyarit Uyyanonvara IT Program, Image and Vision Computing Lab. School of Information, Computer and Communication Technology (ICT) Sirindhorn International Institute of Technology (SIIT) Thammasat University http://www.siit.tu.ac.th/bunyaritbunyarit@siit.tu.ac.th02 5013505 X 2005

38. Growth Rate of Running Time • Changing the hardware/ software environment • Affects T(n) by a constant factor, but • Does not alter the growth rate of T(n) • The linear growth rate of the running time T(n) is an intrinsic property of algorithm arrayMax

39. Big-Oh Notation • Given functions f(n) and g(n), we say that f(n) is O(g(n))if there are positive constantsc and n0 such that f(n)cg(n) for all n n0 Formal Definition Informal definition Big Oh is a function which is an upper bound of a function in question. It indicates the class of performance of that algorithm. It’s like putting a bicycle and a car into a race. The are in different classes.

40. Algorithm Efficiency

41. More Big-Oh Examples • 7n-2 7n-2 is O(n) need c > 0 and n0 1 such that 7n-2  c•n for n  n0 this is true for c = 7 and n0 = 1 • 3n3 + 20n2 + 5 3n3 + 20n2 + 5 is O(n3) need c > 0 and n0 1 such that 3n3 + 20n2 + 5  c•n3 for n  n0 this is true for c = 4 and n0 = 21 • 3 log n + log log n 3 log n + log log n is O(log n) need c > 0 and n0 1 such that 3 log n + log log n  c•log n for n  n0 this is true for c = 4 and n0 = 2

42. Big-Oh Rules • If is f(n) a polynomial of degree d, then f(n) is O(nd), i.e., • Drop lower-order terms • Drop constant factors • Use the smallest possible class of functions • Say “2n is O(n)”instead of “2n is O(n2)” • Use the simplest expression of the class • Say “3n+5 is O(n)”instead of “3n+5 is O(3n)”

43. More Big-Oh Examples what are Big-O for these functions ? • 7n-2 • 3n3 + 20n2 + 5 • 21,207n – 211 • n4 + 23n4 – 20n3 + 2n – 211 • 3 log n + log log n

44. Big-Oh and Growth Rate • The big-Oh notation gives an upper bound on the growth rate of a function • The statement “f(n) is O(g(n))” means that the growth rate of f(n) is no more than the growth rate of g(n) • We can use the big-Oh notation to rank functions according to their growth rate from best to worst • O(1) • O(log n) • O(n) • O(n log n) • O(n2) • O(n3) • O(2n) • O(n!)

45. Common Running Times Lecture 02.5 ITS033 – Programming & Algorithms Asst. Prof. Dr. Bunyarit Uyyanonvara IT Program, Image and Vision Computing Lab. School of Information, Computer and Communication Technology (ICT) Sirindhorn International Institute of Technology (SIIT) Thammasat University http://www.siit.tu.ac.th/bunyaritbunyarit@siit.tu.ac.th02 5013505 X 2005

46. Constant Time: O(1) An algorithm is O(1) when its running time is independent of the number of data items. The algorithm runs in constant time. Example of an algorithm with O(1) The storing of the element involves a simple assignment statement and thus has efficiency O(1).

47. Logarithmic Time : O(log n) An algorithm is O(log n) is among the best algorithms. Typically a result of cutting a problem’s size by a constant factor on each iteration of the algorithm. When the number of elements doubles, the number of operations increases by just 1.

48. Linear Time Algorithms: O(n) An algorithm is O(n) when its running time is proportional to the size of the list. When the number of elements doubles, the number of operations doubles.

49. n-log-n Algorithms: O(n log n) An algorithm is O(n log n) are from many divide-and-conquer algorithms. A little bit slower than O(n) but it is still consider very good algorithm.

50. Quadractic & Cubic : O(n2) O(n2) • Algorithms with running time O(n2) are quadratic. • practical only for relatively small values of n. • Whenever n doubles, the running time of the algorithm increases by a factor of 4. • Algorithms with running time O(n3)are cubic. • efficiency is generally poor; doubling the size of n increases the running time eight-fold.