Analysis and Design of Algorithms

1. Analysis and Design of Algorithms Dr. AdilYousif University of Science & Technology

2. Course Syllabus • Introduction • Analysis of Algorithm • Brute Force and Exhaustive Search • Recursive and Iterative Algorithm • Divide-and-Conquer • Dynamic Programming • Greedy Techniques • NP- Complete Problem

3. Course Objectives • This course introduces students the analysis and design of computer algorithms. • Outcome Upon completion of this course, students will be able to do the following: • Analyze the asymptotic performance of algorithms. • Demonstrate a familiarity with major algorithms • Apply important algorithmic design paradigms and methods of analysis. • Synthesize efficient algorithms in common engineering design situations.

4. More Information • Textbook • Introduction to Design & Analysis Computer Algorithm 2nd, Sara Baase, Allen Van Gelder, Adison-Wesley, 2000. • Others • Introduction to Algorithms 2nd,Cormen, Leiserson, Rivest and Stein, The MIT Press, 2001. • Algorithms, Richard Johnsonbaugh, Marcus Schaefer, Prentice Hall, 2004. • Introduction to The Design and Analysis of Algorithms 3rd Edition,AnanyLevitin, Adison-Wesley, 2007.

5. Course Evaluation 30Marks (Lab & Assignments) 70 Marks (Final Exam)

6. Algorithms and their analysis • An algorithm is a step-by-step procedure for solving a problem in a finite amount of time. Input Algorithm Output

7. Purpose • To estimate how long a program will run. • To estimate the largest input that can reasonably be given to the program. • To compare the efficiency of different algorithms. • To help focus on the parts of code that are executed the largest number of times. • To choose an algorithm for an application.

8. Purpose (Example) • Suppose a machine that performs a million floating-point operations per second (106 FLOPS), then how long an algorithm will run for an input of size n=50? • 1) If the algorithm requires n2 such operations:

9. Purpose (Example) • Suppose a machine that performs a million floating-point operations per second (106 FLOPS), then how long an algorithm will run for an input of size n=50? • 1) If the algorithm requires n2 such operations: • 0.0025 second

10. Purpose (Example) • Suppose a machine that performs a million floating-point operations per second (106 FLOPS), then how long an algorithm will run for an input of size n=50? • 1) If the algorithm requires n2 such operations: • 0.0025 second • 2) If the algorithm requires 2n such operations: • A) Takes a similar amount of time (t < 1 sec) • B) Takes a little bit longer time (1 sec < t < 1 year) • C) Takes a much longer time (1 year < t)

11. Purpose (Example) • Suppose a machine that performs a million floating-point operations per second (106 FLOPS), then how long an algorithm will run for an input of size n=50? • 1) If the algorithm requires n2 such operations: • 0.0025 second • 2) If the algorithm requires 2n such operations: • over 35 years!

12. Time Complexity Is a Function Specifies how the running time depends on the size of the input. A function mapping “size” of input “time” T(n) executed .

13. Definition of Time • # of seconds (machine, implementation dependent). • # lines of code executed. • # of times a specific operation is performed (e.g., addition).

14. input size running time Number of times basic operation is executed execution time for basic operation Theoretical analysis of time efficiency Time efficiency is analyzed by determining the number of repetitions of the basic operation as a function of input size Basic operation: the operation that contributes most towards the running time of the algorithm. T(n) ≈copC(n)

15. Input size and basic operation examples

16. Input size and basic operation examples

17. Input size and basic operation examples

18. Input size and basic operation examples

19. Input size and basic operation examples

20. Theoretical analysis of time efficiency Time efficiency is analyzed by determining the number of repetitions of the basic operation as a function of input size

21. Which Input of size n? Efficiency also depends on the particular input • For instance: search a key in a list of n letters • Problem input: a list of n letters • How many different inputs?

22. Which Input of size n? Efficiency also depends on the particular input For instance: search a key in a list of n letters There are 26n inputs of size n.Which do we considerfor the time efficiency C(n)?

23. Best-case, average-case, worst-case • Worst case: W(n) – maximum over inputs of size n • Best case: B(n) – minimum over inputs of size n • Average case: A(n) – “average” over inputs of size n • NOT the average of worst and best case • Under some assumption about the probability distribution of all possible inputs of size n, calculate the weighted sum of expected C(n) (numbers of basic operation repetitions) over all possible inputs of size n.

24. Example: Sequential search • Problem: Given a list of n elements and a search key K, find an element equal to K, if any. • Algorithm: Scan the list and compare its successive elements with K until either a matching element is found (successful search) or the list is exhausted (unsuccessful search) • Worst case • Best case • Average case

25. An example • Compute gcd(m, n) by applying the algorithm based on checking consecutive integers from min(m, n) down to gcd(m, n) • Input size? • Best case? • Worst case? • Average case?

26. Algorithm Descriptions Methods • Nature languages: Arabic, English, etc. • Flowcharts. • Flowcharts • Pseudo-code: codes very close to computer languages, e.g., C programming language. • Programs: C programs, C++ programs, Java programs.

27. Properties of Algorithms • Finiteness • Absence of Ambiguity • Definition of Sequence • Feasibility • Input • Output

28. What is Programming? • Phases of Programming • Design • Implementation • Testing • Maintenance

29. Algorithm vs. Programs • A computer program is one concrete implementation of an algorithm using a particular computer language • The design phase should produce an algorithm • The implementation phase should produce a program • The design phase is typically much longer than the programming phase

30. Flowchart and Pseudocode • Three tools are used to convert algorithms into computer programs: • Flowchart - Graphically depicts the logical steps to carry out a task and shows how the steps relate to each other. • Pseudocode - Uses English-like phrases with some Visual Basic terms to outline the program.

31. Problem solving example • How many stamps do you use when mailing a letter? • One rule of thumb is to use one stamp for every five sheets of paper or fraction thereof.

32. Algorithm 1. Request the number of sheets of paper; call it Sheets. (input) 2. Divide Sheets by 5. (processing) 3. Round the quotient up to the next highest whole number; call it Stamps. (processing) 4. Reply with the number Stamps. (output)

33. Flowcharts • Graphically depict the logical steps to carry out a task and show how the steps relate to each other.

34. Flowchart symbols

35. Flowchart symbols continued

36. Flowchart example

37. Pseudocode • Uses English-like phrases to outline the task.

38. Pseudocode example Determine the proper number of stamps for a letter Read Sheets (input) Set the number of stamps to Sheets / 5 (processing) Round the number of stamps up to the next whole number (processing) Display the number of stamps (output)

39. Divide-and-conquer method • Used in problem solving – take a large problem and break it into smaller problems solving the small ones first • Breaks a problem down into modules

40. Statement structures • Sequence – follow instructions from one line to the next without skipping over any lines • Decision - if the answer to a question is “Yes” then one group of instructions is executed. If the answer is “No,” then another is executed • Looping – a series of instructions are executed over and over

41. Sequence Flowchart

42. Decision flow chart

43. Looping flow chart

44. Direction of Numbered NYC Streets Algorithm • Problem: Given a street number of a one-way street in New York City, decide the direction of the street, either eastbound or westbound • Discussion: in New York City even numbered streets are Eastbound, odd numbered streets are Westbound

45. Flowchart

46. Pseudocode Program: Determine the direction of a numbered NYC street Get street If street is even Then Display Eastbound Else Display Westbound End If

47. Class Average Algorithm • Problem: Calculate and report the grade-point average for a class • Discussion: The average grade equals the sum of all grades divided by the number of students Output: Average grade Input: Student grades Processing: Find the sum of the grades; count the number of students; calculate average

48. Flowchart

49. Pseudocode Program: Determine the average grade of a class Initialize Counter and Sum to 0 Do While there are more data Get the next Grade Add the Grade to the Sum Increment the Counter Loop Computer Average = Sum / Counter Display Average

50. Tips and tricks of flowcharts • Flowcharts are time-consuming to write and difficult to update • For this reason, professional programmers are more likely to favor pseudocode and hierarchy charts • Because flowcharts so clearly illustrate the logical flow of programming techniques, they are a valuable tool in the education of programmers