TE Computer (Year 2018-19) Course : Design and Analysis of Algorithms Unit I : Fundamentals

1. TE Computer (Year 2018-19)Course : Design and Analysis of AlgorithmsUnit I : Fundamentals Prepared By Prof. Anita Thengade MITCOE

2. Caution • Power off (or ring off) your mobile phone. Keep your mobile phones in the bag and not on desk. • Feedback (positive or negative) is encouraged and welcome, and IMPORTANT. (ask questions!) • Thinking, before, in and after class is encouraged and IMPORTANT. Anita Thengade , MIT WPU

3. Course Prerequisite • Data Structure • Discrete Mathematics • C, Java or other programming languages • Theory of Computation Anita Thengade , MIT WPU

4. Objectives & Outcomes Course Objectives : • To develop problem solving abilities using mathematical theories • To analyze the performance of algorithms • To study algorithmic design strategies Anita Thengade , MIT WPU

5. Objectives & Outcomes • Course Outcomes: • On completion of the course, student will be able to– • Formulate the problem • Analyze the asymptotic performance of algorithms • Decide and apply algorithmic strategies to solve given problem • Find optimal solution by applying various methods Anita Thengade , MIT WPU

6. Expected Outcomes • Student should be able to • Define algorithm formally and informally. • Explain the idea of different algorithms • Describe the process of algorithm design and analysis • Design recursive and non-recursive algorithms Anita Thengade , MIT WPU

7. Unit I Syllabus Content Unit-I: Fundamentals The Role of Algorithms in Computing - What are algorithms, Algorithms as technology, Evolution of Algorithms, Design of Algorithm, Need of Correctness of Algorithm, Confirming correctness of Algorithm – sample examples, Iterative algorithm design issues. Anita Thengade , MIT WPU

8. The Role of Algorithms in Computing -What are algorithms • Algorithm Design –I (Problems, Algorithms, Programs, Pseudo Codes) Anita Thengade , MIT WPU

9. Unit I References Anita Thengade , MIT WPU

10. History,The Role of Algorithms in Computing - What are algorithms, Algorithms as technology Anita Thengade , MIT WPU

11. Why Study this Course? • Donald E. Knuth stated “Computer Science is the study of algorithms” • Cornerstone of computer science. Programs will not exist without algorithms. • Basic for any Computer Science Degree • Computational Primitives –CG –Geometric Algorithms • Communication Network –Shortest path Algorithms • Genome Structure in Bioinformatics –Dynamic Programming • Search engines –Page Rank Algorithm by Google • Challenging (i.e. Good for brain !!!) • Real Blend of Creativity and Precision • Very interesting if you can concentrate on this course Anita Thengade , MIT WPU

12. Algorithm: A brief History • Muhammad ibn Musa al-Khwarizmi, one of the most influential mathematicians in 9th century in Baghdad, wrote a textbook in Arabic about adding, multiplying, dividing numbers, and extracting square roots and computing π. www.lib.virginia.edu/science/parshall/khwariz.html • Many centuries later, decimal system was adopted in Europe, and the procedures in Al Khwarizmi’s book were named after him as “Algorithms”. Anita Thengade , MIT WPU

13. What is an Algorithm? • An algorithm is a sequence of unambiguous instructions for solving a computational problem, i.e., for obtaining a required output for any legitimate input in a finite amount of time. Anita Thengade , MIT WPU

14. Algorithm • More precisely, an algorithm is a method or process to solve a problem satisfying the following properties: • Finiteness • terminates after a finite number of steps • Definiteness • Each step must be rigorously and unambiguously specified • Input • Valid inputs must be clearly specified. • Output • can be proved to produce the correct output given a valid input. • Effectiveness • Steps must be sufficiently simple and basic. • Goal : Design a fast algorithm. Anita Thengade , MIT WPU

15. Examples of Algorithms – Computing the Greatest Common Divisor of Two Integers • gcd(m, n): the largest integer that divides both m and n. • //First try –School level algorithm: • Step1: factorize m.( m  m1*m2*m3…) • Step2: factorize n. (n  n1*n2*…) • Step 3: Identify common factors , multiply and return. Anita Thengade , MIT WPU

16. Methods of Specifying an Algorithm • Natural language • Ambiguous Example : “Mike ate the sandwich on a bed.” • Pseudocode • A mixture of a natural language and programming language-like structures • Precise and concise. • Pseudocode in this course • omits declarations of variables • use indentation to show the scope of such statements as for, if, and while. • use  for assignment Anita Thengade , MIT WPU

17. Pseudocode of Euclid’s Algorithm • Algorithm :Euclid(m, n) • //Computes gcd(m, n) by Euclid’s algorithm • //Input: Two non negative, not-both-zero integers m and n • //Output: Greatest common divisor of m and n gcd(m, n) = gcd(n, m mod n) • While (m does not divide n) do r  n mod m n  m m  r • return m • Questions: • Finiteness: how do we know that Euclid’s algorithm actually comes to a stop? • Definiteness: non-ambiguity • Effectiveness: effectively computable. • Which algorithm is faster, the Euclid’s or previous one? Anita Thengade , MIT WPU

18. What can we learn from the previous examples? • Each step of an algorithm must be unambiguous. • The same algorithm can be represented in several different ways. (different pseudo codes) • There might exists more than one algorithm for a certain problem. • Algorithms for the same problem can be based on very different ideas and can solve the problem with dramatically different speeds. Anita Thengade , MIT WPU

19. Fundamentals of Algorithmic Problem Solving • Understanding the problem • Asking questions, do a few examples by hand, think about special cases, etc. • Deciding on • Exact vs. approximate problem solving • Appropriate data structure • Design an algorithm • Proving correctness • Analyzing an algorithm • Time efficiency : how fast the algorithm runs • Space efficiency: how much extra memory the algorithm needs. • Coding an algorithm Anita Thengade , MIT WPU

20. Algorithm Design and Analysis Process Anita Thengade , MIT WPU

21. Algorithm Design -I • Problems • Algorithms • Computer Programs • Pseudo-Codes Anita Thengade , MIT WPU

22. Algorithm Design -I • Aspects of algorithm design • creating an efficient algorithm to solve a problem in an efficient way using minimum time and space. • To solve a problem with different approaches • efficient with respect to time consumption, • Or memory efficient. • However, one has to keep in mind that both time consumption and memory usage cannot be optimized simultaneously. • Programmers commonly deal with problems, algorithms, and computer programs. These are three distinct concepts. Anita Thengade , MIT WPU

23. Problem • A problem is a task to be performed. It is best thought of in terms of inputs and matching outputs. • A problem definition • should not include any constraints on how the problem is to be solved. • should include constraints on the resources that may be consumed by any acceptable solution • The solution method should be developed only after the problem is precisely defined and thoroughly understood. Anita Thengade , MIT WPU

24. Problem • Problems can be viewed as functions in the mathematical sense. • A function is a matching between inputs (the domain) and outputs (the range). • An input to a function might be a single value or a collection of information. The values making up an input are called the parameters of the function. • A specific selection of values for the parameters is called an instance of the problem. • Different instances might generate the same output. (e.g. Sorting algorithm feed with different arrays as different inputs still results Anita Thengade , MIT WPU

25. Algorithm • An algorithm is a method or a process followed to solve a problem. If the problem is viewed as a function, then an algorithm is an implementation of the function that transforms an input to the corresponding output. • A problem can be solved by many different algorithms. A given algorithm solves only one problem (i.e., computes a particular function). • The advantage of several solutions to a problem • Solution ‘A’ might be more efficient than solution ‘B’ • for a specific variation of the problem, • or for a specific class of inputs to the problem, • For example, one sorting algorithm might be the best for sorting a small collection of integers, another might be the best for sorting a large collection of integers, and a third might be the best for sorting a collection of variable-length strings. • We see that different algorithms can be used to solve the same problem, and that different programs can be used to describe the same algorithm. Anita Thengade , MIT WPU

26. Computer Programs • To achieve the criterion of definiteness, algorithms are written in a programming language. • A Computer Program is an expression of an algorithm in a programming language. We often think of a computer program as an instance, or concrete representation, of an algorithm in some programming language. Sometimes words such as procedure, function and subroutine are used synonymously for program. Anita Thengade , MIT WPU

27. Algorithm Design Summary - A problem is a function or a mapping of inputs to outputs. –An algorithm is a recipe for solving a problem whose steps are concrete and unambiguous. The algorithm must be correct, of finite length, and must terminate for all inputs. –A program is an instantiation of an algorithm in a computer programming language. Anita Thengade , MIT WPU

28. Algorithm Design Techniques Anita Thengade , MIT WPU

29. Algorithm Design Techniques Anita Thengade , MIT WPU

30. Algorithm Design Techniques Anita Thengade , MIT WPU

31. Algorithm Design Techniques Anita Thengade , MIT WPU

32. Algorithm Design Techniques Anita Thengade , MIT WPU

33. Algorithm as a Technology • Would you have any reason to study algorithms? The answer is ‘Yes’, we would like to demonstrate that our solution method terminate sand does so with the correct answer; we would probably want our implementation to be within the bounds of good software engineering practice, but we would most often use whichever method was the easiest to implement. • Computers may be fast, but they are not infinitely fast. And memory may be inexpensive, but it is not free. Computing time is therefore a bounded resource, and so is space in memory. You should use these resources wisely, and algorithms that are efficient in terms of time or space will help you do so. Anita Thengade , MIT WPU

34. Algorithm as a Technology • Insertion sort typically has a smaller constant factor than merge sort, so that c1 < c2. Remember that the constant factors can have far less of an impact on the running time than the dependence on the input size ‘n’. • -For example, assume a faster computer ‘A’ (1010instructions/sec) running insertion sort against a slower computer ‘B’ (107instructions/sec) running merge sort. Suppose that c1=2，c2=50 and n= 107. Anita Thengade , MIT WPU

35. Algorithm as a Technology Anita Thengade , MIT WPU

36. Algorithm as a Technology • Efficiency: 1.Different algorithms devised to solve the same problem often differ dramatically in their efficiency. 2.These differences can be much more significant than differences due to hardware and software. Anita Thengade , MIT WPU

37. Issues related to algorithms • How to design algorithms • Need Correctness of the Algorithm • How to analyze algorithm efficiency Anita Thengade , MIT WPU

38. Iterative Algorithm Design Issues • Correct use of Loops • Efficiency of Algorithm • Estimation and Specifying Execution Times • Order Notation • Algorithm Strategies Anita Thengade , MIT WPU

39. Order Notation: Asymptotic Analysis • Core Idea: Cannot compare actual times; hence compare Growth or how time increases with input • Common Growth Functions • Big-Oh (O(:)), Big-Omega ((:)), and Big-Theta ((.)) Notations • Solve recurrence with Growth Functionssize Anita Thengade , MIT WPU

40. Anita Thengade , MIT WPU

41. Anita Thengade , MIT WPU

42. Asymptotic Analysis • Used to formalize that an algorithm has running time or storage requirements that are ``never more than,'' ``always greater than,'' or ``exactly'' some amount Anita Thengade , MIT WPU

43. ASYMPTOTICS NOTATIONSO-notation (Big Oh) • Asymptotic Upper Bound • For a given function g(n), we denote O(g(n)) as the set of functions: • O(g(n)) = { f(n)| there exists positive constants c and n0 such that 0 ≤ f(n) ≤ c g(n) for all n ≥ n0 } Anita Thengade , MIT WPU

44. ASYMPTOTICS NOTATIONSΘ-notation • Asymptotic tight bound • Θ (g(n)) represents a set of functions such that: • Θ (g(n)) = {f(n): there exist positive constants c1, c2, and n0such that 0 ≤ c1g(n) ≤ f(n) ≤ c2g(n) for all n≥ n0 } Anita Thengade , MIT WPU

45. ASYMPTOTICS NOTATIONSΩ-notation • Asymptotic lower bound • Ω (g(n)) represents a set of functions such that: • Ω(g(n)) = {f(n): there exist positive constants c and n0 such that 0 ≤ c g(n) ≤ f(n) for all n≥ n0} Anita Thengade , MIT WPU

46. O-notation ------------------Less than equal to (“≤”) • Θ-notation ------------------Equal to (“=“) • Ω-notation ------------------Greater than equal to (“≥”) Anita Thengade , MIT WPU

47. Bounds of a Function Anita Thengade , MIT WPU

48. c1, c2 & n0 -> constants • T(n) exists between c1n & c2n • Below n0 we do not plot T(n) • T(n) becomes significant only above n0 Anita Thengade , MIT WPU

49. Examples of algorithms for sorting techniques and their complexities • Insertion sort : O(n2) • Selection sort : O(n2) • Quick sort : O(n logn) • Merge sort : O(n logn) Anita Thengade , MIT WPU

50. Examples Big Oh (O - Notation) • Example: 2n + 10 is O(n) • 2n + 10 <= cn • (c - 2) n >= 10 • n >= 10/(c - 2) • Pick c = 3 and n0 = 10 Anita Thengade , MIT WPU