Introduction to Analysis of Algorithms

1. Introduction to Analysis of Algorithms

2. Design Fundamentals • Key points • Machines that hold data • Language for describing data manipulation. • Functions that describe what kind of refined data can be produced from raw data. • Structures for representing data.

3. Analyzing Algorithms • Why • Measuring efficiency of an Algorithm. • Efficiency checked by • Correctness • Implementation • Simplicity • Execution time and Memory space req. • New ways of doing same task better.

4. Time and Space • Time Complexity • Amount of computer time an algorithm needs to execute the program and get the intended result. • Space Complexity • Amount of memory required for running an algorithm.

5. Types of Analysis • Priori Analysis • Machine independent • Done before implementation • Posteriori Analysis • Target Machine dependent • Done after implementation

6. Analysis of an Algorithm • Lets take an example • Finding maximum of n numbers and its location in the vector. • Requirements • Vector say A of size n. • Variables to store largest value and its index location (say large and j) • Design the Algorithm • Calculate the time complexity/Order of Magnitude

7. Order of Magnitude • Order of Magnitude of an Algorithm is the sum of number of occurrences of statements contained in it. • Other terms • Worst case Running Time • Best case Running Time • Average case Running Time

8. Insertion Sort (A, n) Another algorithm

9. Asymptotic Notations • Concerned with how the running time of an algorithm increases with the size of input in the limit, as the size of input increases without bound. • Notations • O notation • Ω notation • Θ notation

10. O notation • Formally defined as • For non-negative functions T(n) and f(n), the function T(n) = O(f(n)) if there are positive constants c and n0such that T(n) ≤ c*f(n) for all n, n ≥ n0 • Known as the Big-Oh • If graphed, f(n) serves as an upper bound to the curve you are analysing. • Describes the worst that can happen.

11. c*f(n) T(n) n0 n  Big Oh Notation O notation

12. Ω Notation • Formally defined as • For non-negative functions, T(n) and f(n), the function T(n) = Ω(f(n)) if there are positive constants c and n0 such that T(n) ≥ c*f(n) for all n, n ≥ n0. • f(n) is the lower bound for T(n). • Describes best that can happen for a given data input size.

13. Ω Notation Omega Notation T(n) c*f(n) n0 n 

14. Θ Notation • Formally defined as • For non-negative functions, T(n) and g(n), the function T(n) = Θ(g(n)) if there exist positive constants c1, c2 and n0 such that c1*g(n) ≤ T(n) ≤ c2*g(n) for all n, n ≥ n0. • Describes the average case for the input data size n.

15. Θ Notation c2*g(n) T(n) c1*g(n) n0 n  Θ Notation

16. Assumptions while finding Time Complexity • The leading constant of highest power of n and all lower powers of n are ignored in f(n) • Example for Insertion sort • T(n) = O(f(n)) • Best case f(n) = (c1+c2+c3+c4+c7)n – (c2+c3+c7) • Therefore T(n) = O(n)

17. Some Problems….

18. Randomized Algorithms • A randomized algorithm is just one that depends on random numbers for its operation. • Normally use a randomizer. • Decisions made in the algorithm depend on the output of the randomizer.

19. Pseudorandom Numbers • The computer is not capable of generating truly random numbers • The computer can only generate pseudorandom numbers--numbers that are generated by a formula • Pseudorandom numbers look random, but are perfectly predictable if you know the formula • Devices for generating truly random numbers do exist

20. Getting (pseudo)random numbers in Java • import java.util.Random; • new Random(long seed) // constructor • new Random() // constructor, uses System.timeInMillis() as seed • void setSeed(long seed) • nextBoolean() • nextFloat(), nextDouble() // 0.0 < return value < 1.0 • nextInt(), nextLong() // all 232 or 264 possibilities • nextInt(int n) // 0 < return value < n • nextGaussian()

21. Randomized Algorithms …contd • Result may not always be 100% correct. • Correctness characterized by some randomness or probability. • Need to run the algorithms for a longer time • Categories: Numerical, Monte-Carlo, Las Vegas, Sherwood.

22. Randomized Algorithms… contd • Examples • Primality Testing • Identifying the Repeated Element • Randomized Quicksort • Advantages • Simple • Efficient • Disadvantages • Does not always result in better performance

23. Points to be noted • We generate random numbers • For their own sake • For simulation • Experimental in nature • Longer you run them, the better your results • It is what you do when you do not • know what else to do • Should be in every programmers toolbox

24. Recursive Algorithms • Algorithms which use the concept of recursion to solve problems • Examples • Fibonacci series • Factorial • Algorithms with divide and conquer approach

25. Recursive Algorithms… contd • Advantages • Easier to implement • Efficient • Disadvantages • Load on compiler.

26. Next … • Divide and Conquer