Algorithm design and analysis

1. Algorithm design and analysis Lecture 2 Data structure

2. What is Algorithm Analysis? *How to estimate the time required for an algorithm *Techniques that drastically reduce the running time of an algorithm *A mathematical framework that more rigorously describes the running time of an algorithm

3. Which algorithm is better? -Efficient -Less time (running time) -Correct -Easy to implement

4. What do we need? Correctness: Whether the algorithm computes the correct solution for all instances Efficiency: Resources needed by the algorithm 1-Time: Number of steps. 2- Space: amount of memory used. Measurement “model”: Worst case, Average case and Best case.

5. How can we calculate running time? We need a model to apply our algorithms on it, and each one gets the same result. This model is called “RAM” Random Access Machine. -Perform simple operations ( +, *, - ,=, if , call) And takes 1 time step

6. RAM model 1-has one processor 2-executes one instruction at a time 3-each instruction takes "unit time“ 4-has fixed-size operands, and 5-has fixed size storage (RAM and disk).

7. Many criteria affect the running time of an algorithm, including: 1-speed of CPU, bus and peripheral hardware 2- design think time, programming time and debugging time 3- language used and coding efficiency of the programmer 4- quality of input (good, bad or average)

8. Algorithm Analysis/1 *we estimate the algorithm's performance based on the number of key and basic operations it requires to process an input of a given size. *For a given input size n we express the time T to run the algorithm as a function T(n)

9. Algorithm Analysis 2 -Formally, let T(A,L,M) be total run time for algorithm A if it were implemented with language L on machine M. Then the complexity class of algorithm A is O(T(A,L1,M1) U O(T(A,L2,M2)) U O(T(A,L3,M3)) U

10. cont *Call the complexity class V; then the complexity of A is said to be f if V = O(f) *The class of algorithms to which A belongs is said to be of at most linear/quadratic/ etc. growth in best case if the function TA best(n) is such (the same also for average and worst case)

11. Input Size *Time and space complexity - This is generally a function of the input size - E.g., sorting, multiplication * How we characterize input size depends: - Sorting: number of input items - Multiplication: total number of bits - Graph algorithms: number of nodes & edges - Etc

12. Running Time -Number of primitive steps that are executed - Except for time of executing a function call most statements roughly require the same amount of time

13. EXAMPLE

14. Example we have two persons first called A ,and second one called B, both of them should perform some operations using own method of solutionso we asked both of them to add number from 0 to 1024

15. cont That’s why we choose person B because person B is faster than person A -Easy method - correct result - less time .

16. Example running time for loop

17. Analysis Running Time 1- Best case :minimum number of steps (time) 2-Average case: average number of steps (time) 3- Worst case : Maximum number of steps.

19. Asymptotic Performance -In this course, we care most about asymptotic performance How does the algorithm behave as the problem size gets very large?  Running time  Memory/storage requirements  Bandwidth/power requirements/logic gates/etc.

20. Asymptotic Notation By now you should have an intuitive feel for asymptotic (big-O) notation: - What does O(n) running time mean? O(n2 )? O(n lg n)? - How does asymptotic running time relate to asymptotic memory usage?

21. cont The main idea of asymptotic analysis is to have a measure of efficiency of algorithms that doesn’t depend on machine specific constants, and doesn’t require algorithms to be implemented and time taken by programs to be compared

22. cont Asymptotic notations are mathematical tools to represent time complexity of algorithms for asymptotic analysis. The following 3 asymptotic notations are mostly used to represent time complexity of algorithms.

23. 2-Big O Notation The Big O notation defines an upper bound of an algorithm, it bounds a function only from above. For example, consider the case of Insertion Sort. It takes linear time in best case and quadratic time in worst case. We can safely say that the time complexity of Insertion sort is O(n^2)

24. 2-Big O Notationif 0 ≤f(n) ≤ cg(n)at n ≥ n0 and c >0

25. cont • f we use Θ notation to represent time complexity of Insertion sort, we have to use two statements for best and worst cases:1. The worst case time complexity of Insertion Sort is Θ(n^2).2. The best case time complexity of Insertion Sort is Θ(n). • The Big O notation is useful when we only have upper bound on time complexity of an algorithm. Many times we easily find an upper bound by simply looking at the algorithm

26. 3-Ω Notation Just as Big O notation provides an asymptotic upper bound on a function, Ω notation provides an asymptotic lower bound. -the Omega notation is the least used notation among all three.

27. If f(n)≥ cg(n)≥ n0 and c>0

28. 1-Θ Notation theta notation bounds a functions from above and below, so it defines exact asymptotic behavior.A simple way to get Theta notation of an expression is to drop low order terms and ignore leading constants. For example, consider the following expression.3n3 + 6n2 + 6000 = Θ(n3)

29. cont Dropping lower order terms is always fine because there will always be a n0 after which Θ(n3) has higher values than Θ(n2) irrespective of the constants involved

30. theta notation If f(n)≥ cg(n)≥ n0 And f(n) ≤ cg(n)at n ≥ n0

31. How to find g (n)? Simply : 1- dismiss lower powers (keep only the highest power) 2- Replace the highest power coefficient with a constant (c ) Ex: y=4x4+8x3+10x2+14x+17 Y=c. x4

32. What will be the notations? So what are the difference between them ? Value of constant and n0 e.x) f (n)= 4n4 + 8n3+-10n2+14n +17 c.g(n)= c(n4) f (n)= 4n4 + 8n3+-10n2+14n +17≤ CN4 ASSUMN0=1 THEN: 4+8-10+14+17 ≤ c so 33 ≤ c