Algorithmic Complexity

1. Algorithmic Complexity

2. Algorithm Efficiency • Efficiency • Amount of resources used by algorithm • Time, space • Measuring efficiency • Benchmarking • Asymptotic analysis

3. Time complexity • The (time) complexity of a program (for a given input) is the number of elementary instructions that this program executes. • This number is computed with respect to the size n of the input data. • We thus make the assumption that each elementary instruction takes the same amount of time (because it is probably one processor instruction)

4. Space complexity • The (space) complexity of a program (for a given input) is the number of elementary objects that this program needs to store during its execution. • This number is computed with respect to the size n of the input data. • We thus make the assumption that each elementary object needs the same amount of space (which is almost true for the basic types: one machine word).

5. Time-space tradeoff • It is the situation in which the computation time can be reduced at the cost of increased memory use. • As the relative costs of hard drive space has for some time been getting cheaper at a much faster rate than other components of computers • Often, by exploiting a space–time tradeoff, a program can be made to run much faster.

6. Asymptotic Analysis • Approach • Mathematically analyze efficiency • Calculate time as function of input size n • T  O[ f(n) ] • T is on the order of f(n) • “Big O” notation • Advantages • Measures intrinsic efficiency • Dominates efficiency for large input sizes

7. Sequential/Linear Search Algorithm • Algorithm Get the search criterion (key) Get the first number While ( (number < key) and (still more records) ) Get the next number End_while If ( record = key ) Then success Else there is no match in the file End_else

8. Linear Search Algorithm • Analysis of # of guesses needed for 1…n • If number = 1, requires 1 guess • If number = n, requires n guesses • On average, needs n/2 guesses • Time = O( n ) = Linear time

9. Binary Search • The algorithm continually divides the list into two parts.

10. What’s the Pattern? • List of 11 took 4 tries • List of 32 took 5 tries • List of 250 took 8 tries • List of 512 took 9 tries • 32 = 25 and 512 = 29 • 8 < 11 < 16 23 < 11 < 24 • 128 < 250 < 256 27 < 250 < 28

11. Binary Search Complexity • Analysis of # of guesses needed for 1…n • We say that the binary search algorithm runs in log2 n time. (Also written as lg n) • Lg n means the log to the base 2 of some value of n. • 8 = 23 lg 8 = 3 16 = 24 lg 16 = 4 • There are no algorithms that run faster than lg n time. • So Binary search is more efficient than linear search.

12. Analyzing algorithms  • The “process” of determining how much resources (time, space) are used by a given algorithm. • We want to be able to make quantitative assessments about the value (goodness) of one algorithm compared to another

13. Types of Case Analysis • Can analyze different types (cases) of algorithm behavior • Types of analysis • Best case • Worst case • Average case

14. Types of Case Analysis • Best case • Smallest number of steps required • Not very useful • Example  Find item in first place checked

15. Types of Case Analysis • Worst case • Largest number of steps required • Useful for upper bound on worst performance • Real-time applications (e.g., multimedia) • Quality of service guarantee • Example  Find item in last place checked

16. Types of Case Analysis • Average case • Number of steps required for “typical” case • Most useful metric in practice