Analysis of Algorithms

1. Analysis of Algorithms Chien-Chung Shen CIS/UD cshen@udel.edu

2. Goal of Algorithm Analysis • Study the performance of algorithms • Two aspects • time • space • Make meaningful comparison between algorithms

3. Issues of Relative Performance (1) Algorithm A Algorithm B 19 seconds 17 seconds • Which algorithm is more efficient (e.g., faster)? • Specify a common machine model and analyze the number of steps (operations) required under the model

4. Issues of Relative Performance (2) • Details (“characteristics”) of dataset • e.g., some sorting algorithms run faster if the data are already partially sorted; other algorithms run slower in this case • What should we do? • Analyze the worst casescenario • Sometimes useful to analyze average case performance, but it is usually harder • it is not clear what set of cases to average over

5. Issues of Relative Performance (3) • “Size” of the problem • A sorting algorithm that is fast for small lists might be slow for long lists • What should we do? • Express runtime (or number of operations) as a function of problem size (N), and to compare the functions asymptotically as the problem size increases

6. Summary • Resolving issues of (1) heterogeneous hardware, (2) different characteristics of datasets, and (3) different sizes of problems leads to simpler classification of algorithms • For instance, [∝ == is proportional to] • if run time of Algorithm A ∝size of input, n • if run time of Algorithm B ∝n2 then A is expected to be faster/more efficient than B for large values of n

7. Basics of Algorithm Analysis • Algorithm A takes 100n + 1 steps to solve a problem with size n ; Algorithm B takes n2 +n +1 steps • Crossover point? • n == 99 • Location of the crossover point depends on the details of the algorithms, inputs, and hardware • For algorithmic analysis, functions with the same leading term are considered equivalent, even if they have different coefficients (n)

8. Big-O Notation • An “order of growth” is a set of functions whose asymptotic growth behavior is considered equivalent • For instance, 2n, 100n,and n + 1 belong to the same order of growth, which is written O(n) in Big-O notation [grow linearly] • All functions with the leading termn2 (no matter what coefficient) belong to O(n2); they are quadratic • Common orders of growth logb n vs. logc n change base == multiply by a constant cn vs. bn both belong to the same order of growth

9. Some Questions The difference between two algorithms with the same order of growth is usually a constant factor, but the difference between a good algorithm (e.g., linear) and a bad algorithm (e.g., exponential) is unbounded!

10. Analysis of Basic Operations • Most arithmetic operations are constant time, irrespective of the “magnitude” of the operands • it takes the same amount of time to compute 1*2 and 13579*24680 • Indexing operations—reading or writing elements in a sequence or dictionary—are also constant time regardless of the size of the data structure • A for loop that traverses a sequence or dictionary is usually linear, as long as all of the operations in the body of the loop are constant time