ITEC 2620M Introduction to Data Structures

ITEC 2620MIntroduction to Data Structures Instructor: Prof. Z. Yang Course Website: http://people.math.yorku.ca/~zyang/itec2620m.htm Office: TEL 3049

Complexity Analysis

Key Points of this Lecture • Analysis of non-recursive algorithms • Estimation • Complexity Analysis • Big-Oh Notation

Factors in Cost Estimation • Does the program’s execution depend on the input? • Math.max(a, b); • always processes two numbers  constant time • maxValue(anArray); • processes n numbers  varies with array size

Value of Cost Estimation • Constant time programs • run once, always the same… • estimation not really required • Variable time programs • run once • future runs depend on relative size of input • based on what function?

Cost Analysis • Consider the following code: sum = 0; for (i=1; i<=n; i++) for (j=1; j<=n; j++) sum++; • It takes longer when n is larger.

Asymptotic Analysis • “What is the ultimate growth rate of an algorithm as a function of its input size?” • “If the problem size doubles, approximately how much longer will it take?” • Quadratic • Linear (linear search) • Logarithmic (binary search) • Exponential

Big-Oh Notation • Big-Oh represents the “order of” the cost function • ignoring all constants, find the largest function of n in the cost function • Selection Sort • n * n/2 compares + n swaps • O(n2) • Linear Search • n compares + n increments + 1 initialization • O(n)

Simplifying Conventions • If f(n) is in O(g(n) and g(n) is in O(h(n)), then f(n) is in O(h(n)). • If f(n) is in O(kg(n)) for any constant k>0, then f(n) is in O(g(n)). • If f1(n)is in O(g1(n)) and f2(n)is in O(g2(n)), then f1(n)+ f2(n) is in O(max(g1(n), g2(n)) ). • If f1(n)is in O(g1(n)) and f2(n)is in O(g2(n)), then f1(n)f2(n) is in O( (g1(n)g2(n)). • Summary • only focus on the largest function of n • ignore smaller terms (1&3) • ignore constants (2)

Simplifying Conventions (Cont’d) • size of functions of n • n! – factorial growth • cn – exponential growth • nc – polynomial growth • logn – logarithmic growth • c – constant • Sequential code sections • addition, find largest term (3) • Nested code sections (e.g. looping) • multiplication (4) • n times through loop  n/2 compares + 1 swap

Examples • Example 1: • Matrix multiplication Anm * Bmn = Cnn • Example 2: selectionSort(a); for (int i = 0; i < n; i++) binarySearch(i,a);

Trade-Offs and Limitations • “What is the dominant term in the cost function?” • What if the constant term is larger than n? • What happens if both algorithms have the same complexity? • Selection sort and Insertion sort are both O(n2) • Constants can matter • Same complexity (obvious) and different complexity (problem size)

ITEC 2620M Introduction to Data Structures