1 / 20

ALGORITHM ANALYSIS

Explore the analysis of resource consumption by a processor in algorithm implementation, focusing on time and memory usage. Learn about the importance of worst-case scenarios and proof of correct algorithm functionality by Debasit Mitra.

kmoye
Download Presentation

ALGORITHM ANALYSIS

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ALGORITHM ANALYSIS Analysis of Resource consumption by processor Time Memory Number of processors Power Proof that the algorithm works correctly Debasis Mitra

  2. ALGORITHM ANALYSIS Time is measured with Step Counts in an Algorithm Why? Not a constant number, but as a function of Input Size Why? We are interested primarily in Worst Case Scenario: Why? Debasis Mitra

  3. WHY STEP-COUNTED? Code run time varies depending on Processor speed Language I/O … What is the number of steps by the following fragment? For i=1:100 { for j= 1: 2 {print i}}; Debasis Mitra

  4. Why time-complexity as function of input-size: n? What is the number of steps by the following fragment? For i=1:n { for j= 1: m {print i}}; Algorithms are to work on different problem sizes What is the “input size” Number of elements in input, e.g. array size in a search problem Debasis Mitra

  5. Why Worst-case scenario? What is the time-complexity (#steps) search alg below? Input array A[1..n] = [3, 9, 5, 7], search key k =3; For i=1:n { if k = = A[i] {print i; return}}; Best case time complexity is “normally” meaningless Sometimes Average-case analysis is done, assuming some distribution of input types Debasis Mitra

  6. Upper bound (Big-Oh): t(N) = O(f(N)) if there exists a positive constant c (real) and a corresponding integer n0 such that t(N)  c.f(N) when N  n0, for some integer n0. Related definitions: Lower bound: t(N) = (g(N)) if there exists a positive constant c (real) and a corresponding integer n0 such that t(N)  c.g(N) when N n0. Tight bound: T(N) = (h(N)) if and only if t(N) = O(h(N)) and t(N) = (h(N)). Reminding of Complexity Functions Debasis Mitra

  7. Lim n->inf [f(n) / g(n)] = constant, implies both f(n) and g(n) have the same order of terms in n, or, f(n) = (g(n)) Example: f(n) = 3n2 + 5n, g(n) = 7n2, the limit is 3/7 Lim n->inf [f(n) / g(n)] = 0, implies g(n) has higher order term in n than that in f(n), or, f(n) = O(g(n)) Example: f(n) = 3n2 + 5n, g(n) = 2n3, the limit is 0 Lim n->inf [f(n) / g(n)] = inf, implies f(n) has higher order term in n than that in g(n), or, f(n) = (g(n)) Example: f(n) = 3n2 + 5n, g(n) = 22n, the limit is infinity A short-cut way of comparing two functions f(n) and g(n): Debasis Mitra

  8. Points to Note Order complexities are typically monotonically increasing functions with respect to problem-input size n. We typically consider only dominant term ignoring the other ones: 2(n2) + 3n we ignore the 3n, and the constant multiplicand 2 (absorbed in c). So, 2(n2) + 3n = O(n2). It is actually ( n2). O is often loosely (and confusingly) used for  in the literature, even in this class! n2+ 3n = O(n2), also = O(n3), and = O(n4), … However, we will prefer the closest one O(n2) as the acceptable order notation for the function [actually we are referring to ( n2)]. Increasing order of functions: Constantor O(1), O(log N), O(log2 N), …, O(N), O(N logN), O(N2), O(N2log N), O(N3), …, O(2N), O(3N), … Debasis Mitra

  9. GENERAL RULES: Loops: number of iterations/ recursions with respect to the input problem instance size N Nested loops: multiplied for I = 1 through N do for j = 1 through N do -whatever- Analysis: O(N*N) Consecutive statements: add steps, which leads to the max for I = 1 through N do -whatever- for I = 1 through N do for j = 1 through N do -moreover- Analysis: O(N) + O(N2) --> O(N2) Conditional statements: max of the alternative paths (worst-case) If (condition) S1 Else S2 Debasis Mitra

  10. Why Study Complexity? Code Fibonacci Series recursive & iterative algorithms; and time against input sizes Debasis Mitra

  11. Background Needed • Data Structures: Big-O notations, Linked list, Sorting, Searching, recursion,,, • Discrete Mathematics: Set theory, Logic, Recurrence equation, Matrix operations,,, • Programming experience Debasis Mitra

  12. Syllabus overview • Introduction • Four algorithm types: divide-conquer, greedy, dynamic programming, branch-bound • Graph algorithms (a few only!) • Complexity theory – NP-completeness • An advanced topic, e.g. linear programming • Undergrad: GPU coding project; Grad: An advanced topic self-study report Debasis Mitra

  13. Syllabus objective • More conscious problem solver • Exposure to proving correctness of algorithm (and its connection to both solving a problem, and computing its complexity) • Fundamental understanding of computing problem complexity • Undergrad project: GPU computing; Grad project: introduction to a current topic Debasis Mitra

  14. Example: MAXIMUM SUBSEQUENCE SUM (MSQS) Problem MSSQ: Given an array of numbers find a subsequence whose sum is maximum out of all such subsequences. Example: 3, 4, –7, 1, 9, -2, 3, -1 (e.g. rise and fall in stock-market) Answer: 11 (for subsequence 1+ 9 -2+ 3 = 11) [Note: for all positive integers the answer is sum of the whole array.] Debasis Mitra

  15. MSQS Algorithm 1 • Input: Array A[N], e.g. [3, 4, –7, 1, 9, -2, 3, -1 ] • Output: Maximum subsequence sum, e.g., 11 • Algorithm 1: • maxSum = 0; // We expect non-negative value here at the end • for (i = 0 through N-1) do • for (j = i through N-1) do // choice of subsequence i through j • { thisSum = 0; • for (k = i through j) do // addition loop • thisSum = thisSum + a[k]; // O(N3) • if (thisSum > maxSum) then • maxSum = thisSum; // O(N2) • } • return maxSum; End Algorithm 1. • Analysis of Algorithm 1: i=0N-1 j=iN-1 k=ij 1 = … = O(N3) Debasis Mitra

  16. MSQS Algorithm 2 • Input: Array A[N], e.g. [3, 4, –7, 1, 9, -2, 3, -1 ] • Output: Maximum subsequence sum, e.g., 11 • Algorithm 2: • maxSum = 0; // We expect non-negative value here at the end • for (i = 0 through N-1) do • thisSum = 0; • for (j = i through N-1) do • { thisSum = thisSum + a[j]; // reuse the partial sum from • // the previous iteration • if (thisSum > maxSum) then • maxSum = thisSum; • } • return maxSum; • End Algorithm 2. • Analysis of Algorithm 2: i=0N-1 j=iN-1 1 = … = O(N2) Debasis Mitra

  17. MSQS Algorithm 3 Complexity: T(n) = 2T(n/2) + O(N) T(1) = 1 Solve: T(n) = O(n log n) Inductive Proof: (base) n=1; (hypothesis) lines 15, 16 & 18-32 work correctly for n=2^k (step) These are only three options; So, lines 34-35 returns correct sum. Weiss, textbook, 1999 Debasis Mitra

  18. MSQS Algorithm 4 • Input: Array A[N], e.g. [3, 4, –7, 1, 9, -2, 3, -1 ] • Output: Maximum subsequence sum, e.g., 11 • Algorithm 4: • maxSum = 0; thisSum = 0; • for (j = 0 through N-1) do • { thisSum = thisSum + a[j]; // reuse the partial sum from • // the previous iteration • if (thisSum > maxSum) then • maxSum = thisSum; • else if (thisSum < 0) then • thisSum =0; // ignore computations so far • } • return maxSum; End Algorithm 4. • Analysis of Algorithm 4: i=0N-1 O(1) = … = O(N) Exercise: (1) How to track the start-end indices for the maxSum solution? (2) Prove the algorithm 4, stating any underlying assumption/hypothesis. Debasis Mitra

  19. Proof of MSQS Algo-4 • Lemma 1: Maximum-sub-sequence-sum is non negative. Proof: trivial. (the problem is defined as such) • Lemma 2: No pre-fix of a non-negative maximum-sub-sequence-sum (MSQS) can be negative. • Proof by contradiction: Presume such a MSQS P =(W Q), W is prefix and Q is suffix parts. • We know, ∑P = ∑W + ∑Q; if ∑W <0 then ∑P < ∑Q; Hence, ∑P is not MSQS. • Thm: MSQS Algo-4 finds the MSQS correctly. • Proof sketch by induction on loop invariant maxSum: • Base: for null input {} maxSum=0 is MSQS; • Hypothesis: at every iteration j = k, maxSum is MSQS up to (a1, a2, …, ak-1) • Step: if maxSum+ak <0 then by Lemma 2 it cannot be prefix in any MSQS, • and step 8 rejects the sequence built so far. • Input: Array A[N], e.g. [3, 4, –7, 1, 9, -2, 3, -1 ] • Output: Maximum subsequence sum, e.g., 11 • Algorithm 4: • maxSum = 0; thisSum = 0; • for (j = 0 through N-1) do • { thisSum = thisSum + a[j]; // reuse the partial sum from • // the previous iteration • if (thisSum > maxSum) then • maxSum = thisSum; • else if (thisSum < 0) then • thisSum =0; // ignore computations so far • } • return maxSum; End Algorithm 4. Debasis Mitra

  20. What do we analyze in an algorithm, why, and how do we analyze? Basic O-notations How complexity matters Exercise: Code the following two fibonacci series algorithms and increase input values, check time. What are the step/time-complexities? Space-complexities? Input: an integer n; Output: Fibonacci number for n Summary Debasis Mitra

More Related