Analysis of Algorithms CS 477/677

Analysis of AlgorithmsCS 477/677 Instructor: Monica Nicolescu Lecture 4

Methods for Solving Recurrences • Iteration method • Substitution method • Recursion tree method • Master method CS 477/677 - Lecture 4

The recursion-tree method Convert the recurrence into a tree: • Each node represents the cost incurred at that level of recursion • Sum up the costs of all levels Used to “guess” a solution for the recurrence CS 477/677 - Lecture 4

Example 2 E.g.:T(n) = 3T(n/4) + cn2 • Subproblem size at level i is: n/4i • Subproblem size hits 1 when 1 = n/4i i = log4n • Cost of a node at level i = c(n/4i)2 • Number of nodes at level i = 3ilast level has 3log4n = nlog43nodes • Total cost: • T(n) = O(n2) CS 477/677 - Lecture 4

Example 2 - Substitution T(n) = 3T(n/4) + cn2 • Guess: T(n) = O(n2) • Induction goal: T(n) ≤ dn2, for some d and n≥ n0 • Induction hypothesis: T(n/4) ≤ d (n/4)2 • Proof of induction goal: T(n) = 3T(n/4) + cn2 ≤ 3d (n/4)2 + cn2 = (3/16) d n2 + cn2 = d n2 + cn2 + (3/16) d n2 - d n2 = d n2 + cn2 - (13/16) d n2 ≤ d n2 if: cn2 - (13/16) d n2 ≤ 0 d ≥ (16/13)c Therefore: T(n) = O(n2) CS 477/677 - Lecture 4

for all levels Example 3 W(n) = W(n/3) + W(2n/3) + n • The longest path from the root to a leaf is: n  (2/3)n  (2/3)2 n  …  1 • Subproblem size hits 1 when 1 = (2/3)in  i=log3/2n • Cost of the problem at level i = n • Total cost: •  W(n) = O(nlgn) CS 477/677 - Lecture 4

Example 3 - Substitution W(n) = W(n/3) + W(2n/3) + n • Guess: W(n) = O(nlgn) • Induction goal: W(n) ≤ dnlgn, for some d and n ≥ n0 • Induction hypothesis: W(k) ≤ d klgk for any k < n (n/3, 2n/3) • Proof of induction goal: Try it out as an exercise!! CS 477/677 - Lecture 4

Master’s method • “Cookbook” for solving recurrences of the form: where, a ≥ 1, b > 1, and f(n) > 0 Idea: compare f(n) withnlogba • f(n)is asymptotically smaller or larger than nlogba by a polynomial factor n • f(n) is asymptotically equal with nlogba CS 477/677 - Lecture 4

regularity condition Master’s method • “Cookbook” for solving recurrences of the form: where, a ≥ 1, b > 1, and f(n) > 0 Case 1: if f(n) = O(nlogba-)for some  > 0, then: T(n) = (nlogba) Case 2: if f(n) = (nlogba), then:T(n) = (nlogba lgn) Case 3: if f(n) = (nlogba+) for some  > 0, and if af(n/b) ≤ cf(n) for some c < 1 and all sufficiently large n, then: T(n) = (f(n)) CS 477/677 - Lecture 4

Why nlogba? • Case 1: • If f(n) is dominated by nlogba: • T(n) = (nlogbn) • Case 3: • If f(n) dominates nlogba: • T(n) = (f(n)) • Case 2: • If f(n) = (nlogba): • T(n) = (nlogba logn) • Assume n = bk k = logbn • At the end of iterations, i = k: CS 477/677 - Lecture 4

Examples T(n) = 2T(n/2) + n a = 2, b = 2, log22 = 1 Comparenlog22withf(n) = n  f(n) = (n)  Case 2  T(n) = (nlgn) CS 477/677 - Lecture 4

Examples T(n) = 2T(n/2) + n2 a = 2, b = 2, log22 = 1 Comparenwithf(n) = n2  f(n) = (n1+) Case 3 verify regularity cond. a f(n/b) ≤ c f(n)  2 n2/4 ≤ c n2  c = ½ is a solution (c<1)  T(n) = (n2) CS 477/677 - Lecture 4

Examples (cont.) T(n) = 2T(n/2) + a = 2, b = 2, log22 = 1 Comparenwithf(n) = n1/2 f(n) = O(n1-) Case 1 T(n) = (n) CS 477/677 - Lecture 4

Examples T(n) = 3T(n/4) + nlgn a = 3, b = 4, log43 = 0.793 Comparen0.793 withf(n) = nlgn f(n) = (nlog43+)Case 3 Check regularity condition: 3(n/4)lg(n/4) ≤ (3/4)nlgn = c f(n), c=3/4 T(n) = (nlgn) CS 477/677 - Lecture 4

Examples T(n) = 2T(n/2) + nlgn a = 2, b = 2, log22 = 1 • Compare n with f(n) = nlgn • seems like case 3 should apply • f(n) must be polynomially larger by a factor of n • In this case it is only larger by a factor of lgn CS 477/677 - Lecture 4

The Sorting Problem • Input: • A sequence of nnumbers a1, a2, . . . , an • Output: • A permutation (reordering) a1’, a2’, . . . , an’ of the input sequence such that a1’ ≤ a2’ ≤ · · · ≤ an’ CS 477/677 - Lecture 4

Why Study Sorting Algorithms? • There are a variety of situations that we can encounter • Do we have randomly ordered keys? • Are all keys distinct? • How large is the set of keys to be ordered? • Need guaranteed performance? • Does the algorithm sort in place? • Is the algorithm stable? • Various algorithms are better suited to some of these situations CS 477/677 - Lecture 4

Stability • A STABLE sort preserves relative order of records with equal keys Sort file on first key: Sort file on second key: Records with key value 3 are not in order on first key!! CS 477/677 - Lecture 4

Insertion Sort • Idea: like sorting a hand of playing cards • Start with an empty left hand and the cards facing down on the table • Remove one card at a time from the table, and insert it into the correct position in the left hand • compare it with each of the cards already in the hand, from right to left • The cards held in the left hand are sorted • these cards were originally the top cards of the pile on the table CS 477/677 - Lecture 4

Example CS 477/677 - Lecture 4

1 2 3 4 5 6 7 8 a1 a2 a3 a4 a5 a6 a7 a8 key INSERTION-SORT Alg.:INSERTION-SORT(A) for j ← 2to n do key ← A[ j ] Insert A[ j ] into the sorted sequence A[1 . . j -1] i ← j - 1 while i > 0 and A[i] > key do A[i + 1] ← A[i] i ← i – 1 A[i + 1] ← key • Insertion sort – sorts the elements in place CS 477/677 - Lecture 4

Loop Invariant for Insertion Sort Alg.:INSERTION-SORT(A) for j ← 2to n do key ← A[ j ] Insert A[ j ] into the sorted sequence A[1 . . j -1] i ← j - 1 while i > 0 and A[i] > key do A[i + 1] ← A[i] i ← i – 1 A[i + 1] ← key Invariant: at the start of each iteration of the for loop, the elements in A[1 . . j-1] arein sorted order CS 477/677 - Lecture 4

Proving Loop Invariants • Proving loop invariants works like induction • Initialization (base case): • It is true prior to the first iteration of the loop • Maintenance (inductive step): • If it is true before an iteration of the loop, it remains true before the next iteration • Termination: • When the loop terminates, the invariant gives us a useful property that helps show that the algorithm is correct CS 477/677 - Lecture 4

Readings • Chapter 2 CS 477/677 - Lecture 4

Analysis of Algorithms CS 477/677