1 / 41

MS 101: Algorithms

MS 101: Algorithms. Instructor Neelima Gupta ngupta@cs.du.ac.in. Table Of Contents. Mathematical Induction: Review Growth Functions. Review: Mathematical Induction. Suppose S(k) is true for fixed constant k Often k = 0 S(n)  S(n+1) for all n >= k Then S(n) is true for all n >= k.

dreama
Download Presentation

MS 101: Algorithms

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. MS 101: Algorithms Instructor Neelima Gupta ngupta@cs.du.ac.in

  2. Table Of Contents Mathematical Induction: Review Growth Functions

  3. Review: Mathematical Induction • Suppose • S(k) is true for fixed constant k • Often k = 0 • S(n)  S(n+1) for all n >= k • Then S(n) is true for all n >= k

  4. Proof By Mathematical Induction • Claim:S(n) is true for all n >= k • Basis: • Show formula is true when n = k • Inductive hypothesis: • Assume formula is true for an arbitrary n • Step: • Show that formula is then true for n+1

  5. Strong Induction • Strong induction also holds • Basis: show S(0) • Hypothesis: assume S(k) holds for arbitrary k <= n • Step: Show S(n+1) follows • Another variation: • Basis: show S(0), S(1) • Hypothesis: assume S(n) and S(n+1) are true • Step: show S(n+2) follows

  6. Lets do it • Prove 1 + 2 + 3 + … + n = n(n+1) / 2 • Prove a0 + a1 + … + an = (an+1 - 1)/(a - 1) for all a  1

  7. Growth Functions • Big O Notation • In general a function • f(n) is O(g(n)) if there exist positive constants c and n0such that f(n)  c  g(n) for all n  n0 • Formally • O(g(n)) = { f(n):  positive constants c and n0such that f(n)  c  g(n)  n  n0 • Intuitively, it means f(n) grows no faster than g(n). • Examples: • n^2, n^2 – n • n^3, n^3 – n^2 – n

  8. f(n) = n2 g(n) = n2 – n Is f(n) = O(g(n))? Sol: g(n) = n2 – n = n2/2 + n2/2 - n ≥ n2/2 for n ≥ 2 = ½ f(n) f(n) ≤ 2g(n) for n ≥ 2 Hence, f(n) = O(g(n))

  9. f(n) = n3 g(n) = n3 - n2 – n Is f(n) = O(g(n))? Sol: g(n) = n3 - n2 – n = n3/2 + n3/2 - n2 - n ≥ n3/2 + n3/2 - n3/4 - n3/4 for n ≥ 4 = ½ f(n) f(n) ≤ 2g(n) for n ≥ 2 Hence, f(n) = O(g(n))

  10. f(n) = n3 g(n) = n3 - n2 – n Is g(n) = O(f(n))? Sol: Clearly, g(n) = n3 - n2 – n ≤3 n3for all n ≥ 1 = 3 f(n) g(n) ≤ 3f(n) V n ≥ 1 Hence, g(n) = O(f(n)).

  11. Omega Notation • In general a function • f(n) is (g(n)) if  positive constants c and n0such that 0  cg(n)  f(n)  n  n0 Intuitively, it means f(n) grows at least as fast as g(n). • Examples: • n^2, n^2 + n • n^3, n^3 + n^2 – n

  12. Ques: f(n) = n2, g(n) = n2 + n Is f(n) = Ω(g(n))? Sol: n2 = ½ (n2 + n2 ) ≥ ½ (n2 + n) f(n) ≥ c g(n) for c = ½ m =1

  13. f(n) = n3 g(n) = n3 + 4n2 - 5n Is g(n) = Ω(f(n))? Sol: g(n) = n3 + 4n2 - 5n ≥ n3 /2 +n3 /2 - 4 n2 - 5n2 for all n ≥ 1 = n3 /2 +n2 (n/2 - 9) ≥ n3 /2 for n ≥ 18 = 1/2 f(n) Hence, g(n) = Ω (f(n)).

  14. Theta Notation • A function f(n) is (g(n)) if  positive constants c1, c2, and n0 such that c1 g(n)  f(n)  c2 g(n)  n  n0

  15. Relations Between Q, W, O Theorem : For any two functions g(n) and f(n), f(n) = (g(n)) iff f(n) =O(g(n)) and f(n) = (g(n)). Proof: Simple. Do it.

  16. Assignment No 1 Q1 • Already proved a0 + a1 + … + an = (an+1 - 1)/(a - 1) for all a  1 • What is the sum for a = 2/3 as n  infinity? Is it O(1)? Is it big or small? • For a = 2, is the sum = O(2^n)? Is it big or small? Q2 • Show that a polynomial of degree k = theta(n^k).

  17. Other Asymptotic Notations • A function f(n) is o(g(n)) if for every positive constant c, there exists a constant n0 > 0 such that f(n) < c g(n)  n  n0 • A function f(n) is (g(n)) if for every positive constant c, there exists a constant n0 > 0 such that c g(n) < f(n)  n  n0 • Intuitively, • () is like > • () is like  • () is like = • o() is like < • O() is like 

  18. Arrange some functions • f(n) = O(g(n)) => f(n) = o(g(n)) ? • Is the converse true? • Let us arrange the following functions in ascending order (assume log n = o(n) is known) • n, n^2, n^3, sqrt(n), n^epsilon, log n, log^2 n, n log n, n/log n, 2^n, 3^n

  19. Relation between n & n2 Solution: let c > 0 be any constant such that n < c n2 i.e. 1 < c n i.e. n > 1 / c Hence n < c n2 for n > 1 / c i.e. n = o( n2) we can also write it as n < n2.

  20. Relation between n2 & n3 Solution: let c> 0 be any constant such that n2 < c n3 Þ 1 < c n Þ n > 1 / c Hence n2 < c n3 " n > 1 / c i.e. n2 = o( n3) we can also write it as n2 < n3. Combining the previous result we can write n < n2 < n3

  21. Relation between n & n1/2 Solution: let c> 0 be any constant such that n1/2 < c n i.e n < c2 n2 i.e 1 < c2 n i.e n > 1/ c2 i.e. n1/2 < c n for n > 1/c2 i.e. n2 = o( n3) we can also write it as n1/2 < n. Combining the previous result n1/2 < n < n2 < n3

  22. Relation between n & log n • For the time being we can assume the result log ( n ) = o(n) Þ log ( n ) < n we will prove it later.

  23. Relation between n1/2 & log n Assume log n = o(n) let c > 0 be any constant for c/2 > 0 there exists m > 0 such that log n < (c/2) n for n > m changing variables from n to n1/2 we get log(n1/2 ) < (c/2) n1/2 for n1/2 > m ½ log( n ) < (c/2) n1/2 for n > m2

  24. Contd.. let m2 = k log( n ) < c n1/2 forn > k Since c > 0 was chosen arbitrarily hence log n = o( n1/2 ) or log n < n1/2 Combining the results we get log n < n1/2 < n < n2 < n3

  25. Relation between n2 & nlog n • Since log n = o(n) for c > 0, n0 > 0 such that n n0, we have log n < c n Multiplying by n on both sides we get n log( n ) < c n2 n n0 • nlog n = o( n2 ) • nlog n < n2

  26. Relation between n & nlog n Solution: let c> 0 be any constant such that n < c n log (n) Þ1 < c log( n ) Þ log( n) > 1 / c Þn > e1/c i.e. n < c n log n " n > e1/c Since c was chosen arbitrarily \ n = o( n log n ) or n < n log n Combining the results we can get log n < n1/2 < n < n logn < n2 < n3

  27. Relation between n & n/log n • We know that n = o(nlogn) for c > 0, n0 > 0 such that n n0, we have n < c n log n dividing both sides by log n we get n/ log( n) < c n n n0 Þ n / logn = o(n) i.e. n / logn < n

  28. Assignment No 2 • Show that log^M n = o(n^epsilon) for all constants M>0 and epsilon > 0. Assume that log n = o(n). Also prove the following Corollary: log n = o(n/log n) • Show that n^epsilon = o(n/logn ) for every 0 < epsilon < 1 .

  29. Hence we have, log n < n/log n < n1/2 < n < n logn < n2 < n3

  30. Assignment No 3 • Show that • lim f(n)/g(n) = 0 => f(n) = o(g(n)). n →∞ • lim f(n)/g(n) = c => f(n) = θ(g(n)). n →∞, where c is a positive constant. • Show that log n = o(n). • Show that n^k = o(2^n) for every positive constant k.

  31. Show by definition of ‘small o’ that a^n = o(b^n) whenever a < b , a and b are positive constants. Hence we have, log n < n/log n < n1/2 < n < n logn < n2 < n3 <2n < 3n

  32. Why the constants ‘c’ and ‘m’? • Suppose we have two algorithms to solve the problem say sorting: Insertion Sort and Merge sort for eg. • Why should we have more than one algorithm to solve the same problem? • Ans: efficiency. • What’s the measure of efficiency? • Ans: System resources for example ‘time’. • How do we measure time?

  33. Contd.. • IS(n) = O(n^2) • MS(n) = O(nlog n) • MS(n) is faster than IS(n). • Suppose we run IS on a fast machine and MS on a slow machine and measure the time (since they were developed by two different people living in different part of the globe), we may get less time for IS and more for MS…wrong analysis • Solution: count the number of steps on a generic computational model

  34. Computational Model: Analysis of Algorithms • Analysis is performed with respect to a computational model • We will usually use a generic uniprocessor random-access machine (RAM) • All memory equally expensive to access • No concurrent operations • All reasonable instructions take unit time • Except, of course, function calls • Constant word size • Unless we are explicitly manipulating bits

  35. Running Time • Number of primitive steps that are executed • Except for time of executing a function call, in this model most statements roughly require the same amount of time • y = m * x + b • c = 5 / 9 * (t - 32 ) • z = f(x) + g(y) • We can be more exact if need be

  36. But why ‘c’ and ‘m’? • Because • We compare two algorithms on the basis of their number of steps and • the actual time taken by an algorithm is ‘c’ times the number of steps.

  37. Why ‘m’? • We need efficient algorithms and computational tools to solve problems on big data. For example, it is not very difficult to sort a pack of 52 cards manually. However, to sort all the books in a library on their accession number might be tedious if done manually. • So we want to compare algorithms for large input.

  38. An Example: Insertion Sort InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} }

  39. Assignment 4 • Show that Insertion Sort takes O(n^2) steps by counting each and every step. • Is it O(n)? • Is it O(n^3)?

  40. Lower Bound Notation • We say InsertionSort’s run time is (n) • Proof: • Suppose run time is an + b • Assume a and b are positive (what if b is negative?) • an  an + b

  41. Up Next • Solving recurrences • Substitution method • Master theorem

More Related