Understanding Orders of Growth in Discrete Mathematics

1. 이산수학(Discrete Mathematics) 함수의 증가(Growth of Functions) 강원대학교 컴퓨터과학전공 문양세

2. Orders of Growth – Introduction 2.2 Growth of Functions For functions over numbers, we often need to know a rough measure of how fast a function grows. (과연 함수는 얼마나 빨리 증가하는가?) If f(x) is faster growing than g(x), then f(x) always eventually becomes larger than g(x) in the limit (for large enough values of x). (f(x)가 g(x)보다 빨리 증가한다면, 궁극적으로 f(x)의 값이 g(x)의 값보다 커진다.)

3. Orders of Growth – Motivation 2.2 Growth of Functions • Suppose you are designing a web site to process user data. • Suppose database • program A takes fA(n)=30n+8 microseconds to process any n records, while • program B takes fB(n)=n2+1 microseconds to process the n records. • Which program do you choose, knowing you’ll want to support millions of users? A

4. Visualizing Orders of Growth 2.2 Growth of Functions On a graph, as you go to the right, a faster growing function eventually becomes larger... fA(n)=30n+8 Value of function  fB(n)=n2+1 Increasing n 

5. Concept of Orders of Growth 2.2 Growth of Functions We say fA(n)=30n+8is order n, or O(n). It is, at most, roughly proportional to n. fB(n)=n2+1 is order n2, or O(n2). It is roughly proportional to n2. Any O(n2) function is faster-growing than any O(n) function. For large numbers of user records, the O(n2) function will always take more time.

6. Definition: O(g), at most order g 2.2 Growth of Functions Let g be any function RR. • Define “at most order g”, written O(g), to be: {f:RR | c,k: x>k: f(x)  cg(x)}. • “Beyond some point k, function f is at most a constant c times g (i.e., proportional to g).” • (x > k일 때, f(x)  cg(x)를 만족하는 c, k가 존재하면, f는 O(g)라 한다.) • “f is at most order g”, or “f is O(g)”, or “f=O(g)” all just mean that fO(g). • Sometimes the phrase “at most” is omitted.(일반적으로 그냥 “오더 g”라 이야기한다.)

7. Points about the Definition 2.2 Growth of Functions Note that f is O(g) so long as any values of c and k exist that satisfy the definition. But: The particular c, k, values that make the statement true are not unique: Any larger value of c and/or k will also work. (조건을 만족하는 c, k는 여러 개 있을 수 있다.) You are not required to find the smallest c and k values that work. (Indeed, in some cases, there may be no smallest values!)(조건을 만족하는 가장 작은 c, k를 찾을 필요는 없다. 보여주기만 하면 된다.) However, you should prove that the values you choose do work. (증명하기 위해서는 만족하는 c, k를 하나는 찾아야 한다.)

8. “Big-O” Proof Example 2.2 Growth of Functions • Show that 30n+8 is O(n). • Show c,k: n>k:30n+8  cn. • Let c=31, k=8. Assume n>k(=8). • Then cn = 31n = 30n + n > 30n+8, so 30n+8 < cn. • Show that n2+1 is O(n2). • Show c,k: n>k: n2+1  cn2. • Let c=2, k=1. Assume n>1. • Then cn2 = 2n2 = n2+n2 > n2+1, or n2+1< cn2.

9. cn =31n n>k=8  Big-O Example, Graphically 2.2 Growth of Functions Note 30n+8 isn’t less than nanywhere (n>0). It isn’t even less than 31neverywhere. But it is less than 31neverywhere to the right of n=8. E.g., c = 31, k = 8 30n+8 30n+8O(n) Value of function  n Increasing n 

10. Examples of Big-O (1/2) 2.2 Growth of Functions • f(x) = anxn + an-1xn-1 + … + a1x + a0 • f(x) = anxn + an-1xn-1 + … + a1x + a0  |an|xn + |an-1|xn-1 + … + |a1|x + |a0| = xn(|an| + |an-1|/x + … + |a1|/xn-1 + |a0|/xn)  xn(|an| + |an-1| + … + |a1| + |a0|) = cxn = O(xn) • f(n) = 1 + 2 + 3 + … + n • f(n) = 1 + 2 + 3 + … + n  n + n + n + … + n = n2 = O(n2) c

11. Examples of Big-O (2/2) 2.2 Growth of Functions • f(n) = n! • f(n) = n! = n∙(n-1)∙(n-2)∙…∙3∙2∙1  n∙n∙n∙…∙n∙n∙n = nn = O(nn) • f(n) = log(n!) • f(n) = log(n!)  log(nn) = nlogn= O(nlogn)

12. Useful Facts about Big-O (1/2) 2.2 Growth of Functions • Big O, as a relation, is transitive: fO(g)  gO(h)  fO(h) • Sums of functions: If gO(f) and hO(f), then g+hO(f). • c>0, O(cf)=O(f+c)=O(fc)=O(f) • E.g., O(2x2) = O(x2+2) = O(x2-3) = O(x2) • If f1=O(g1) and f2= O(g2), then f1+f2= O(max(g1,g2)) • E.g., if f1(x) = x2 = O(x2), f2(x) = x3 = O(x3), then (f1+f2)(x) = x2 + x3 = O(x3) = O(max(x2, x3))

13. Useful Facts about Big-O (2/2) 2.2 Growth of Functions • f1=O(g1) and f2= O(g2), then f1f2= O(g1g2) • E.g., if f1(x) = x2 = O(x2), f2(x) = x3 = O(x3), then (f1f2)(x) = x2∙x3 = x5 = O(x5) = O(x2∙x3)) • Other useful facts… •  f,g & constantsa,bR, with b0, • af = O(f) (e.g. 3x2 = O(x2)) • f+O(f) = O(f) (e.g. x2+x = O(x2)) • Also, the followings hold (그냥 참고하세요): • |f|1-b= O(f); (e.g.x1 = O(x)) • (logb |f|)a= O(f) (e.g. log x = O(x)) • g=O(fg) (e.g.x = O(x log x)) • fg O(g) (e.g.x log x  O(x)) • a=O(f) (e.g. 3 = O(x))

14. Definition: (g), exactly order g 2.2 Growth of Functions If fO(g) and gO(f) then we say “g and f are of the same order” or “f is (exactly) order g” and write f(g). (f(x)=O(g(x))이고 g(x)=O(f(x))이면, f(x)=(g(x))이며 g(x)=(f(x))이다.) Another equivalent definition: (g)  {f:RR | c1c2k x>k: |c1g(x)||f(x)||c2g(x)| } (c1g(x) f(x) c2g(x)를 만족하는 c1과 c2가 존재하면 f(x)=(g(x))이다.)

15. Rules for  2.2 Growth of Functions Mostly like rules for O( ), except: f,g>0 & constantsa,bR, with b>0,af (f), but  Same as with O.f  (fg) unless g=(1)  Unlike O.|f|1-b (f), and  Unlike with O.(참고)(logb|f|)c (f).  Unlike with O. (참고)

16. Examples of  (1/2) 2.2 Growth of Functions • f(n) = 1 + 2 + 3 + … + n = (n2)? • f(n) = 1 + 2 + 3 + … + n  n + n + n + … + n = n2 • f(n) = 1 + 2 + 3 + … + n = (n∙(n + 1))/2 = n2/2 + n/2  n2/2 • n2/2 f(n)  n2, i.e., c1 = ½, c2 = 1 f(n) = (n2)

17. Examples of  (2/2) 2.2 Growth of Functions • f(y) = 3y2 + 8ylogy= (y2)? • f(n) = 3y2 + 8ylogy 11y2 (if y > 1) (since 8ylogy 8y2) • f(n) = 3y2 + 8ylogy  y2 (if y > 1) • y2 f(y)  11y2, i.e., c1 = 1, c2 = 11 f(y) = (y2)