資訊科學數學 9 : Integers, Algorithms & Orders

1. 資訊科學數學9 :Integers, Algorithms & Orders 陳光琦助理教授 (Kuang-Chi Chen) chichen6@mail.tcu.edu.tw

2. Mini Review Methods

3. Useful Congruence Theorems Theorem 1: Let a, bZ, mZ+. Then:ab (mod m)  kZa=b+km. Theorem 2: Let a, b, c, dZ, mZ+. Then if ab (mod m) and cd (mod m), then: . a+c b+d (mod m), and . ac  bd (mod m)

4. Integers and Algorithms

5. Integers & Algorithms Topics: • Euclidean algorithm for finding GCD’s. • Base-b representations of integers. - Especially: binary, hexadecimal, octal. - Also: Two’s complement representation of negative numbers. • Algorithms for computer arithmetic: - Binary addition, multiplication, division.

6. Euclid’s Algorithm for GCD • Finding GCDs by comparing prime factorizations can be difficult when the prime factors are not known! • Euclid discovered: For all ints. a, b,gcd(a, b) = gcd((a mod b), b). • Sort a, b so that a>b, and then (given b>1)(a mod b) < a, so problem is simplified. Euclid325-265 B.C.

7. Euclid’s Algorithm Example • gcd(372,164) = gcd(372 mod 164, 164). - 372 mod 164 = 372164[372/164] = 372164·2 = 372328 = 44. • gcd(164, 44) = gcd(164 mod 44, 44). - 164 mod 44 = 16444[164/44] = 16444·3 = 164132 = 32. • gcd(44, 32) = gcd(44 mod 32, 32) = gcd(12, 32) = gcd(32 mod 12, 12) = gcd(8, 12) = gcd(12 mod 8, 8) = gcd(4, 8) = gcd(8 mod 4, 4) = gcd(0, 4) = 4.

8. Euclid’s Algorithm Pseudocode procedure gcd(a, b: positive integers) whileb  0 begin r≔amodb; a≔b; b≔r; end return a Sorting inputs not needed b/c order will be reversed each iteration. Fast! Number of while loop iterationsturns out to be O(log(max(a,b))).

9. The “base b expansion of n” Base-b Number Systems • Ordinarily, we write base-10 representations of numbers, using digits 0-9. • But, 10 isn’t special! Any base b>1 will work. • For any positive integers n,b, there is a unique sequence ak ak-1… a1a0of digitsai<b such that:

10. Used only because we have 10 fingers Used internally in all modern computers Octal digits correspond to groups of 3 bits Particular Bases of Interest • Base b=10 (decimal):10 digits: 0,1,2,3,4,5,6,7,8,9. • Base b=2 (binary):2 digits: 0,1. (“Bits”=“binary digits.”) • Base b=8 (octal):8 digits: 0,1,2,3,4,5,6,7. • Base b=16 (hexadecimal):16 digits: 0,1,2,3,4,5,6,7,8,9,A,B,C,D,E,F Hex digits give groups of 4 bits

11. Converting to Base b (An algorithm, informally stated.) • To convert any integer n to any base b>1: 1. To find the value of the rightmost (lowest-order) digit, simply compute n mod b. 2. Now, replace n with the quotient [n/b]. 3. Repeat above two steps to find subsequent digits, until n is gone (=0).

12. Orders of Growth

13. Orders of Growth • For functions over numbers, we often need to know a rough measure of how fast a function grows. • If f(x) is faster growing than g(x), then f(x) always eventually becomes larger than g(x) in the limit (for large enough values of x). • Useful in engineering for showing that one design scalesbetter or worse than another.

14. Orders of Growth - Motivation • Suppose you are designing a web site to process user data (e.g., financial records). • Suppose database program A takes fA(n)=30n+8 microseconds to process any n records, while program B takes fB(n)=n2+1 microseconds to process the n records. Q: Which program do you choose, knowing you’ll want to support millions of users? A

15. fB(n)=n2+1 Visualizing Orders of Growth • On a graph, asyou go to theright, the faster-growing function always eventuallybecomes thelarger one... fA(n)=30n+8 Value of function  Increasing n 

16. Concept of Order of Growth • We say fA(n) = 30n+8is (at most) order n, or O(n). - It is, at most, roughly proportional to n. • fB(n) = n2+1 is order n2, or O(n2). - It is (at most) roughly proportional to n2. • Any function whose exact (tightest)order is O(n2) is faster-growing than any O(n) function. - Later we’ll introduce Θ for expressing exact order. • For large numbers of user records, the exactly order n2 function will always take more time.

17. Definition: O(g), at mostorder g Let g be any function RR. • Define “at most order g”, written O(g), to be: {f: RR | c, k: x>k: f(x)  cg(x)}. - “Beyond some point k, function f is at most a constant c times g (i.e., proportional to g).” • “f is at most order g”, or “f is O(g)”, or “f=O(g)” all just mean that fO(g). (Often the phrase “at most” is omitted.)

18. Points about The Definition • Note that f is O(g) so long as any values of c and k exist that satisfy the definition. • But, the particular c, k, values that make the statement true are not unique: Any larger value of c and/or k will also work. • No need to find the smallest c and k values that work. (Indeed, in some cases, there may be no smallest values!) However, you should prove that the values you choose do work.

19. “Big-O” Proof Examples • Show that 30n+8 is O(n). Show c, k: n>k:30n+8  cn. Let c=31, k=8. Assume n>k=8. Thencn = 31n = 30n + n > 30n+8, so 30n+8 < cn. • Show that n2+1 is O(n2). Show c, k: n>k: n2+1  cn2. Let c=2, k=1. Assume n>1. Then cn2 = 2n2 = n2+n2 > n2+1, or n2+1< cn2.

20. cn =31n n>k=8  Big-O Example, Graphically • Note 30n+8 isn’tless than nanywhere (n>0). • It isn’t evenless than 31neverywhere. • But it is less than31neverywhere tothe right of n=8. 30n+8 30n+8O(n) Value of function  n Increasing n 

21. Useful Facts about Big O • Big O, as a relation, is transitive: fO(g)  gO(h)  fO(h) • O with constant multiples, roots, and logs... f (in (1)) & constantsa, bR, with b0,af, f 1-b, and (logbf)aare all O(f). • Sums of functions:If gO(f) and hO(f), then g+hO(f).

22. More Big-O Facts • c>0, O(cf) = O(f+c) = O(fc) = O(f) f1O(g1)  f2O(g2)  - f1 f2 O(g1g2) - f1+f2 O(g1+g2) = O(max(g1, g2)) = O(g1) if g2O(g1) (Very useful!)

23. Orders of Growth - So Far • For any g: RR, “at most order g”,O(g)  {f: RR | c, k x>k |f(x)|  |cg(x)|}. - Often, one deals only with positive functions and can ignore absolute value symbols. • “fO(g)” often written “f is O(g)”or “f = O(g)”. - The latter form is an instance of a more general convention...

24. Order-of-Growth Expressions • “O(f)” when used as a term in an arithmetic expression means: “some function f such that fO(f)”. E.g., “x2+O(x)” means “x2 plus some function that is O(x)”. • Formally, we can think of any such expression as denoting a set of functions: x2+O(x) : {g | fO(x): g(x) = x2+f(x)}

25. Order of Growth Equations • Suppose E1 and E2 are order-of-growth expressions corresponding to the sets of functions S and T, respectively. Then the “equation” E1=E2 really meansfS, gT : f = gor simply ST. Example: x2 + O(x) = O(x2) means fO(x): gO(x2): x2+f(x) = g(x)

26. Useful Facts about Big O •  f, g & constantsa, bR, with b0, - af = O(f) (e.g. 3x2 = O(x2)) - f+O(f) = O(f) (e.g. x2+x = O(x2)) • Also, if f = (1) (at least order 1), then: - |f|1-b = O(f) (e.g.x1 = O(x)) - (logb |f|)a= O(f) (e.g. log x = O(x)) - g = O(fg) (e.g.x = O(x log x)) - fg O(g) (e.g.x log x  O(x)) - a= O(f) (e.g. 3 = O(x))

28. Definition: (g), exactly orderg • If fO(g) and gO(f), then we say “g and f are of the same order” or “f is (exactly) order g” and write f(g). • Another, equivalent definition:(g)  {f:RR | c1c2k>0 x>k: |c1g(x)||f(x)||c2g(x)| } - “Everywhere beyond some point k, f(x)lies in between two multiples of g(x).”

29. Rules for  • Mostly like rules for O( ), except: •  f,g>0 & constantsa,bR, with b>0,af (f), but  Same as with O.f  (fg) unless g=(1)  Unlike O.|f| 1-b (f), and  Unlike with O.(logb |f|)c (f).  Unlike with O. • The functions in the latter two cases we say are strictly of lower order than (f).

30.  Example • Determine whether: • Quick solution:

31. Other Order-of-Growth Relations • (g) = {f | gO(f)}“The functions that are at least orderg.” • o(g) = {f | c>0 k x>k : |f(x)| < |cg(x)|}“The functions that are strictly lower order than g.” o(g)  O(g)  (g). • (g) = {f | c>0 k x>k : |cg(x)| < |f(x)|}“The functions that are strictly higher order than g.” (g)  (g)  (g).

32. Relations Between the Relations • Subset relations between order-of-growth sets. RR ( f ) O( f ) • f o( f ) ( f ) ( f )

33. Why o(f)  O(x)(x) • A function that is O(x), but neither o(x) nor (x):

34. Strict Ordering of Functions • Temporarily let’s write fg to mean fo(g),f~g to mean f(g) • Note that: • Let k>1. Then the following are true:1  log log n log n ~ logkn logknn1/k n n log nnkkn n! nn …

35. Review: Orders of Growth Definitions of order-of-growth sets, g:RR • O(g) : {f|  c>0 k x>k |f(x)| < |cg(x)|} • o(g) : {f | c>0 k x>k |f(x)| < |cg(x)|} • (g) : {f | gO(f)} • (g) : {f | go(f)} • (g) : O(g)  (g)

37. Addition of Binary Numbers procedureadd(an−1…a0, bn−1…b0: binary representations of non-negative integers a, b) carry := 0 forbitIndex := 0 to n−1 {go through bits} bitSum := abitIndex+bbitIndex+carry {2-bit sum} sbitIndex := bitSummod 2 {low bit of sum} carry := bitSum / 2 {high bit of sum} sn := carry returnsn…s0: binary representation of integer s

38. Two’s Complement • In binary, negative numbers can be conveniently represented using two’s complement notation. • In this scheme, a string of n bits can represent any integer i such that −2n−1 ≤ i < 2n−1. • The bit in the highest-order bit-position (#n−1) represents a coefficient multiplying −2n−1; the other positions i < n−1 just represent 2i, as before. • The negation of any n-bit two’s complement number a = an−1…a0 is given by an−1…a0 + 1. The bitwise logical complement of the n-bit string an−1…a0.

39. Correctness of Negation Algorithm Theorem: For an integer a represented in two’s complement notation, −a = a + 1. Proof:a = −an−12n−1 + an−22n−2 + … + a020, so −a = an−12n−1 − an−22n−2 − … − a020. Note an−12n−1= (1−an−1)2n−1 = 2n−1 − an−12n−1. But 2n−1 = 2n−2 + … + 20 + 1. So we have −a = − an−12n−1 + (1−an−2)2n−2 + … + (1−a0)20 + 1 = a + 1.

40. Subtraction of Binary Numbers proceduresubtract(an−1…a0, bn−1…b0: binary two’s complement reps. of integers a, b) returnadd(a, add(b,1)) { a + (−b) } Note that this fails if either of the adds causes a carry into or out of the n−1 position, since 2n−2+2n−2 ≠ −2n−1, and −2n−1 + (−2n−1) = −2n isn’t representable! We call this an overflow.

41. Multiplication of Binary Numbers proceduremultiply(an−1…a0, bn−1…b0: binary representations of a,bN) product := 0 fori := 0 to n−1 ifbi = 1 then product := add(an−1…a00i, product) returnproduct i extra 0-bitsappended afterthe digits of a

42. Binary Division with Remainder procedurediv-mod(a,d Z+) {Quotient & rem. of a/d.} n := max(length of a in bits, length of d in bits) fori := n−1downto 0 ifa ≥ d0i then {Can we subtract at this position?} qi:=1 {This bit of quotient is 1.} a := a − d0i {Subtract to get remainder.} else qi:= 0 {This bit of quotient is 0.} r := a returnq,r {q = quotient, r = remainder}