461191 Discrete Mathematics Lecture 4: Induction and Recursion

1. 461191 Discrete MathematicsLecture 4: Induction and Recursion San Ratanasanya CS, KMUTNB

2. Today’s Topics • Last week review • Administrivia • Proof Review • Mathematical Induction • Recursive Definitions • Recursive Algorithms • Program Correctness

3. Last week review Algorithms, The Integers, and Matrices

4. Algorithm is a finite set of precise instructionsfor performing a computation or for solving a problem.

5. Pseudocode Algorithm 1 Finding the maximum element in a Finite Sequence. procedure max (a1, a2,…, an : integers) max := a1 for i := 2 to n if max < ai then max := ai {max is the largest element}

6. Algorithm 2The Linear Search Algorithm procedure linear search (x : integer; a1, a2,…, an :distinct integers) i := 1 while (i ≤ n and x ≠ ai) i := i + 1 if i ≤n then location := i else location := 0 {location is the subscript of term that equals x, or is 0 if not found}

7. Algorithm 3Binary Search Algorithm procedure binary search (x : integer; a1, a2,…, an : increasing integers) i := 1 {i is left endpoint of search interval} j := n {j is right endpoint of search interval} while i < j begin m :=  (i + j)/2  if x > am then i := m + 1 else j := m end if x = ai then location := i else location := 0 {location is the subscript of term equal to x, or 0 if x is not found}

8. Pseudocodes Procedurebubblesort(a1,…,an: real numbers with n  2) for i := 1 to n-1 for j := 1 to n-1 if aj > aj+1then interchange aj and aj+1 {a1,…,an isin increasing order}

9. The growth of functions DEFINITION 1 : Let f and g be functions from the set of integers or the set of real numbers to the set of real numbers. We say that f(x) is O(g(x)) if there are constants C and k such that, |f(x)| ≤ C |g(x)| when x > k

10. Example 1 : show that f(x) = x2+2x+1 is O(x2) case 1 0 ≤ x2+2x+1 ≤ x2+2x2+x2 = 4x2, when x >1 So f(x) = O(x2) , when C = 4 and k = 1 Ans case 2 when x >2 , 2x < x2 0 ≤ x2+2x+1 ≤ x2+x2+x2 = 3x2,when x >2 so f(x) = O(x2) when C = 3 and k = 2 Ans In this example, g(x) = O(f(x)) since x2 ≤ x2+2x+1 so, we say that f(x) and g(x) are of the same order

11. THEOREM 1 : Let f(x) = anxn+an-1xn-1+…+a1x+a0 where a0,a1,…an are real numbers.Then f(x) is O(xn) Proof : |f(x)| = | anxn+an-1xn-1+…+a1x+a0 | ≤ |an|xn+|an-1|xn-1+…+|a1|x+|a0| = xn (|an|+|an-1|x-1+…+ |a1|x(n-1)+ |a0|x-n-1) ≤ xn (|an|+|an-1|+…+ |a1|+ |a0|) This shows that |f(x)| ≤ C xn when C = (|an|+|an-1|+…+ |a1|+ |a0| and x > 1.

12. Example 2 : 1+2+3+…+n = O(?) ≤ n +n +…+n = n × n = n2 = O(n2) , C = 1 , k = 1 Example 3 : f(n) = n! = O(?) = 1 * 2 * 3 * …* n ≤ n * n * n * …* n = nn ≤ C |g(n)| = O(nn), C = k = 1

13. Since… …n! ≤ nn log n! ≤ log nn = n log n log n! = O(n log n) ...n < 2n n = O(2n) , k = C = 1 ...log n < n log n = O(n) , k = C = 1 ...logbn = log n/log b < n/log b logbn = O(n) , k = 1 , C = 1/log b

14. THEOREM 2 : If f1(x) = O(g1(x)) and f2(x) = O(g2(x))then (f1+f2)(x) = O(max(g1(x), g2(x))) since |f1(x)| ≤ C1 |g1(x)| , when x > k1 and |f2(x)| ≤ C2 |g2(x)| , when x > k2 |(f1+f2)(x)| = |f1(x) + f2(x)| ≤ |f1(x)| + |f2(x)| |f1(x)| + |f2(x)| < C1|g1(x)| + C2|g2(x)| ≤ C1|g(x)| + C2|g(x)| = (C1+C2) |g(x)| = C |g(x)| when C = C1+C2 , and g(x) = max|(g1(x),g2(x)| , k = max(k1,k2)

15. THEOREM 3 : Let f1(x) = O(g1(x)) and f2(x) = O(g2(x)) (f1 f2)(x) = O( g1(x) g2(x) ) Example 4 : f(n) = 3n log (n!)+(n2+3) log n = O(?), where n = positive integer ≤ O(n) O(n log n) + O(2n2) O(log n) ≤ O(n2 log n) + O((2n2) log n) = O(3n2 log n) = O(n2 log n) Example 5 : f(x) = (x+1) log (x2+1) + 3x2 = O(?) ==================================================== Commonly Estimated Functions : 1, log n , n , n log n , n2 , 2n , n!

16. Complexity Analysis of Algorithms Computational Complexity: 1) Time Complexity: An analysis of the time required to solve a problem. Time complexity is described in terms of the number of operations required instead of actual computer time, such as comparison, addition, subtraction, multiplication, division, etc. There are 3 possible cases in time complexity best case, average case, and worst case

17. 2) Space Complexity: An analysis of the computer memory required to solve a problem. space complexity are tied in with the particular data structures used to implement the algorithm.

18. Example 1 Describe the time complexity of Algorithm 1 of Section 2.1 for finding the maximum element in a set. Solution: “The number of comparisons will be used as the measurement of the time complexity of the algorithm, since comparisons are the basic operation used.” • when i = n + 1, there are exactly 2(n - 1) + 1 = 2n - 1 comparisons

19. Example 2 Describe the time complexity of linear search algorithm.

20. n = 2k or 2k < n < 2k+1 2k elements in the first loop 2k-1 elements in the second loop : : 21 elements in the k-1th loop 20 elements in the kth loop At most 2k+2 comparisons = (2 log n ) + 2 = O(log n) k loops

21. Example 3 Describe the time complexity of the binary search algorithm. Solution: For simplicity, assume there are n = 2k elements in the list a1, a2,…,an, where k is a non-negative integer. Note that k = log n (If n, the number of elements in the list, is not a power of 2, the list can be considered as part of a larger list with 2k + 1 elements, where 2k < n <2k + 1. Here 2k + 1 is the smallest power of 2 larger than n).

22. Example 4 Average case performance of Linear Search Algorithm 2i + 1 comparisons i = 1 : 3 i = 2 : 5 : i = n : 2n + 1 Average = (3 + 5 + 7 +n…+ 2n + 1)/2 = (2(1+2+n…+ n) + n)/n Since 1 + 2 + 3 +…+ n = n(n + 1)/2 So, the average = 2[n(n + 1)/2] + n = n + 1 + 1 = n + 2 = O(n)

24. Polynomial worst-case complexity = tractable O(nb), b is an integer ≥ 1 But when b is large or C is very large  problem (takes too long time to solve) “Intractable” : problem that cannot be solved within Polynomial time.

25. Big O estimates of time complexity cannot directly tell the actual amount of computer time used. • Knowing the constants C and k makes | f(n)| ≤ C |(g(n))|when n > k • The time each type of operations used is not equivalent. • Type of computers can also be different in terms of specification, and generation.

26. Asymptotic Notations • Big O (Upper bound) • f(n) grow faster than g(n) • |f(n)|  C|g(n)| when n > k  f(n) = O(g(n)) • Big  (Lower bound) • f(n) grow slower than g(n) • |f(n)|  C|g(n)| when n > k  f(n) = (g(n)) • Big  (Tight bound) • f(n) and g(n) have the same rate of growth • C1|g(n)|  |f(n)|  C2|g(n)| when n > k  f(n) = (g(n)) • C and k called witness pair.

27. The Integers & Division Definition 1: If a and b are integers with a ≠ 0, we say that a divides b if there is an integer c such that b = ac. When a divides b we say that a is a factor of b and that b is multiple of a. The notation a | b denotes that a divides b. We write a | b when a does not divide b. THEOREM 1: Let a, b and c be integers. Then 1. if a | b and a | c, then a | (b + c); 2. if a | b, then a | bc for all integers c; 3. if a | b and b | c, then a | c.

28. Definition 2: A positive integer p greater than 1 is called prime if the only positive factors of p are 1 and p. A positive integer that is greater than 1 and is not prime is called composite. THEOREM 2: THE FUNDAMENTAL THEOREM OF ARITHMETIC Every positive integer greater than 1 can be written uniquely as a prime or as the product of two or more primes where the prime factors are written in order of non-decreasing size.

29. Example 1 The prime factorizations of 100, 641, 999, and 1024 are given by 100 = 2255 = 2252, 641 = 641, 999 = 33337 = 33  37, 1024 = 2222222222 = 210 THEOREM 3: If n is a composite integer, then n has a primedivisor less than or equal to

30. THE DIVISION ALGORITHM THEOREM 6: THE DIVISION ALGORITHM Let a be an integer and d a positive integer. Then there are unique integers q and r, with 0 ≤ r < d, such that a = dq + r

31. Definition 3: In the equality given in the division algorithm, d is called the divisor, a is called the dividend, q is called the quotient, and r is called the remainder. Definition 4: Let a and b be integers, not both zero. The largest integer d such that d|a and d|b is called the greatest common divisorofaand b. The greatest common divisor of a and b is denoted by gcd(a, b).

32. Definition 5: The integer a and b are relatively primeif their greatest common divisor is 1. Example: gcd (17, 22) = 1 17, 22 = relatively prime Definition 6: The integer a1, a2,…, an are pairwise relatively prime if gcd(ai, aj) = 1 whenever 1 ≤ i < j ≤ n.

33. Example: 10, 17, 21 are pairwise relatively prime? 10, 19, 24 are pairwise relatively prime?

34. Definition 7: The least common multiple of the positive integers a and b is the smallest positive that is divisible by both a and b. The least common multiple of a and b is denoted lcm(a, b). Example: What is the least common multiple of 233572 and 2433? Solution: We have lcm(233572, 2433) = 2max(3, 4)3max(5, 3)7max(2, 0) = 243572

35. THEOREM 7: Let a and b be positive integers. Then ab = gcd(a, b)  lcm(a, b)

36. MODULAR ARITHMATIC Let a be an integer and m be a positive integer. We denote “a mod m” the remainder when a is divided by m. Definition 8: If a and b are integers and m is a positive integer, then a is congruent to b modulo m, if m divides a - b. The notation a ≡ b (mod m)is used to indicate that a is congruent to b modulo m. If a and b arenot congruent modulo m, a ≡ b (mod m).

37. Theorem 8: Let a and b be integers, and let m be a positive integer. Then a ≡ b (mod m) if and only ifa mod m = b mod m. Example: Determine whether 17 is congruent to 5 modulo 6 and whether 24 and 14 are congruent modulo 6. Solution: Since 6 divides 17 - 5 = 12, we see that 17 ≡ 5 (mod 6). However, since 24 - 14 = 10 is not divisible by 6, we see that 24 ≡ 14 (mod 6).

38. Applications of Number Theory • Puzzle • Cryptography • Caesar’s Ciphering • RSA • Computer Graphics • Signal processing • Thermodynamics • Quantum Physics • etc. Chinese Remainder Theorem Modular Arithmetic

39. Matrices • Matrix equality • Equals in number of rows and columns, and elements • Matrix Arithmetic • Addition, multiplication • Identity matrix • AIn = ImA = A • Transpose and Inverse matrix • Boolean product

40. Administrivia • Homework due today • 1.6-1.8 • Homework 3 is out today • Study by yourself • Prolog and Python • Programming Assignments will be coming soon!!

41. Proof Review: Basic Proof Methods

42. Nature & Importance of Proofs • In mathematics, a proof is: • A sequence of statements that form an argument. • Must be correct (well-reasoned, logically valid) and complete (clear, detailed) that rigorously & undeniably establishes the truth of a mathematical statement. • Why must the argument be correct & complete? • Correctness prevents us from fooling ourselves. • Completeness allows anyone to verify the result.

43. Rules of Inference • Rules of inference are patterns of logically valid deductions from hypotheses to conclusions. • We will review “inference rules” (i.e., correct & fallacious), and “proof methods”.

44. Visualization of Proofs A proof A Particular Theory Rules Of Inference … The Axiomsof the Theory Various Theorems

45. Inference Rules - General Form • Inference Rule – • Pattern establishing that if we know that a set of hypotheses are all true, then a certain related conclusion statement is true. Hypothesis 1 Hypothesis 2 …  conclusion “” means “therefore”

46. Inference Rules & Implications Each logical inference rule corresponds to an implication that is a tautology. Hypothesis 1 Inference rule Hypothesis 2 …  conclusion Corresponding tautology: ((Hypoth. 1)  (Hypoth. 2)  …)  conclusion

47. Formal Proofs • A formal proof of a conclusion C, given premises p1, p2,…,pnconsists of a sequence ofsteps, each of which applies some inference rule to premises or to previously-proven statements (as hypotheses) to yield a new true statement (the conclusion). • A proof demonstrates that if the premises are true, then the conclusion is true (i.e., valid argument).

48. Common Fallacies • A fallacy is an inference rule or other proof method that is not logically valid. • May yield a false conclusion! • Fallacy of affirming the conclusion: • “pq is true, and q is true, so p must be true.” (No, because FT is true.) • Fallacy of denying the hypothesis: • “pq is true, and p is false, so q must be false.” (No, again because FT is true.)

49. Common Fallacies - Examples “If you do every problem in this book, then you will learn discrete mathematics. You learned discrete mathematics.” p: “You did every problem in this book” q: “You learned discrete mathematics” • Fallacy of affirming the conclusion: pq and q does not imply p • Fallacy of denying the hypothesis: pq and  p does not imply  q

50. Inference Rules for Quantifiers • xP(x)P(o) (substitute any object o) • P(g) (for g a general element of u.d.)xP(x) • xP(x)P(c) (substitute a newconstantc) • P(o) (substitute any extant object o) xP(x) Universal instantiation Universal generalization Existential instantiation Existential generalization