Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

Advanced Algorithms Analysis and Design By Dr. Nazir Ahmad Zafar Dr Nazir A. ZafarAdvanced Algorithms Analysis and Design

Lecture No. 39Number Theoretic Algorithms(Theorems and Algorithms) Dr. Nazir A. ZafarAdvanced Algorithms Analysis and Design

Today Covered • Some More Proofs • GCD as a Linear Combination • Finding GCD, a Recursive Theorem • Euclid’s Algorithm • Extended Euclid’s Algorithm • Time Complexity of Euclid’s Algorithm • Residues and Reduced set of Residues • Groups and Rings Dr Nazir A. ZafarAdvanced Algorithms Analysis and Design

Method of Proof by Contraposition Steps in proving by contraposition • Express the statement to be proved in the form:  x  D, P(x)  Q(x) • Rewrite the statement in the contrapositive form  x  D,  Q(x)   P(x) • Prove the contrapositive by direct proof • Suppose that x is an arbitrary but particular element of D such that Q(x) is false • Show that P(x) is false Dr. Nazir A. Zafar9

Examples: Proof by Contraposition Example 1 Prove that for all integers n, if n2 is even then n is also even Proof • Express the above statement in the form:  x  D, P(x)  Q(x) • Suppose that D = Z, Even(n, 2)  n2 is even Even(n)  n is even • We have to prove that  n  Z, Even(n, 2)  Even(n) Dr. Nazir A. Zafar9

Examples: Proof by Contraposition • Contraposition of the above statement  n  Z,  Even(n)   Even(n, 2) is even • Now we prove above contrapositive by direct proof • Suppose that n is an arbitrary element of Z such that,  Even(n) (n is not even) i.e., n is odd • n2 = n.n = odd. odd = odd • n2 is odd •  Even(n, 2) is even • Hence,  n  Z,  Even(n)   Even(n, 2) is even • Therefore,  n  Z, Even(n, 2)  Even(n) is even • Hence  n  Z, if n2 is even then n is even Dr. Nazir A. Zafar9

Examples: Proof by Contraposition Example 2 Prove that for all integers n, if n2 is divisible by 7 then n is divisible by 7. Proof • Express the above statement in the form:  x  D, P(x)  Q(x) • Suppose that D = Z, Div(n, 2, 7)  n2 is divisible by 7 Div(n, 7)  n is divisible by 7 • We have to prove that  n  Z, Div(n, 2, 7)  Div(n, 7) Dr. Nazir A. Zafar9

Examples: Proof by Contraposition • Contraposition of the above statement  n  Z,  Div(n, 7)   Div(n, 2, 7) • Now we prove above contrapositive by direct proof • Suppose that n is an arbitrary element of Z such that,  Div(n, 7) (n is not divisible by 7) • n does contain any factor of 7 • n2does contain any factor of 7 • Hence,  n  Z,  Div(n, 7)   Div(n, 2, 7) • Therefore,  n  Z, Div(n, 2, 7)  Div(n, 7) • Hence,  n  Z, if n2 is divisible by 7 then n is divisible by 7. Dr. Nazir A. Zafar9

Lemma 1 Statement : The square of an odd integer is of the form 8m + 1 for some integer m. Proof: • Suppose n is an arbitrary odd integer. • By quotient remainder theorem any integer has the form 4m, 4m + 1, 4m + 2 OR 4m+3 • Now since n is an odd integer, hence n can be represented as 4m + 1 OR 4m+3 • Now we have to prove that squares of 4m + 1 and 4m + 3 are of the form 8m + 1. Dr. Nazir A. Zafar9

Lemma 1 Case 1 Square of 4m + 1 (4m + 1)2 = 16m2 + 8m + 1 = 8(2m2 + m) + 1 = 8m’ + 1, where m ‘ = (2m2 + m) Case 2 Square of 4m + 3 (4m + 3)2 = 16m2 + 24m + 9 = 8(2m2 + 3m + 1) + 1 = 8m’’ + 1, where m’’ = (2m2 + 3m + 1) • Hence any odd integer has the form 8m + 1 for some m Dr. Nazir A. Zafar9

Theorem 1 Statement: • If a and b are any integers, not both zero, then gcd(a, b) is the smallest positive element of the set {ax + by : x, y Z} of linear combinations of a and b. Proof Let s be the smallest positive such linear combination of a and b, i.e. s = ax + by, for some x, y Z By quotient remainder theorem a = qs + r = qs + a mod s, where q = ⌊a/s⌋. a mod s = a – qs = a - q(ax + by) = a (1 - qx) + b(-qy) Dr. Nazir A. Zafar10

Theorem 1 • Hence a mod sis a linear combination of a and b. • But, since a mod s < s, therefore, a mod s = 0 • Now a mod s = 0 s | a • Similarly we can prove that, s | b. • Thus, s is a common divisor of both a and b, • Therefore, s  gcd(a, b) (1) • We know that if d | a and d | b then d | ax + by for all x, y integers. • Since gcd(a, b)| a and gcd(a, b) | b, hence gcd(a, b) | s and s > 0 imply that gcd(a, b) ≤ s. (2) • By (1) and (2), gcd(a, b) = s Dr. Nazir A. Zafar10

Corollary Statement; • For all integers a and b and any nonnegative integer n, gcd(an, bn) = n gcd(a, b). Proof • If n = 0, the corollary is trivial. • If n > 0, then gcd(an, bn) is the smallest positive element of the set {anx + bny}, i.e. gcd(an, bn) = min {anx + bny} = min{n.{ax + by}} = n. min{ax + by} n times smallest positive element of set {ax + by}. • Hence gcd(an, bn) = n.gcd(x, y) Dr. Nazir A. Zafar10

Relatively Prime Integers • Two integers a, b are said to be relatively prime if their only common divisor is 1, i. e, if gcd(a, b) = 1. Generalized Form of Relatively Prime Integers • We say that integers n1, n2, ..., nkare pairwise relatively prime if, whenever i ≠ j, we have gcd(ni, nj) = 1. Dr. Nazir A. Zafar10

Lemma 2 Statement • For any integers a, b, and p, if both gcd(a, p) = 1 and gcd(b, p) = 1, then gcd(ab, p) = 1. Proof • As, gcd(a, p) = 1, there exist integers x, y such that ax + py = 1 (1) • gcd(b, p) = 1, there exist integers x’, y’ such that bx’ + py’ = 1 (2) • Multiplying equations (1) and (2) and rearranging, ab(x x′) + p(ybx′ + y′ax + pyy′) = 1, abx’’ + py’’ = 1 • Since 1 is a positive linear combination of ab and p, • Hence gcd(ab , p) = 1,which completes the proof Dr. Nazir A. Zafar10

Lemma 3 • Now p ∤ a  gcd(a, p) = 1 • And, p ∤ b  gcd(b, p) = 1 • Since only divisors of p are 1 and p, and by assumption p divides neither a nor b. • Above Lemma 2, states that for any integers a, b, and p, if both gcd(a, p) = 1 and gcd(b, p) = 1, then gcd(ab, p) = 1. • Now, gcd(ab, p) = 1, contradicts our assumption that p | ab, since p | ab implies gcd(ab, p) = p. • This contradiction completes the proof. Dr. Nazir A. Zafar10

Theorem 2: GCD Recursion Theorem Statement • For any nonnegative integer a and any positive integer b, gcd(a, b) = gcd(b, a mod b). Proof • If we will be able to prove that gcd(a, b) and gcd(b, a mod b) divide each other, It will complete the proof of the theorem. This is because both are nonnegative. Case 1 • We first show that gcd(a, b) | gcd(b, a mod b). • If we let d = gcd(a, b). • By quotient remainder theorem: (a mod b) = a - qb, where q = ⌊a/b⌋. Dr. Nazir A. Zafar10

Theorem 2: GCD Recursion Theorem Case 2 • We now show that: gcd(a, a mod b) | gcd(a, b). • If we let, d = gcd(b, a mod b), then d | b and d | (a mod b). • By quotient remainder theorem a = qb + (a mod b), where q = ⌊a/b⌋, • a is a linear combination of b and a mod b,  d | a • Now, d | a and d | b  d | gcd(a, b) • Hence, gcd(a, a mod b) | gcd(a, b) (B) • By (A) and (B): gcd(a, b) = gcd(b, a mod b). Dr. Nazir A. Zafar10

Example: Compute gcd (1970, 1066) a = 1970, b = 1066 • 1970 = 1 x 1066 + 904 = gcd(1066, 904), R = 904 • 1066 = 1 x 904 + 162 = gcd(904, 162), R = 162 • 904 = 5 x 162 + 94 = gcd(162, 94), R = 94 • 162 = 1 x 94 + 68 = gcd(94, 68), R = 68 • 94 = 1 x 68 + 26 = gcd(68, 26), R = 26 • 68 = 2 x 26 + 16 = gcd(26, 16), R = 16 • 26 = 1 x 16 + 10 = gcd(16, 10), R = 10 • 16 = 1 x 10 + 6 = gcd(10, 6), R = 6 • 10 = 1 x 6 + 4 = gcd(6, 4), R = 4 • 6 = 1 x 4 + 2 = gcd(4, 2), R = 2 • 4 = 2 x 2 + 0 = gcd(2, 0), R = 0 Hence gcd(1970, 1066) = 2 Dr. Nazir A. Zafar14

Euclid’sAlgorithm EUCLID(a, b) 1 if b = 0 2 then return a 3 else return EUCLID(b, a mod b) Example • Compute the gcd of 30 and 21 Solution • EUCLID(30, 21) = EUCLID(21, 9) = EUCLID(9, 3) = EUCLID(3, 0) = 3 • Here, there are three recursive invocations of EUCLID. • The correctness of EUCLID follows from Theorem 2 • And the fact that if the algorithm returns a in line 2, then b = 0, and so gcd(a, b) = gcd(a, 0) = a Dr. Nazir A. Zafar10

Euclid’s Algorithm Note: • The algorithm cannot recurse indefinitely • This is because the second argument strictly decreases in each recursive call • And this second argument is also always nonnegative. • Hence it must be 0 after some number of calls • Therefore, EUCLID always terminates with the correct answer. Dr. Nazir A. Zafar10

Running Time of Euclid’s Algorithm • We analyze the worst-case running time of EUCLID as a function of the size of a and b. • We assume without loss of generality that a > b ≥ 0. • This assumption justified because if b > a ≥ 0, then EUCLID(a, b) makes recursive call EUCLID(b, a). • That is, if first argument is less than second one, EUCLID spends one recursive call swapping a, b • Similarly, if b = a > 0, the procedure terminates after one recursive call, since a mod b = 0. • The overall running time of EUCLID is proportional to the number of recursive calls it makes. • Our analysis makes use of the Fibonacci numbers Fk, defined earlier in the first part of our course Dr. Nazir A. Zafar10

Running Time of Euclid’s Algorithm Statement • If a > b ≥ 1 and the invocation EUCLID(a, b) takes k ≥ 1 recursive calls, then a ≥ Fk+2 and b ≥ Fk+1. Proof The proof is by induction on k. Case 1 • For base case, let k = 1. Then, b ≥ 1 = F2, and since a > b, we must have a ≥ 2 = F3. • Hence the statement is true for k = 1 • Please note that, b > (a mod b), in each recursive call, i.e., first argument is strictly larger than the second and hence the assumption that a > b therefore holds for each recursive call. Dr. Nazir A. Zafar10

Running Time of Euclid’s Algorithm Case 2 • Now suppose that the lemma is true for k – 1 i.e., if a > b ≥ 1 and invocation EUCLID(a, b) takes k-1 ≥ 1 recursive calls, then a ≥ Fk+1 and b ≥ Fk. Case 3 • Now we have to prove that statement is true for k i.e. if a > b ≥ 1 and invocation EUCLID(a, b) takes k ≥ 1 recursive calls, then a ≥ Fk+2 and b ≥ Fk+1. • Since k > 0, and b > 0, and EUCLID(a, b) calls EUCLID(b, a mod b) recursively, which in turn makes k - 1 recursive calls. • Since we know that statement is true for k-1, hence b ≥ Fk+1, and (a mod b) ≥ Fk. Dr. Nazir A. Zafar10

Running Time of Euclid’s Algorithm • Now we have b + (a mod b) = b + (a - ⌊a/b⌋ b) (1) • Since, a > b > 0, therefore, ⌊a/b⌋ ≥ 1  ⌊a/b⌋ b ≥ b  b - ⌊a/b⌋ b  0  a + b - ⌊a/b⌋ b  0 + a  b + (a - ⌊a/b⌋ b)  a • By (1), b + (a mod b) = b + (a - ⌊a/b⌋ b) ≤ a • b + (a mod b) ≤ a • Thus, a ≥ b + (a mod b) ≥ Fk+1 + Fk = Fk+2 . • Hence, a ≥ Fk+2, It completes proof of the theorem Dr. Nazir A. Zafar10

Extended Euclid’s Algorithm EXTENDED-EUCLID(a, b) 1 if b = 0 • then return (a, 1, 0) • (d’, x’, y’)  EXTENDED-EUCLID(b, a mod b) • (d, x, y)  (d’, y’, x’ -  a/b y’) • return (d, x, y) Proof of Correctness d’ = bx’+ (a mod b)y’ d = bx’+ (a -  a/bb)y’ gcd(a, b) = gcd(b, a mod b) d = ay’ + b(x’ -  a/by’) Dr. Nazir A. Zafar10

Reduced set of residues mod n • Complete set of residues is: 0 . . . n-1 • Reduced set of residues consists of all those numbers (residues) which are relatively prime to n • And it is denoted by Zn* = {k : gcd(k, n) = 1, 0  k < n} • The number of elements in reduced set of residues is called the Euler Totient Function (n) Example • For n = 10, find reduced list of residues of n • All residues: {0, 1, 2, 3, 4, 5, 6, 7, 8, 9} • Reduced residues (primes) = {1, 3, 7, 9}, (n) = 4 Dr. Nazir A. Zafar10

Group Definition of a Group Group is a set, G, together with a binary operation : G * G  G, usually denoted by a*b, such that the following properties are satisfied : • Associativity : (a*b)*c = a*(b*c) for all a, b, c  G • Identity :  e  G, such that e*g = g = g*e for all g  G. • Inverse : For each g  G, there exists the g’, inverse of g, such that g’*g = g*g’ = e Dr. Nazir A. Zafar11

The Multiplicative Group Z*n Zn* = {k : gcd(k, n) = 1, 1  k < n} For any positive integer n, Zn* forms a group under multiplication modulo n. Proof: • Binary Operation Let a, b Zn*, gcd(a, n) = 1; gcd(b, n) = 1 gcd(ab, n) = gcd(a, n)*gcd(b,n) = 1*1 = 1 • Associativity holds, • 1 is the identity element. • inverse of each element exits Hence (Zn* ,*) forms a group. Dr. Nazir A. Zafar11

Rings Definition A ring is a set R with two binary operations + : R × R → R and · : R × R → R (where × denotes the Cartesian product), called addition and multiplication, such that: • (R, +) is an abelian group with identity element 0 • (a + b) + c = a + (b + c) • 0 + a = a + 0 = a • For every a in R, there exists an element denoted −a, such that a + −a = −a + a = 0 • a + b = b + a Dr. Nazir A. Zafar22

Rings Definition (Contd..) • (R, ·) is a monoid with identity element 1: • (a·b)·c = a·(b·c) • 1·a = a·1 = a • Multiplication distributes over addition: • a·(b + c) = (a·b) + (a·c) • (a + b)·c = (a·c) + (b·c) Note • Ring addition is commutative so that a + b = b + a • But ring with multiplication is not required to be commutative i.e. a·b need not equal b·a. • Rings that satisfy commutative property for multiplication are called commutativerings. • Not all rings are commutative. Dr. Nazir A. Zafar22

Rings • Rings need not have multiplicative inverses either. • An element a in a ring is called a unit if it is invertible with respect to multiplication • An element a is called invertible under multiplication if there is an element b in the ring such that a·b = b·a = 1, • This b is uniquely determined by a and we write a−1 = b. Lemma • The set of all units in R forms a group under ring multiplication Dr. Nazir A. Zafar22

Example: Rings Example Prove that Z (+, *) ( the set of integers) is a ring. Solution + and * are binary operation on Z because sum and product of two integers are also an integer • Now,  a, b, c  Z • (a + b) + c = a + (b + c), • 0 + a = a + 0 = a • a + (−a) = (−a) + a = 0 • a + b = b + a Hence (Z, +) is an abelian group with identity element 0 Dr. Nazir A. Zafar22

Example: Rings • Since,  a, b, c  Z • (a·b)·c = a·(b·c) • 1·a = a·1 = a Hence (Z, ·) is a monoid with identity element 1 • Finally  a, b, c  Z • a·(b + c) = (a·b) + (a·c) • (a + b)·c = (a·c) + (b·c) i.e., multiplication is distributive over addition Hence we can conclude that Z (+, *) is a ring Dr. Nazir A. Zafar22

Dr Nazir A. Zafar Advanced Algorithms Analysis and Design