Chapter 2. Basic structures: Sets, Functions, Sequences and Sums

Chapter 2. Basic structures: Sets, Functions, Sequences and Sums Cheng-Chia Chen

2.1 Sets • Basic structure upon which all other (discrete and continuous ) structures are built. • a set is a collection of objects. • an object is anything of interest, maybe itself a set. • Definition 1. • A set is a collection of objects. • The objects in a set are called the elements or members of the set. • If x is a member of a set S, we say S contains x. • notation: x ÎS vs x Ï S • Ex: In 1,2,3,4,5, the collection of 1,3 5 is a set.

Set description • How to describe a set:? 1. List all its member. • the set of all positive odd integer < 10 = ? • The set all decimal digits = ? • the set of all upper case English letters = ? • The set of all nonnegative integers = ? 2. Set builder notation: • P(x) : a property (or a statement or a proposition) about objects. e.g., P(x) = “ x > 0 and x is odd” • then {x | P(x) } is the set of objects satisfying property P. • P(3) is true => 3 Î {x | P(x)} • P(2) is false => 2 Ï {x | P(x)}

Conventions • N =def {x | x is a natural numbers } = { 0,1,2,3,...} • N+ =def {1,2,3,...} • Z =def {...,-3,-2,-1,0,1,2,3,...} • R =def the set of real numbers. • Problem: The same set may have many different descriptions. • {x | 0 < x <10 /\ x is odd } • {1,3,5,7,9}, {5,3,1,9,7} • {9,7,1,3,5}.

Set predicates Definition 2. • Two sets S1, S2 are equal iff they have the same elements • S1 = S2 iff "x (x Î S1 <-> x Î S2) • Ex: {1,3,5} = {1,5,3} = {1,1,3,3, 5} • Graphical representation of sets: • Venn Diagrams: • rectangle: Universal set: U • circles: sets • points: particular elements • point x inside circle S => x Î S • point x outside S => x Ï S.

Set predicates (cont’d) • Null set ={} = Æ =def the collection of no objects. Def 3’: [empty set] for-all x x Ï Æ. Def 3. [subset] • A Í B iff all elements of A are elements of B. • A Í B  for-all x (x Î A ⇒ x Î B)). Def 3’’: A Ì B =def A Í B /\ A ¹ B. • Exercise : Show that: • 1. For all set A (Æ Í A) • 2. (A Í B /\ B Í A)  (A = B) • 3. A Í Æ ⇒A = Æ • Diagram representation of the set inclusion relationship.

Size or cardinality of a set Def. 4: | A | = the size(cardinality) of A = # of distinct elements of A. • Ex: • |{1,3,3,5}| = ? |{}| = ? • | the set of binary digits } | = ? • |N| = ? ; |Z| = ? ; | {2i | i ∈ N} = ? • |R| = ? Def. 5. • A set A is finite iff |A| is a natural number ; o/w it is infinite. • Two sets are of the same size (cardinality) iff there is a 1-1 & onto mapping between them.

countability of sets Def. • A set A is said to be denumerable iff |A| = |N|. • A set is countable iff either |A| = n for some n in N or |A| = |N|. • Exercise: Show that • 1. |N| = |Z | = | Q | = |{4,5,6,...}| • 2. |R| = | (-1, 1) | = |(0,1)| • 3. |(0,1)| is uncountable By exercise 1,2,3, • R is not countable. • Q and Z is countable.

The power set Def 6. • If A is a set, then the collection of all subsets of A is also a set, called the power set of A and is denoted as P(A) or 2A. • Ex: • P({0,1,2}) = ? • P({}) = ? • |P({1,2,..., n})| = ? • Elements in a set are not ordered. But sometimes we need to distinguish between (1,3,4) and (3,4,1) --> ordered n-tuples

Cartesian Products Def. 7 [n-tuple] • If a1,a2,...,an (n > 0) are n objects, then “(a1,a2,...,an)” is a new object, called an (ordered) n-tuple [ with ai as its ith element. ] • Any ordered 2-tuple is called a pair. • (a1,a2,...,am) = (b1,b2,...,bn) means • (1) m = n and • (2) ai = bi for all 1 ≤ i ≤ m.

Cartesian product Def. 8: [Cartesian product] A x B =def {(a,b) | a ∈ A and b ∈ B } A1 x A2 x ...x An =def {(a1,...,an) | ai ∈ Ai for all 1 ≤ i ≤ n}. Ex: A = {1,2}, B = {a,b,c} , C = {0,1} 1. A x B = ? ; 2. B x A = ? 3. A x {} = ? ; 4. A x B x C = ? Def. 8.1: Any subset of AB is called a relation from A to B. problem: 1. when will A x B = B x A ? 2. |A x B | = ?

The diagonalization principle (D.P.) • R: a binary relation on A (i.e. a subset of AxA) . D (the diagonal set for R ) =def { x | x Î A and (x,x) Ï R}. For each x Î A, let Ra (the raw of a) = { b Î A | (a,b) Î R}. Then D ≠ Ra for all a Î A. Example: Let A = {a,b,c,d,e,f} and R = Ra ={b,d} Rb={b,c} Rc={c} Rd={b,c,e,f} Re={e,f} Rf={a,c,d,e} D = {a,d,f}. D¹Rx since for each xÎA, xÎD iff (x,x)ÏR iff xÏRx.

Generalization of the diagonalization principle A, B : two sets with a mapping f: A B. R :a relation from B to A (i.e., a subset of BxA). For all x ∈ A, let • Rf(x) ≡ { y | y ∈ A ∧ (f(x),y) ∈ R } ⊆ A, and • D ≡ { x | x ∈ A ∧ (f(x),x) ∉ R } ⊆ A. Then • D ≠ Rf(x) for all x ∈ A. Pf: Analogous to the previous one. Notes: 1. C = {Rf(x) | x ∈ A} = { Ry | y ∈ B} ⊆ 2A is a set of subsets of A. 2. C ≠ 2A (∵ D ∈ 2A but D ∉ C ).

Generalization of the diagonalization principle f:A B ; R: BxA Rfa ={b,d} Rfb={b,c} Rfc={c} Rfd={b,c,e,f} Rfe={e,f} Rff={a,c,d,e} D = {a,d,f}. D¹Rfx since for each f(x)ÎB, xÎD iff (f(x),x)ÏR iff xÏRf(x).

Show that |A| ≠ |2A| Pf: (1) The case that A is finite is trivial since |2A| = 2|A| > |A| and there is no bijection b/t two finite sets with different sizes. (2) Assume |A| = |2A|, i.e., there is a bijection f: A -> 2A. Define the so-called diagonal setD in terms of f as follows: Let D = {x ∈ A | x Ï f(x) }. Now (*) D is a subset of A and (**) D f(x) for any x ∈ A: since x ∈ D iff x Ï f(x) for each x ∈ A. ==> D is not a subset of A (∵ f is onto to 2A), a contradiction to (*). Hence f must not exist!!

Application of the diagonalization principle (skipped) • Theorem: The set 2N is uncountable. • pf: 1. direct from |A| ¹ |2A| with A = N. • 2. another proof: suppose 2N is denumerable. Then • there is a bijection f: N -> 2N. • let 2N = {S0,S1,S2,...} where Si = f(i). • Now the diagonal set • = { k | k Ï Sk }. By diagonalization principle, D¹ Sk for any k, but D is a subset of N, by assumption • D must equal to Sk for • some k. a contradiction!

Application of the diagonalization principle Theorem: The real set (0,1) is uncountable. pf: If n is a real in (0,1), then it can be represented as an infinite sequence n = 0.n1n2n3… where each nk is one of [0,…,9]. Ex: ½ = 0.500000… 1/3 = 0.3333… p-3 = 0.14159…. Now suppose the set (0,1) is denumerable. Then there must exist a bijection f: N  (0,1). We now apply diagonization principle to define a number d from f as follows: let d = 0.d0d1d2…. where dk is the 9’s complement of the kth digit of the fraction part of f(k).

Now it can be shown that d ¹ f(n) for all n since dn = 9 – f(n)n¹ f(n)n, where f(n)k is the kth digit of the fraction part of f(n). This contradicts the facts that d ∈ (0,1) and f(N) = (0,1).

Program you are asked to design P HALT(P,X) Pr(x:String) { ….. } P(X) Halt ? true yes false no “It’s a test …” X The Halting Problem (Section 3.1, 6th edition; p176) • L : any of your favorite programming languages (C, C++, Java, BASIC, etc. ) • Problem: write an L-program HALT(P,X), which takes another L-program P(-) and string X as input, and HALT(P,X) returns true if P(X) halts and returns false if P(X) does not halt.

How hard is the halting problem ? • Consider the program int f( int n ) { if ( n == 1 ) return 1 ; if (n % 2 == 0 ) { // n is even f(n % 2) ; } else { // n is odd f( n x 3 + 1) ; }} • It is conjectured that f(n) halts (returns 1) for all n ≥ 1. Ex: f(10)  5  16  8  4  2  1 (halts)

Halt(P,X) does not exist • Ideas leading to the proof: Problem1 : What about the truth value of the sentence: L: L is false Problem 2 : Let S = {X | X X}. Then does S belong to S or not ? The analysis: S  S => S  S; S  S => S  S. Problem 3 : 矛盾說:1. 我的矛無盾不穿　2. 我的盾可抵擋所有茅　　　結論: 1. 2. 不可同時為真。 Problem 4 : 萬能上帝:　萬能上帝無所不能 => 可創造一個不服從他的子民 => 萬能上帝無法使所有子民服從=> 萬能上帝不是萬能 . 　　結論:萬能上帝不存在。 Conclusion: • 1. S is not a set!! • 2. If a language is too powerful, it may produce expressions that is meaningless or can not be realized. • Question: If HALT(P,X) can be programmed, will it incur any absurd result like the case of S? Ans: yes!!

H(P) HALT(P,X) Loop P P(P) Halt ? true yes Pr(x:String) { ….. } P halt false no P H(P) : a program that HALT(-,-) cannot predicate Notes: 1. H(P) is simply { L : if(HALT(P,P)) then goto L } 2. H uses HALT(-,-) as a subroutine. 3. H(P) halts iff HALT(P,P) returns false iff P(P) does not halt. 4. Let input P be H  H(H) halts iff H(H) does not halt.  HALT is not a correct implementation!

1.5 Set operations • union, intersection,difference , complement, • Definition. 1. AÈ B = {x | x ∈ A or x ∈ B } 2. AÇ B = {x | x ∈ A and x ∈ B } 3. If A Ç B = {} => call A and B disjoint. 4. A - B = {x | x ∈ A but x ∉ B } 5. ~ A = U - A • Venn diagram representations • Ex: U = {1,..,10}, A = { 1,2,3,5,8} B = {2,4,6,8,10} => AÈ B , AÇ B , A - B , ~ A = ?

Set identities (page 124, 6th edition) • Identity laws: A È {} = A ; A ⋂ U = A • Domination law: U È A = A ; {}⋂ A = {} • Idempotent law: A ⋂A = A ; AÈA = A • complementation: ~~A = A • commutative : A⋂B = B ⋂A ; AÈB=BÈA • Associative: A⋂(B⋂C) = (A⋂B)⋂C; ___?_____ • Distributive: A È (B ⋂ C) = ? ; ____?_____ • De Morgan laws: ~(AUB) = _?_ ; ~(A ⋂ B)= ~A U ~B • Absorption law : AÈ(A ⋂ B) = A; A ⋂ (AÈB) = A • Complement law: AÈ~A = U; A⋂~A = {}

Prove set equality 1. Show that ~(A È B) = ~A Ç ~B by show that • 1. ~(A È B) Í ~A Ç~B • 2. ~A Ç~B Í ~(A È B) pf: (1. By definition and logic reasonng)Let x be any element in ~(A È B) . Then x  ~(AUB) iff ~(x  AUB) iff x  A and x  B iff x ~A and x  ~B. 2. show (1) by using set builder and logical equivalence. ~(AUB) = { x | x  AUB } = {x | x ~A and x  ~B } = {x | x ~A} Ç {x | x ~A } = ~A Ç ~B.

Membership Table 3. Show distributive law : A U (B ⋂ C) = (AUB) ⋂ (AUC) by using membership table. Let x be any element. Then we need only consider 8 cases as to whether x is a member of A or B or C. In all cases, we find that x A U (B ⋂ C) iff x(AUB)⋂(AUC) . Hence the equality holds.

Set equality reasoning 4 show ~(AÈ (BÇ C)) = (~C È ~B) Ç ~A by set identities. pf: ~(AÈ (BÇ C)) = ~A Ç ~(B Ç C ) = ~(B Ç C ) Ç ~A = (~B È~C) Ç ~A

Generalized set operations Def. 6 • A1,A2,...An: n sets • B = {A1,A2,...An } 1. A1È A2È An = ÈB = È{i=1,..n} Ai =def ? 2. A1Ç A2Ç An = ÇB = Ç {i=1,..n} Ai=def ? quiz: if B = {} => ÈB = ?; ÇB = ? • Venn diagram representation of A1È A2È A3 and A1Ç A2Ç A3 . Example: A = {0,2,4,6,8}, B = {0,1,2,3,4}, C={0,3,6,9} => • A1È A2È A3 = ? • A1Ç A2Ç A3 = ?

Set representation (in computer) • Unordered list or array • union, intersection, ~: time consuming. • Bit string: • U = {a1,...,an} is the universal set. • A: any subset of U • A can be represented by the string: s(A) = x1 x2 x3....xn where xi = 1 if a1 ∈ A and 0 o/w. • fast set operations • suitable only if U is not large. • constant size representation.

1.6 Functions • Def. 1 [functions] A, B: two sets 1. a partial function f from A to B is a set of pairs (x, y) ∈ AxB s.t., for each x ∈ A there is at most one y ∈ B s.t. (x,y) ∈ f. 2. f is a (total) function if for for each x ∈ A there is exactly one y ∈ B with (x,y) ∈ f. 3. if (x,y) ∈ f, we write f(x) = y. 4. f :A B means f is a function from A to B. Def. 2. If f:A  B then 1. A: the domain of f; 2. B: the codomain of f if f(a)=b => 3. b is the image of a 4. a is the preimage of b 5. range(f) = ? 6. preimage(f) = ?

Types of functions • Def 4. f A x B ; S: a subset of A, T: a subset of B 1. f(S) =def ? { y B | x  S with (x,y)  f } 2. f-1(T) =def ? { x A | y  T with (x,y)  f } Def. [1-1, onto, injection, surjection, bijection] f: A -> B. • f is 1-1 (an injection) iff ? • f is onto (surjective, a surjection) iff ? • f is 1-1 & onto <=> f is bijective (a bijection, 1-1 correspondence)

Properties of functions • If there is an onto mapping from A to B, then there is a 1-1 mapping from B to A. • If there is a 1-1 mapping from A to B then there is an onto mapping from B to A. • The onto (≥ ) and the 1-1(≤ ) relations between sets are all preorders (i.e., refexive and transitive ) and are converse to each other. • If there is a onto mapping from A to B and a onto mapping from B to A, then there is a 1-1 and onto mapping from A to B. pf: (1) Let f:AB an onto. For each b ∈ B, let g(b) = {x ∈ A | f(x) = b} ≠ ∅ ⇒ the function h:BA defined by h(b) = any x ∈ g(b) is 1-1. (2,3) Similar to (1). (4) is hard (as an exercise?).

Real valued functions • F:A B is a real valued function iff A and B are subsets of R. • f1,f2: real valued functions => 1. f1+f2 (x) = ? 2. f1• f2 (x) = f1(x) x f2(x). 3. f is increasing iff ? 4. f is strictly increasing iff ?

operations on functions • A, B, C: any sets ; f: A B; g: B  C, then 1. [identity function] idA : A A s.t. idA(x) = x for all x in A. 2. [composition of g and f] gf: A  C s.t. gf(x) = g(f(x)) for x in A. 3. [inverse] If f is a bijection, then f-1: B  A s.t. f-1(y) = x iff f(x) = y for y in B. • Graphical representations

Properties of operations on functions • f: A  B. g: B C; h: C D, then 1. idA f = f = f idB 2. f(gh) = (fg) h --- assoc. 3. (fg)-1 = g-1 f-1 If f: A  A is a bijection then 4. f f-1 = f-1 f= idA . • Sequence: finite A-sequence: a: [n] (or [1,n]) A w A-sequence: a : N (or N+)  A. B indexed A-sequence : f:BA.

The growth of functions ( Section 3.2, 6th edition) • Summation rules • S ai =def a1 + a2+...an+ (...) • a + (a+d) + (a+2d)+... = ? sol: let S = a + (a+d) + (a+2d)+... (a + (n-1)d). Then S = (a + (n-1)d) + (a+(n-2)d) +… (a+d)+a 2S = (2a + (n-1)d) + …. (2a+ nd) = (2a+ nd) x n. => S = [a + (n-1)d]n/2 = (a1 + an)n/2. • a + ar +arr + ... = sol: let S = a + ar + ar2+... +a r (n-1). Then rS = ar + ar2 + ar3 +… arn (1-r)S = a - arn. => S = a(1-rn)/(1-r).

The growth of functions • If f is a positive valued function, then what do the following terms mean? • O(f) means ? • W(f) means ? • Q(f) means ? • useful in the analysis of the efficiency of computer algorithms.

Problem, algorithm and Complexity • A problem is a general question: • description of parameters [input] • description of solution[output] • description of how the outputs are related to the inputs. • An algorithm is a step by step procedure to get the related output (i.e., answer) from the given input. • a recipe • a computer program • We want the most efficient algorithm • fastest (mostly) • most economical with memory (sometimes) • expressed as a function of problem input size

a 10 5 9 3 c 6 9 b d Example: Traveling Salesman Problem • Parameters: • Set of cities • Inter-city distances

a 10 5 9 3 c 6 9 b d Solution [output] • Solution: • The shortest tour passing all cities • Example: a,b,d,c,a has length 27

Problem Size • What is appropriate measure of problem size? • m nodes? • m(m+1)/2 distances? • Use an encoding of the problem • alphabet of symbols: a,b,c,d,0-9, |. • strings: abcd||10|5|9|6|9|3. • Measures • Problem Size: length of encoding. • Time Complexity: how long an algorithm takes, as function of problem size.

Time Complexity • What is tractable? • A function f(n) is O(g(n)) whenever • ∃ c > 0 ∃ n0 > 0 s.t. |f(n)| ≤ c ∙ |g(n)| for all n > n0. • A polynomial time algorithm is one whose time complexity is O(p(n)) for some polynomial p(n). • An exponential time algorithm is one whose time complexity cannot be bounded by a polynomial (e.g., nlog n or 2n).

Tractability • Basic distinction: • polynomial time = tractable • exponential time = intractable problem size execution time in microseconds(μs)

Effect of Speed-ups for different order of functions Wait for faster hardware! Consider maximum problem size you can solve in an hour (or in (N = 3600 x 106 )x speedup steps). f(Nk) = N  f(?) = N x speeup

Asymptotic notations • Asymptotic analysis: • care the value of f(n) only when n is very large. • discard the difference of f and g when limit f/g is bounded between two positive numbers [a,b]. • O((g(n))= { f(n) |  c and k > 0 s.t. f(n)  cg(n) for all n > k } • Q((g(n))= { f(n) | c2,c1 and k > 0 s.t. c2g(n)  f(n)  c1 g(n) for all n > k } • W((g(n)) = { f(n) | c and k > 0 s.t. c g(n)  f(n) for all n > k }

Examples Ex1: Show that f(x) = x2 + 2x+ 1 is O(x2). pf: Let c = 4, k = 1, It is easy to check that if x > k, then f(x) = x2 + 2x+ 1 ≤ x2 + 2 x2 + x2 ≤ c x2. Hence by definition, f(x) = O(x2). Ex2: Show that f(x) = x2 + 2x+ 1 is W(x2). pf: Let c = 1, k = 1. Obviously, x2+2x+1 ≥ x2 = cx2 for all x >1=k. Hence f(x) = W(x2).

Ex2: Show that f(x) = x2 + 2x+ 1 is Q(x2). pf: Let c1 = 1,c2=4, k = 1. c1x2 ≤ x2+2x+1 ≤ c2 x2 for all x >1=k. Hence f(x) = Q(x2). Proposition: f = Q(g) iff f = O(g) and f = W(g). pf: (=>:) direct from definition. (<=:) if f = O(g) = W(g) => ∃ c1, k1 s.t f(x) ≤ c1 g(x) for all x > k1. ∃ c2, k2 s.t f(x) ≥ c2 g(x) for all x > k2. let k = max(k1,k2) => c2 g(x) ≤ f(x) ≤ c1 g(x) for all x > k. Hence f(x) = Q(g).

Using the asymptotic notations • We use f(n) = D(g(n)) to mean f(n) D(g(n)), where D = Q,O,W. • analogy: • O (asymptotic upper bound):  f(n) = O(g(n)) behaves like f(n)  g(n) • W (asymptotic lower bound):  f(n) = W(g(n)) behaves like f(n)  g(n) • Q (asymptotic tight bound : = f(n) = Q (g(n)) behaves like f(n) = g(n)

Exercises • Let f and g be positive real function. Show that • f = O(g) iff g = W(f) 2. f= Q(g) iff limit n g/f is bounded above 0

Chapter 2. Basic structures: Sets, Functions, Sequences and Sums