Sets and Counting

1. Sets and Counting JA Ritcey EE 416 Revised Oct 5 2009 Please elaborate with your own illustrations!

2. Elementary Set Theory • Universe – superset containing any element of interest. This is the complement of the empty set. • Subset – collection of none, some, or all elements • Set operations – complement, union, intersection • Cardinality – the number of elements in a set, size • Cardinality can be finite, countable, or uncountable • Venn diagrams are useful pictorial representations • Set operations can be carried out in MATLAB!

3. Set Operations • U = { u_1, … , u_N } with card(U) = N • Count by placing elements of U in one-to-one correspondence with the natural numbers • You should be able to carry out set operations by taking unions, intersections, and complements. • You should be able to handle both lists and geometric regions.

4. Disjoint or Mutually Exclusive Sets • 2 sets a disjoint or mutually exclusive if they have no common elements • disjoint means the intersection is the empty set • Disjoint sets are simpler than those with overlap • A collection of sets is mutually disjoint if no two intersect. Together they partition the union. • Partition – a collection of disjoint subsets that is complete and together yields the entirety

5. Partitions • A set S is partitioned by a collection of disjoint subsets that together encompass all of S • The days of the week are partitioned into work days and weekend • The numbers {1,2,3, …} are partitioned into odd and even • Partitions are used to when something is held invariant over each subset

6. Sensor Nodes Partitioned by Radius from Acces Point

7. Cartesian Products • Given 2 sets A = { a_i } B = { b_j } create the • Ordered pair c = (a_i, b_j ) • C = AxB is called the Cartesian product • |C| = |AxB| = |A|x|B| . This is called the multplication principle in counting (combinatorics) • Select one from column A and 1 from column B • This can be extended to ordered J-tuples • C= A_1 x … x A_J the J-fold product • A vector can be created this way

8. Power Sets • Power Set Pwr(S) is the set of all subsets of any S • If a set S is finite, say |S| = N, then • |Pwr(S)| = 2^N • To see this equate each member of Pwr(S) with the binary string (b_1, b_2, … , b_N ). • The b_i in (0,1) label whether element s_i is in the particular subset. • There are 2^N binary strings of length N • If the cardinality, or size, of A is countable, the power set has uncountable cardinality

9. Properties of the Power Set • Pwr(S) has some closure properties • A collection of sets C is closed under set operations: • (1) If A in C, A^c in C (closed under complements) • (2) If A_1, …A_N in C, their union is in C • Often this last property is extended to countable unions. If A_1, A_2,… in C, their union is in C • Collections of sets with this property are called algebras and sigma-algebras. • When S is uncountable Pwr(S) is too big to assign probabilities to everything – a subtle point

10. Generating some small power sets • function powerset(n); • %function powerset(n); • n1=n-1; • for knt = 0:2^n1-1 • C= bitget(knt,n1:-1:1); • disp([knt C]) • end 0 0 0 0 0 1 0 0 0 1 2 0 0 1 0 3 0 0 1 1 4 0 1 0 0 5 0 1 0 1 6 0 1 1 0 7 0 1 1 1 8 1 0 0 0 9 1 0 0 1 10 1 0 1 0 11 1 0 1 1 12 1 1 0 0 13 1 1 0 1 14 1 1 1 0 15 1 1 1 1

11. Counting Finite Sets • We have already done some counting, using the Multiplications Principle • Cartesian product card |C| = |AxB| = |A| |B| • All pairs must be possible • The size of A and B can be different, but finite • There are 52 = 4x13 cards in a deck • There are 100 = 2x50 US Senators • Lets call a set S an N-set when |S|=N

12. Ordered Selection w/ Repetition • Given a base N-set S • In how many ways can we select k objects • Assume an ordered selected With replacement. So that repetition is possible • We can do this is N^k ways by the multiplication principle • Just equate the selection of an object with an ordered k-tuple The elements are independently selected and each drawn from an N-set (repetitions)

13. Tabular View • Repetition • WithWithout • W/ OrderN^k N!/(N-k)! • WO/ Order (N+k-1)!/ (N-1)! k! N!/(N-k)!k! • (N+k-1 choose k) (N choose k) • It is easier to remember these, when they are considered as a group

14. Ordered Selection W/Replacement • Each Selection is a row • Possible Choices along columns • 7^12 = 7 x7 …x7 • Cartesian Product • Illustrates the Multiplication Principle

15. Examples • Cards = 4 suits x 13 face values = 52 cards • You can place r balls into n cells in n^r ways • n^r = n x n … x n • Harder: How many ways can r flags be put on n flagpoles? • Ans: n x (n+1) x (n+2) x … x (n+r-1) • Why: The second flag can be put above or below the first. So the number of choices increases

16. Ordered Selection wo/ Repetition • Given a base N-set S • In how many ways can we select k objects • Assume an ordered selected without replacement. So that repetition is not possible • We can do this is (N)_k = N(N-1) …(N-k+1) ways, again by the multiplication principle • Just equate the selection of an object with an ordered k-tuple The elements are each drawn from sets of decreasing size • When k=N, we can select in N! ways

17. Tabular View • Repetition • WithWithout • W/ OrderN^k N!/(N-k)! • WO/ Order (N+k-1)!/ (N-1)! k! N!/(N-k)!k! • (N+k-1 choose k) (N choose k) • It is easier to remember these, when they are considered as a group

18. Factorial Interpretation • The factorial n! = n(n-1)(n-2) …(3)(2)(1) 0!=1 • It gets very big very fast, • n! asy sqrt(2pi)n^(n+1/2)exp(-n) • Then (N)(N-1)…(N-k+1) = N! / (N-k)! • Best ways to compute the factorial is through the Gamma Function (gamma.m) More later. • You can permute (reorder) an N-tuple in N! ways

19. Factorial Growth

20. Stirling’s Formula

21. functioncompareGamma(M); m = [1:M]';half=1/2; fact = gamma(m+1); plot( m, fact, 'b*-');grid title('m! = Gamma(m+1) - Grows very fast'); xlabel('m');ylabel('G(m)'); %try logs to compress the dynamic range figure(2) subplot(121); stir = sqrt(2*pi).*m.^(m+half).*exp(-m); semilogy( m, fact, 'b*-',m,stir,'ro');grid subplot(122); RE = abs(stir-fact)./fact; semilogy(m,RE,'r*');grid title('Relative Error'); figure(1) Matlab M-File

22. Unordered selection wo/ Repetition • What if we select without regard to the order. • Solve this by reconsidering ordered selection wo/ replacement of k objects from N-set • Each such selection resulted a k-tuple that indexed the selection a_i is that object selected on the i^th turn • But a k-tuple can be reordered (permuted) in k! ways • Therefore we need to reduce our result to • N(N-1)…(N-k+1)/k! = N! / (N-k)! K! • Combinations - binomial coefficient ( N choose k )

23. Tabular View • Repetition • WithWithout • W/ Order N^k N!/(N-k)! • WO/ Order (N+k-1)!/ (N-1)! k! N!/(N-k)!k! • (N+k-1 choose k) (N choose k) • It is easier to remember these, when they are considered as a group

24. Computing Binomial Coefficient • { n choose k } = n! / (n-k)! k! • But n! = gamma(n+1) = G(n+1) (gamma.m) • { n choose k } = G(n+1)/G(n+1-k) G(k+1) • Even better to use loggamma = log(Gamma(m)) • { n choose k } = • exp( gammln(n+1) -gammaln(n+1-k) -gammaln(k+1) )

25. More Binomial Coefficient • binomialcoef(n) Think (1+x)^n. The coefficients: • 1 1 • 1 2 1 • 1 3 3 1 • 1 3 5 3 1 • 1 4 9 9 4 1 • 1 6 15 19 15 6 1 • Computation – recursion, approximate, logGamma

26. Matlab Code • function binomialcoef(n); • %generates an array of binomial coefficients • k = [0:n]; • lgGn = gammaln(n+1); • lgGnk = gammaln(n+1-k); • lgGk = gammaln(k+1); • bico = exp( lgGn -lgGnk -lgGk ); • bico = floor(bico); • fprintf('%5.0f', bico);

27. Unordered w/ Repetition • This is the most complicated situation so far • Thinks for a long time, then come up with a • Clever way to represent a selection bars & balls • 4 objects choose 2  o||o| select object 1 and 3 • Another choice is |||oo select object 4 twice • To select k objects from N-set • Use N-1 bars and k balls  string of |||oo • But these can be drawn in ( N+k-1 choose k) ways • (N+k-1)!/(N-1)!k!

28. Tabular View • Repetition • WithWithout • W/ Order N^k N!/(N-k)! • WO/ Order (N+k-1)!/ (N-1)! k! N!/(N-k)!k! • (N+k-1 choose k) (N choose k) • It is easier to remember these, when they are considered as a group