Discrete Mathematics 6 th edition, 2005

Discrete Mathematics 6th edition, 2005 Chapter 6 Counting methods and the pigeonhole principle • Basic Principles • Permutations and Combinations • Algorithms for Generating Permutations and Combinations • Introduction to Discrete Probability* • Discrete Probability Theory* • Generalized Permutations and Combinations • Binomial Coefficient and Combinatorial Identities

6.1 Basic Principles • Menu for Quick Lunch • 2 appetizers, 3 main courses, 4 beverages • 24 possible dinners consisting of 1 appetizer, 1 main course and 1 beverage • NHT, NHM, NHC, NHR, • NCT, NCM, NCC, NCR, • NFT, NFM, NFC, NFR, • SHT, SHM, SHC, SHR, • SCT, SCM, SCC, SCR, • SFT, SNM, SFC, SFR APPETIZERS Nachos …………… 2.15 Salad ……………… 1.90 MAIN COURSES Hamburger ……. 3.25 Cheeseburger... 3.65 Fish Filet ……….. 3.15 BEVERAGES Tea ………………… .70 Milk ……………….. .85 Cola ………………. .75 Root Beer ……… .75

Multiplication Principle Multiplication principle If an activity can be performed in k successive steps, • Step 1 can be done in n1 ways • Step 2 can be done in n2 ways • … • Step k can be done in nk ways Then: the number of different ways that the activity can be performed is the product n1n2…nk

Addition Principle • How many 8-bit strings that begin either 101 or 111? • 8-bit strings that begin 101 • can be constructed in five successive steps • select the 4-th bit; select the 5-th bit;… • 2 2 2 2 2 = 25 = 32 8-bit strings • 8-bit strings that begin 111 • can be constructed in five successive steps • select the 4-th bit; select the 5-th bit;… • 2 2 2 2 2 = 25 = 32 8-bit strings • 8-bit strings that begin either 101 or 111 • 32 + 32 = 64 8-bit strings

Addition Principle Addition principle Let X1, X2,…, Xk be a collection of k pairwise disjoint sets, each of which has nj elements, 1 <j<k, then the union of those sets k X = Xj j =1 has n1 + n2 + … + nk elements

A permutationof n distinct elements x1, x2,…, xn is an ordering of the n elements. Example: there are 3! = 6 permutations of three elements a, b, c: abc bac cab acb bca cba 6.2 Permutations and Combinations Theorem 6.2.3 There are n! permutations of n elements

r-permutations • An r-permutation of n distinct elementsis an ordering of an r-element subset of {x1, x2,…, xn} • Example: • The number of 2-permutations of X={a,b,c} is 32 = 6 • ab, ac, ba, bc, ca, cb Theorem 6.2.10: The number of r-permutations of a set of n distinct objects is P(n,r) = n(n-1)(n-2)…(n-r+1), rn

Theorem 6.2.17: The number of r-combinations of a set of n distinct objects is C(n,r) =  =  =  , rn P(n,r) n(n-1)(n-2)…(n-r+1) n! r! r! (n-r)!r! Combinations Let X = {x1, x2,…, xn} be a set containing n distinct elements • An r-combination of X is an unordered selection of r elements of X, for r<n • The number of r-combinations of n distinct element is denoted C(n,r) or () n r

Combinations • How many routes are there • from the lower-left corner of an nn square grid • to the upper-right corner • if we are restricted to traveling only to the right or upward? • Solution • Each route can be described by a string of nR’s and nU’s • ex: n=4, RUURRURU • Any such string can be obtained by selecting n positions for the R’s, without regard to the order of positions with U’s • Thus there are C(2n, n) possible routes

Catalan Numbers • How many routes are there • from the lower-left corner of an nn square grid • to the upper-right corner • if we are restricted to traveling only to the right or upward • and if we are allowed to touch but not go above a diagonal line from the lower-left corner to the upper-right corner? (o) (x)

Catalan Numbers • Solution • Gn + Bn = C(2n, n) • good route: a route that touches but does not go above the diagonal • bad route: a route that goes above the diagonal • Bn = the number of routes for an (n+1)(n-1) grid • Given a bad route, • find the first move that takes it above the diagonal • replace each right move by up move • replace each up move by right move • Given any (n+1)(n-1) route, • find the first move that takes it above the diagonal • rotate the remainder of the route about the dashed line • This function is 1:1 • C(2n, n)- Bn=C(2n, n)- C(2n, n-1) =  -  = ( - )=    =  =  (2n)!(2n)! n!n!(n-1)!(n+1)! (2n)! 1 1 (2n)! 1 n!(n-1)!nn+1 n!(n-1)! n(n+1) (2n)! C(2n, n) (n+1)n!n!n+1 r = n

Catalan Numbers • Eugene-Charles Catalan (1814-1894) • Catalan numbers are defined by the formula Cn = C(2n,n) / (n+1) for n = 0, 1, 2,… The first few terms are:

6.3 Algorithms for Generating Permutations and Combinations • Lexicographic order • Given two strings  = s1s2…sp and  = t1t2…tq over {1,2,…,n} • Define  <  if • p < q and si = ti for all i = 1, 2,…, p • Or for some i,si  ti and for the smallest i, si <ti • Example: if  = 1324,  = 1332, = 132, then  <  and  < .

6.4 Introduction to Discrete Probability • An experiment is a process that yields an outcome • An event is an outcome or a set of outcomes from an experiment • The sample space is the event of all possible outcomes

Probability of an event is the number of outcomes in the event divided by the number of outcomes in the sample space. If S is a finite sample space and E is an event (E is a subset of S) then the probability of E is P(E) = |E| / |S| Example 2 fair dice are rolled. What is the probability that the sum of the numbers on the dice is 10? There are 3 ways: (4,6), (5,5), (6,4)  The size of the event is 3  The probability is 3/36 = 1/12 Probability

6.5 Discrete Probability Theory • When all outcomes are equally likely and there are n possible outcomes, each one has a probability 1/n. • BUT this is not always the case. When all probabilities are not equal, then some probability (possibly different numbers) must be assigned to each outcome.

Probability Function • A probability functionP assigns to each outcome x in a sample space S a number P(x) so that 0P(x)1, for all xS, and  P(x) = 1 x  S • Example • 2 though 6 of die are equally likely appear P(2) = P(3) = P(4) = P(5) = P(6) • 1 is 3 times as likely as any other number to appear P(1) = 3P(2) • 1= P(1) + P(2) + P(3) + P(4) + P(5) + P(6) = 3P(2) + P(2) + P(2) + P(2) + P(2) + P(2) = 8P(2) • P(2) = P(3) = P(4) = P(5) = P(6) = 1/8 P(1) = 3/8

Probability of an Event • The probability of an eventE is defined as the sum of he probabilities of the outcomes in E P(E) =  P(x) x  E • Example • probability of an odd number is P(1)+P(3)+P(5) = 3/8+1/8+1/8 = 5/8

Theorem 6.5.5 Let E be an event. The probability of E, the complement of E, satisfies P(E) + P(E) = 1 Probability of an Event • Suppose that E={x1, …, xk} and E={xk+1, …, xn} • Then k n P(E) =  P(xi) and P(E) =  P(xi) i=1 i=k+1 • Now k n P(E) + P(E) =  P(xi) +  P(xi) i=1 i=k+1 n =  P(xi) = 1 i=1 Proof

Probability of an Event Example: Birthday Problem • Find the probability that among n persons, at least two people have birthdays on the same month and date. • Assume that all month and dates are equally likely, and ignore February 29 birthdays. • E: the event “at least two persons have the same birthday” • E: the event “no two persons have the same birthday” • the size of the sample space: 365n • |E| = 365  364  …  (365-n+1) • P(E) = [ 365  364  …  (365-n+1) ] / 365n • P(E) = 1 - P(E) • n=22  P(E) = 0.475695, n=23  P(E) = 0.507297 > ½

Probability of Two Events Theorem 6.5.9 Given any two events E1 and E2 in a sample space S. Then P(E1E2) = P(E1) + P(E2) – P(E1E2) Proof • Let E1 = {x1, …, xi} E2 = {y1, …, yj} E1 E2 = {z1, …, zk} • Then in the list x1, …, xi,y1, …, yj, z1, …, zk occurs twice • It follows that i j k P(E1E2) =  P(xt) +  P(yt) -  P(zt) t=1 t=1 t=1 = P(E1) + P(E2) – P(E1E2)

Probability of Two Events Example • Among a group of students, some take art and some take computer science. • A student is selected at random. • Let A be the event “the student takes art,” and • let C be the event “the student takes computer science.” • Then AC is the event “the student takes art or computer science or both,” and • AC is the event “the student takes art and computer science.”

Probability of Two Events Example • Two fair dice are rolled. • What is the probability of getting doubles or a sum of 6? • E1: get doubles • P(E1) = 6/36 = 1/6 • E2: get a sum of 6 • P(E2) = 5/36 • E1 E2: get doubles and get a sum of 6 • P(E1 E2) = 1/36 • P(E1 E2) = 1/6 + 5/36 - 1/36 = 5/18 [(1,1), (2,2), (3,3), (4,4), (5,5), (6,6)] [(1,5), (2,4), (3,3), (4,2), (5,1)]

Mutually Exclusive Events • Events E1 and E2 are mutually exclusive if and only if E1E2 = . • Example • E1: get doubles • P(E1) = 6/36 = 1/6 • E2: get a sum of 5 • P(E2) = 4/36 = 1/9 • E1 E2: get doubles and get a sum of 5 • P(E1 E2) = 1/6 + 1/9 = 5/18 [(1,1), (2,2), (3,3), (4,4), (5,5), (6,6)] [(1,4), (2,3), (3,2), (4,1)] Theorem 6.5.11 If E1 and E2 are mutually exclusive events, P(E1  E2) = P(E1) + P(E2)

Conditional Probability • Conditional probability • is the probability of an event E, • given that another event F has occurred. • In symbols P(E|F). • If P(F) > 0 then P(E|F) = P(EF) / P(F) • Example • Weather records show that the probability of high barometric pressure is 0.80, and • the probability or rain and high barometric pressure is 0.10. • The probability of rain given high barometric pressure is P(R|H) = P(RH) / P(H) = 0.10 / 0.80 = 0.125 • R: the event “rain” • H: the event “high barometric pressure”

Independent Events • If the probability of event E does not depend on event F • P(E|F) = P(E) • E and F are independent events • P(E)= P(E|F) = P(EF) / P(F)  P(EF) = P(E)P(F) • Two events E and F are independent if P(EF) = P(E)P(F)

Independent Events Example • Intuitively, if we flip a fair coin twice, the outcome of the second toss does not depend on the outcome of the first toss • H: the event “head on first toss” • T: the event “tail on second toss” • HT: the event “head on first toss and tail on second toss” • P(HT) = P(H)P(T) = ½  ½ = ¼

Pattern Recognition • Pattern recognition places items into classes, based on various featuresof the items. • Given a set of features F we can calculate the probability of a class C, given F: P(C|F) • Place the item into the most probable class, i.e. the one C for which P(C|F) is the highest. • Example: Wine can be classified as Premium (R), Table wine (T) or Swill (S). Let F {acidity, body, color, price} • Suppose a wine has feature F, and P(T|F) = 0.5, P(R|F) = 0.2 and P(S|F) = 0.3. Since P(T|F) is the highest number, this wine will be classified as table wine.

P(F|Cj)P(Cj) n  P(F|Ci)P(Ci) i=1 Bayes’ Theorem Theorem 6.5.20 • Given pairwise disjoint classes C1, C2,…, Cn and a feature set F, then • P(Cj|F) = Proof • P(Cj|F) = P(CjF)/P(F) P(F|Cj) = P(FCj)/P(Cj)  P(Cj|F) = P(CjF)/P(F) = P(F|Cj)P(Cj)/P(F) • F = (FC1)(FC2)… (FCn) • Ci are pairwise mutually exclusive • (FCi) are pairwise mutually exclusive  P(F) = P(FC1)P(FC2)… P(FCn) • P(FCi) = P(F|Ci)P(Ci) n  P(F) =  P(F|Ci)P(Ci) i=1

Example - Bayes’ Theorem (1/4) • The ELISA test is used to detect antibodies in blood and indicates the presence of the HIV virus. • 15% of the patients at on clinic have the HIV virus. • Among those that have the HIV virus, 95% test positive on the ELISA test. • Among those that do not have the HIV virus, 2% test positive on the ELISA test. • Problem: Find the probability that a patient has the HIV virus if the ELISA test is positive. P(Pos|H)=0.95 P(Pos|H)=0.02 P(H)=0.85 P(H)=0.15

Example - Bayes’ Theorem (2/4) • Problem: Find the probability that a patient has the HIV virus if the ELISA test is positive. P(Pos|H)=0.95 P(A)=0.15*0.95 P(Pos|H)=0.02 P(B)=0.85*0.02 B B A A P(H)=0.85 P(H)=0.85 P(H)=0.15 P(H)=0.15

Example - Bayes’ Theorem (3/4) • Problem: Find the probability that a patient has the HIV virus if the ELISA test is positive. • P(A) / P(A  B) P(A | (AB)) = 0.1425 / (0.1425+0.017) = 0.893 P(A)=0.1425 P(B)=0.017 B B A A

Example - Bayes’ Theorem (4/4) The ELISA test • is used to detect antibodies in blood and indicate the presence of the HIV virus. • 15% of the patients at on clinic have the HIV virus. • Among those that have the HIV virus, 95% test positive on the ELISA test. • Among those that do not have the HIV virus, 2% test positive on the ELISA test. • Problem • Find the probability that a patient has the HIV virus if the ELISA test is positive. • Solution • H: the classes that are “has the HIV virus” • H: the classes that are “does not have the HIV virus” • Pos: the feature that is “test positive” • P(H)=0.15, P(H)=0.85, P(Pos|H)=0.95, P(Pos|H)=0.02 • P(H|Pos) = { P(Pos|H) P(H) } / {P(Pos|H) P(H) + {P(Pos|H) P(H) } = (0.95)(0.15) / {(0.95)(0.15) + (0.12)(0.85)} = 0.893 Bayes’ Theorem

At the telemarketing firm SellPhone, Dale, Rusty, and Lee make calls. What are P(D|H), P(R|H), P(L|H)? Example: Telemarketing (1/4)

Example: Telemarketing (2/4) P(H|R) = 0.55 P(H|D) = 0.2 P(H|L) = 0.30 P(L) = 0.35 P(D) = 0.4 P(R) = 0.25

P(D|H) = P(H|D)P(D) P(H) P(R|H) = P(H|R)P(R) P(H) P(L|H) = P(H|L)P(L) P(H) Example: Telemarketing (4/4) 0.08 0.3225 0.13750.3225 P(D|H) = = 0.248 P(R|H) = = 0.426 0.105 0.3225 P(L|H) = = 0.326 P(D|H) + P(R|H) + P(L|H) = 1

6.6 Generalized Permutations and Combinations Example • Problem • How many strings can be formed using the following letters? MIS SIS SIP PI • Solution • 11! ? • no. why? duplication of letters! • Fill 11 blanks with the letters given . • There are C(11,2) ways to choose positions for the two P’s. • There are C(9,4) ways to choose positions for the four S’s. • There are C(5,4) ways to choose positions for the four I’s. • There are 1 position to be filled by M. • By the multiplication principle, the number of ways of ordering the letters C(11,2) C(9,4) C(5,4) = = = 34,650 2 Ps, 4 Ss, 4 Is and 1 M 11! 9! 5! 11! 2!9! 4!5! 4!1! 2!4!4!1!

Proof C(n, n1)C(n-n1, n2)C(n-n1-n2, n3) … C(n-n1-…-nt-1, nt) = … = n!(n-n1)!(n-n1- …-nt-1)! n1!(n-n1)!n2!(n-n1-n2)!nt0! n! n1!n2!...nt! Generalized Permutations and Combinations Theorem 6.6.2: Suppose that a sequence S of n items has • n1 identical objects of type 1, • n2 identical objects of type 2, … and • nt identical objects of type t. Then the number of orderings of S n! n1!n2!...nt!

Generalized Permutations and Combinations Example • Problem • Consider 3 books • a computer science book, a physics book, and a history book. • Suppose that the library has at least six copies of each of these books. • How many ways can we select six books? • Solution • The problem is to choose unordered, repetitions allowed. • We use two bookends to separate 3 kinds of books. • By the bookends’ positions, the selection is determined. • There are 8 positions: 6 books’ positions + 2 bookends’ positions • The possible number of bookends’ positions : C(8,2) 3 2 1 0 4 2 3 0 3

Generalized Permutations and Combinations Theorem 6.6.5: If X is a set containing t elements, the number of unordered, k-element selections from X, repetitions allowed, is C(k+t-1, t-1) = C(k+t-1, k) Proof • Let X = {a1,…,at} • Consider k+t-1 slots and k+t-1 symbols (k’s and (t-1) /’s) • A selection: each placement of these symbols into the slots • The number n1 of ’s up to the 1st /: the selection of n1a1’s. • The number n2 of ’s between the 1st and 2nd /’s: the selection of n2a2’s. • And so on. • There areC(k+t-1, t-1) way to select the positions for the /’s and = The number of way to select the position for the ’s: C(k+t-1, k)

n factors 6.7 Binomial Coefficients and Combinatorial Identities • (a+b)n = (a+b) (a+b) … (a+b) • select either a or b from each of n factors • multiply the selections together • sum all such products • Example: (a+b)3 (a+b)3 = a3+a2b+a2b+ab2+a2b+ab2+ab2+b3 = a3+3a2b+3a2b+b3 Selection from First Factor (a+b) a a a a b b b b Selection from Second Factor (a+b) a a b b a a b b Selection from Third Factor (a+b) a b a b a b a b Product of Selections aaa = a3 aab = a2b aba = a2b abb = ab2 baa = a2b bab = ab2 bba = ab2 bbb = b3

n factors Binomial Coefficient Binomial Coefficients and Binomial Theorem • (a+b)n = (a+b) (a+b) … (a+b) • A term of the form an-kbk arises from choosing b from k factors and a from the other n-k factors. • This can be done in C(n,k) ways.  an-kbk appears C(n,k) times  (a+b)n = C(n,0)anb0 + C(n,1)an-1b1 + … + C(n,n-1)a1bn-1 + C(n,n)a0bn Theorem 6.7.1: Binomial Theorem If a and b are real numbers and n is a positive integer, then (a+b)n =  C(n,k)an-kbk

9! 7! 9! 2!7! 3!4! 2!3!4! Binomial Coefficients and Binomial Theorem Example • Problem • Find the coefficient of x2y3z4 in the expansion of (x+y+z)9 • Solution • (x+y+z)9 = (x+y+z) (x+y+z)…(x+y+z) (9 times) • we obtain x2y3z4 each time we multiply together • x chosen from 2 of the 9 terms, • y chosen from 3 of the 7 terms, and • z chosen from 4 of the 4 terms  the coefficient is C(9,2)C(7,3) = = = 1260

Pascal’s Triangle 1 1 1 1 2 1 1 3 3 1 1 4 6 4 1 1 5 10 10 5 1 1 6 15 20 15 6 1 1 7 21 35 35 21 7 1 1 8 28 56 70 56 28 8 1 1 9 36 84 126 126 84 36 9 1 … coefficients of (a+b)n n = 0 n = 1 n = 2 …

6.7 Binomial Coefficients and Combinatorial Identities Theorem 6.7.6 C(n+1,k) = C(n,k) + C(n,k-1), for 1 <k<n Proof • Let X be a set with n elements • Choose a  X  C(n+1,k) is the number ofk-element subsets ofY=X{a} • Y can be divided into two disjoint classes: • Subsets of Y not containing a. : k-element subsets ofX C(n,k) • Subsets of Y containing a. : (k-1)-element subsets ofX together with a  C(n,k-1) C(n+1,k) = C(n,k) + C(n,k-1)

6.8 The Pigeonhole Principle • First form: If n pigeons fly into k pigeonholesand k < n, some pigeonhole contains at least two pigeons. 9 10 7 8 5 6 11 3 2 12 4 1 pigeonhole

Combinatorial Identities andCombinatorial Argument • Combinatorial Identity • An identity that results from some counting process • Combinatorial Argument • The argument that leads to its formulation

a b c d e f 1 2 3 4 5 X Y Second Form of the Pigeonhole Principle • If X and Y are finite sets with |X| > |Y| and f : X Y is a function, then f(x1) = f(x2) for some x1, x2  X, x1  x2.

a b c d e 1 2 3 X Y Third Form of the Pigeonhole Principle • Let f be a function from a finite set X into a finite set Y. • Suppose |X| = n, |Y| = m. • Let k = n/m. • Then there are at least k values a1, a2,…, ak  X such that f(a1) = f(a2) = … f(ak). • Example: n = 5, m = 3 k = n/m = 5/3 = 2.

Discrete Mathematics 6 th edition, 2005