Download
1 / 95
970 likes | 1.18k Views
COMPS263F. Li Tak Sing Lectures 23-28. Counting and Discrete Probability. Permutations The number of ways to arrange n different objects in a line: n(n-1)(n-2)....1=n! 0!=1 This is because there is only one way in arranging 0 items in a line.
E N D
COMPS263F Li Tak Sing Lectures 23-28
Counting and Discrete Probability Permutations The number of ways to arrange n different objects in a line: n(n-1)(n-2)....1=n! 0!=1 This is because there is only one way in arranging 0 items in a line. The number of permutations of n objects taken r at a time is given by P(n,r)=n(n-1)...(n-r+1)=
Permutations with repeated elements If a bag contains some objects in which m1 are of type 1, m2 are of type 2, .... mk are of type k. The number of permutation is:
Example Worst-case lower bound for comparison sorting A list of n numbers can be arranged in n! ways. So a decision tree for sorting n numbers must contain n! leaves. Since a binary tree of depth d can contain at most 2dleaves, so we have: n!≤2d log2n! ≤d log2n! is approximately nlog2n nlog2n ≤d
People in a circle • n different people are sitting arrange a circle table. • the number of way to sit is (n-1)!.
Combination • The number of ways of taking r objects from n different objects so that the order is not import is: • C(n,r) is the coefficient of xr in the expansion of (x+1)n • C(n,r) can also be written in the form .
Example Suppose we want to build a code to represent 29 objects in which each object is represented as a binary string length n, which consists of k 0's and m 1's and n=k+m. Find n, k and m, where n has the smallest possible value. (n=7,k=4, m=3)
Bag combinations • Now we want to know the number of way to form a bag of k elements from a set of n elements. • This can be though of as arranging n+k-1 balls so that k of them are white balls and n-1 of them are black balls. The n-1 black balls divides the k white balls into n groups. We can then consider the first group as the elements from the 1st element of the set, the second from the second element and so on. So the number of ways= .
Example • In how many ways can four coins be selected from a bags of $1, $2, $5 and $10 coins. () • n how many ways can a committee of 5 people be selected from a group of doctors and nurses. () • In how many ways can five people be selected from a collection of Democrats, Republicans, and Independents? Here we are choosing five-element bags from a set of three characteristics {Democrat, republican, Independent}. ()
Discrete probability The set of all possible outcomes of an experiment is called a sample space or probability space. The elements in a sample space are called sample points or simply points. Any subset of a sample space is called an event. A probability distribution on a sample space S is an assignment of probabilities to the points of S such that the sum of all the probabilities is 1. Le S={x1, x2, ...xn} be a sample space. A probability distribution P on S is a function P: S[0,1]such that P(x1)+P(x2)+...+P(xn)=1.
Discrete probability The probability of an event E is denoted by P(S)=1 P()=0 P(AB)=P(A)+P(B)-P(AB) P(E')=1-P(E)
The birthday problem • Given n people in a room, what is the probability that at least two of the people have the same birthday (month and day)? • Assume that a year has 365 days. So the sample space consists of 365n elements. • Now, we want to calculate the number of cases where no two people have the same birthday. • The first person has 365 choices, the second has 364 choices, the last has 366-n choices.
The birthday problem • The total number of choices are 365*364*...*(366-n). • So the probability is • For 40 person, the probability is 0.891. So it is very likely to have two persons having the same birthday.
Conditional probability • P(A|B)=probability that event A would occur given that B has occurred= • If E1, ...En form a partition of S, then • If A and B are independent event, then P(A)=P(A|B)=P(A|B'). This means that the occurrence of B does not in any way affect the occurrence of A. • Therefore P(A)P(B)=P(AB)
Repeated independent trials • Let there be a game so that the probability of wining is p. Then the game is played for n times. The probability of winning exactly r times is:
Example • A team has probability 2/3 of winning whenever it plays. Find each of the following probabilities that the team will win. • Exactly 4 out of 5 games. • At most 4 out 5 games. • Exactly 4 out of 5 games given that it has already won the first 2 games of a 5-game series. • A student is chosen at random from a class of 80 students that has 20 honor students, 30 athletes, and 40 that are neither honor students nor athlets. • What is the probability that the student selected is an athlete given that he or she is an honors students. • What is the probability that the student selected is an honors student given that he or she is an athlete? • Are the events "honors student" and "athlete" independent?
Solution • a) b)1-c) • S-all students,H-honor students, A-athletes. Therefore |S|=80,|H|=20, |A|=30, |HA|=40, |HA|=|H|+|A|-|HA|=10a)
Solution • b)c) therefore can see that H and A are not independent.
Examples • A computer program uses one of three procedures for each piece of input. The procedures are used with probabilities 1/3, 1/2, 1/6. Negative results are detected at rates of 10%, 20%, and 30% by the three procedures, respectively. Suppose a negative result is detected. Find the probabilities that each of the procedures was used. • A commuter crosses one of three bridges, A, B, or C, to go home from work, crossing A with probability 1/3, B with probability 1/6, and C with probability 1/2. The commuter arrives home by 6 p.m. 75%, 60% and 80% of the time by crossing bridges A, B, and C, respectively. If the commuter arrives home after 6 p.m., find the probability that bridge A was used. Also find the probabilities for bridges B and C.
Expectation • expectation = average behavior • For example, in rolling a die, the probabilities of different outcomes are: 1: 1/6 2: 1/6 3: 1/6 4: 1/6 5: 1/6 6: 1/6So if a die is rolled n times, roughly n/6 of them would be 1, n/6 of them would be 2, etc. We can then calculate the average of the outcome: where P(xr) is the probability of event xr and V(xr) is the value of the outcome.
Example • Find the expected score obtained by guessing all 100 questions of a true-false exam in which a correct answer is worth 1 point and an incorrect answer is worth -1/2 point. So would you suggest a student to do a random guess if he/she does not know the answer? • In a simple rolling die game, if a player bets $1 for a number 1 to 6, and if he wins, he will get 4 dollars. Is this game a fair game?
Average Performance of an Algorithm • If in an algorithm, there are a number of possible inputs with different probabilities, and each input may result in different number of operations being executed, then we can compute the average number of operations accordingly. • Suppose the sample space is {I1, I2,...,Ik}, P(Ij) is the probability of the input being Ij, V(Ij) is the number operations executed if the input is Ij. Then, the average number of operations executed is: • To show that an algorithm A is optimal in the average case for some problem, we need to specify a particular sample space and probability distribution. Then we need to show that AvgA(n)AvgB(n) for all n>0 and for all algorithms B that solve the problem. • However, it is difficult to find the optimal algorithm. So we usually just find the best from known algorithms.
Example • Analysis of sequential search • To search an element X in an array L The algorithm is: i:=n; while i1 and XL[i] do i:=i-1 od The even Ij, j=1,..,n represents the case when X is at position j. In+1 represents the case when X is not in L. So V(Ij)=n-j+1 for j=1,...,n. V(In+1)=n. Assume that q is the probability that X is in L. So P(Ij)=q/n for j=1,...,n. P(In+1)=1-q. AvgA(n)=E(V)=(n+(n-1)+...+1)(n/q)+(1-q)n=q(n+1)/2 + (1-q)n
Example • So if we know that X is in L, then q=1, then AvgA(n)=(n+1)/2. If X is not in L, then q=0, AvgA(n)=n.
Monte Carlo Method • Sometimes it is difficult to find a formula to a problem. We can use a statistical method called Monte Carlo method. • For example, in finding the area of a shape, we can draw the same on a piece of paper. Then, we can randomly put points onto the paper. The percentage of points inside the shape can then be used to estimate the area of the same. • In estimating the average performance of an algorithm, we need to have some random input and then find the average number of operations accordingly.
Example • Assume that we have a sorted list of 15 elements, x1, x2, ..., x15. Calculate the average number of comparisons made by a binary search algorithm to look for a key that may or may not be in the list. Assume that the key has probability 1/2 of being in the list and that each of the events "key=xi" is equally likely for i=1..15. • Generalize the problem to find a formula for the average number of comparisons used to look for a key in a sorted list of size n=2k-1, where k is a natural number. Assume that the key has probability p of being in the list and that each of the events "key=xi" is equally likely for i=1,..,n.
Chapter 11 Regular Languages and Finite Automata The regular Languages Regular languages are constructed from the letters of an alphabet by using the language operations of union, concatenation, and closure. , {}, and {a} are regular languages for all aA. If L and M are regular languages, then the following languages are also regular: LM, ML, and L*. There can be many regular Languages.
The regular languages For example: A={a,b} The following are all regular languages of A: {} {a} {b} {a,b} {aa,bb} Actually, any finite set that contains elements of strings that contain a, b are regular languages of A.
Regular Expressions A regular language is often described by means of an algebraic expression called a regular expression. , , and a are regular expressions for all aA. If R and S are regular expressions, then the following expressions are also regular: (R), R+S, RS, and R* For example, A={a,b} The following are regular expressions of A: , , a, b, +b, b*, a+(ba), (a+b)a, ab*, a*+b*.
Regular Expressions To avoid using too many parentheses, we assume that the operations have the following hierarchy: • * highest (do it first) • . • + lowest (do it last). So a+ba* is equivalent to (a+(b(a*))). In addition, we can omit . There, ab can be written as ab.
Relations between regular expressions and regular languages • L()= • L()={} • L(a)={a} for each aA • L(R+S)=L(R)L(S) • L(RS)=L(R)L(S) • L(R*)=L(R)*
Example L(a*+b*) = L(a*)L(b*) =L(a)* L(b)* ={a}*{b}* ={, a, a2, a3,....} {, b, b2, b3,....} ={, a, b, a2, b2, a3, b3, ..... }
Problems Find a regular expression to describe each of the following languages. • {a,b,c} • {aa,ab,ac} • {a, b, ab, ba, abb, baa, ...., abn, ban,...} • {a, aaa, aaaaa, ...., a2n+1} • {, a, abb, abbbb, ..., ab2n, ... } • {, a, b, c, aa, bb, cc, ..., an, bn, cn, ....} • {ambcn| m,nN}
Equality of two regular expressions • Different regular expressions may be related to the same regular languages.L(a+b)=L(b+a)={a,b} • Two regular expressions are said to be equal if they correspond to the same regular language.S=R if L(S)=L(R) • So we can say that a+b=b+a • (a+b)+(b+a)=a+b • abba
Properties of Regular Expressions 1. (+ properties) R+T=T+R R+=+R=R R+R=R (R+S)+T=R+(S+T) 2. ( properties) R=R= R=R=R (RS)T=R(ST) 3. (Distributive properties) R(S+T)=RS+RT (S+T)R=SR+TR
Properties of Regular Expressions (Closure properties) 4. *=*=. 5. R*=R*R*=(R*)*=R+R*. R*=+R*=(+R)*=( +R)R*=+RR* R*=(R+...+Rk)* for any k1, R*=+R+...+Rk-1+RkR* for any k1. 6. R*R=RR* 7. (R+S)*=(R*+S*)*=(R*S*)*=(R*S)*R*=R*(SR*)* 8. R(SR)*=(RS)*R 9. (R*S)*= +(R+S)*S, (RS*)*=+R(R+S)*
Problems • Find a regular expression for each of the following languages over the alphabet {a,b}. • Strings with even length. • Strings whose length is a multiple of 3. • Strings containing the substring aba. • Strings with an odd number of a's. • Find a regular expression over the alphabet {0,1} to describe the set of all binary numerals without leading zeros (except 0 itself). So the language is the set {0, 1, 10, 11, 100, 101, 110, 111,...}
Example Write down the regular expressions for the following regular languages of the alphabet A={a,b,c} . (a) {abc} (b) {abc, acb} (c) {aibjck| i, j, k N} (d) {ab, bc, ca, ba, cb, ac} (e) {aa, bb, cc}
Solution (a) abc (b) a(bc+cb) (c) a*b*c* (d) a(b+c)+b(c+a)+c(a+b) (e) aa+bb+cc
Example Write down the regular languages for the following regular expressions of alphabet A={a,b,c} (a) (a+b)b* (b) abc* (c) a+c* (d) b(ab)* (e) (a+b)*
Solution (a) {abk, bbk| kN} (b) {abck|kN} (c) {a, bk| kN} (d) {b(ab)k|kN} (e) {}{r1r2…rn| r1, r2, …rn{a,b}}
Finite Automata • This is a way of determining whether a string belongs to a language. • Deterministic Finite Automata • A deterministic finite automaton over a finite alphabet A can be thought of as a finite directed graph with the property that each node emits one labeled edge for each distinct element of A. • The nodes are called states. • There is a special state called the start or initial state, and there is a set of states called final states. • It can be abbreviated as DFA.
Deterministic Finite Automata • The following figure represents a DFA over the alphabet A={a,b} • Start represents the start state. • Final state is presented by a double circle. • Each state emits exactly two arrows, one labeled with a and one labeled with b.
The execution of a DFA • The movement along an edge from one state to another is called a state transition, and the input letter corresponding to the label on the edge is consumed by the state transition. • A DFA accepts a string w in A* if the execution path ends in a final state. Otherwise, the DFA rejects w. The language of a DFA is the set of strings that it accepts. • For example, the above DFA accepts the following strings: • abbaaba • baaabab • bbaa • The DFA rejects the following strings: • abbbb • baaa
Examples • (a+b)* • a(a+b)* • State 2 acts as an error state because every state need to emits two arrows.
Some simple rule: • a+b • a*b • ab*
Problems Draw a DFA to recognize the language of each of the following regular expressions. • a+ba*. • a*b. • ba*.
Nondeterministic Finite Automata • Similar to a DFA, but each state emits zero or more edges, and each edge is labeled either with a letter from A or with . • A state may emit more than one edge labeled with the same letter.
The execution of an NFA • Similar to a DFA except that an edge labeled with the empty string is traversed without consuming an input letter. • Since a state may emit an edge labeled with or it may emit more than one edge labeled with the same letter, there may be alternative paths to acceptance, which is why it is nondeterministic.
Examples • a*a • aa*
