Theory of Probability

1. Theory of Probability Complied by Dr Kunal Pathak

2. Flow of the presentation • Need of the study • Basic Terminology • Mathematical / Classical definition of Probability • Limitation of Mathematical definition • Permutation and Combination • Statistical definitions and Limitations • Axiomatic definitions • Theorems on Probability (based on set theory) • Conditional Probability • Bayes theorem • Markov Chain

3. Need of the Study • If an experiment is repeated under essentially homogeneous and similar conditions, we generally come across two types of situations 1) the result is certain: deterministic e.g. i) the velocity V = u + at ii) Ohm’s law V= IR 2) the result is uncertain: probabilistic e.g. i) life of electric bulb ii) future demand of product

4. Probabilistic approach • Certainty : 1 • Impossibility: 0 • Uncertainty: (0, 1)

5. Basic Terminology • Random Experiment: If in each trial of an experiment conducted under identical conditions, the outcome is not unique, but may be one of the possible outcomes, then such an experiment is called a random experiment. e.g. tossing of a coin, throwing a die, selecting a card from a pack,… • Outcome: The result of a random experiment will be called an outcome. e.g. tossing of a coin: H/T throwing a die: 1/2/3/4/5/6

6. Trial and Event: Any particular performance of a random experiment is called a trial and outcome or combination of outcomes are termed as event. e.g. 1)tossing a coin : H / T (Simple outcome) 2) tossing two coins and getting first ‘Head’ : {HH,HT} (Compound outcome) • Exhaustive Events: The all possible outcomes of a random experiment is known as the exhaustive events. e.g. tossing a coin : {H, T}

7. Favorable Events: The number of cases favorable to event in a trial is the number of outcomes which entail the happening of the event. e.g. throwing a die, getting even number : {2, 4, 6} • Mutually Exclusive Events: Events are said to be mutually exclusive or incompatible if the happening of any one of the them precludes the happening of all the other. i.e. no two or more of them can happen simultaneously in the same trial e.g. tossing of a coin: H/T throwing a die: 1/2/3/4/5/6

8. Equally Likely Events: Outcomes of trial are said to be equally likely if taking into consideration all the relevant evidences, there is no reason to expect one in preference to the others. • Independent Events: Several events are said to be independent if the happening (or non happening) of an event is not affected by the supplementary knowledge concerning the occurrence of any number of the remaining events. e.g.getting consequent heads in tossing of coin in subsequent throws

9. Mathematical / Classical Probability If a random experiment or a trial results in ‘n’ exhaustive, mutually exclusive and equally likely outcomes, out of which ‘m’ are favorable to the occurrence of event ‘E’, the probability ‘p’ of occurrence of E, usually denoted by P(E), is given by

10. Remarks: • Since m >= 0, n > 0 and m < = n, we get 0<=P(E)<=1 • The non-happening of the event E is called the complementary event of E is denoted by E’. The number of cases favorable to E’ is (n-m). Then the probability ‘q’ that E will not happen is given by q= 1 – (m/n)= 1 – p 3. P(E)=1: Certain Event, P(E)=0: impossible event

11. Limitations of Mathematical Probability: • Equally likely events e.g. 1) passing in examination 2) falling of a fan • The experiment is repeated large number of times

12. Examples: • Draw a number at random from the integers 1 through 10. What is the probability that a prime is drawn? • An urn contains four red balls, three green balls and five white balls. If a single ball is drawn, what is the probability that it is green? • In rolling a fair die, what is the probability of a of obtaining at least a 5? • How many different words are possible using all letters of “dobel”?

13. Permutations and Combinations • Permutations: Given n different things, we may arrange them in a row in any order. Each such arrangement is called permutations of the given things. e.g. We have six permutations of the three letters a,b and c, namely abc, acb, bca, cab, cba, bac.

14. Theorem: The number of permutations of n different things taken all at a time is n!=1.2.3…..n. e.g. If there are 10 different screws in a box that needed in a certain order for assembling a certain product, and theses screws are drawn at random from the box, the probability p of picking them in the desired order is very small, namely p=1/10!

15. Theorem: If n given things can be divided into C classes such that things belonging to the same class are alike while things belonging to different classes are different,, then the number of permutations of these things taken all at a time is n!/(n1!.n2!...nC!), n1+n2+…+nC=n e.g. How many password can be generated from the letters of REARRANGE ? 9!/(3!.2!.2!)=15120

16. Theorem: The number of different permutations of n different things taken k at a time without repetition is nPk=n!/(n-k)! and with repetitions is n^k. e.g. In a coded telegram the letters are arranged in group of five letters, called words, we see that the number of different such word is 26^5 And the number of different such words containing each letter not more than once is, 26!/(26-5)!

17. Combination • In a permutation, the order of the selected thing is essential. In contrast, a combination of given things means any selection of one or more things without regard to order. • Theorem: The number of different combinations of n different things, k at a time, without repetition is nCk=n!/(k!.(n-k)! and the number of those combinations with repetitions is (n+k-1)Ck e.g. A student has 7 books on his desk. How many ways 3 books can be selected with and without replacement? a) without replacement: 7C3 b) with replacement: 9C3

18. Examples: • Among the digits 1,2,3,4,5 at first one is chosen and then a second selection is made among the remaining ones. Assuming that all possible outcomes have equal probabilities, find the probability that odd digits will be selected, i) the first time ii) the second time iii) both times. Solution: No of exhaustive events : 5(first time)*4(second time)=20=n i) (1,2),(1,3)….(5,3),(5,4): 12 cases. p=12/20=3/5 ii) (2,1), (3,1)….(3,5),(4,5):12 cases. p=12/20=3/5 iii) (1,3),(1,5)….(5,3): 6 cases. p=6/20

19. 2. Four cards drawn at random from a pack of 52 cards. Find the probability that • They are a king, a queen, a jack and an ace • Two are kings and two are queens • Two are blacks and two are reds • Two are of hearts and two are of diamonds Solution: No of exhaustive events: 52C4 i) (4C1*4C1*4C1*4C1)/52C4 ii) (4C2*4C2)/52C4 iii) (26C2*26C2)/52C4 iv) (13C2*13C2)/52C4

20. 3. A committee of 4 peoples is to be appointed from 3 officers of the production department, 4 officers for purchase department, 2 officers of the sales department and 1 charted accountant. Find the probability of forming the committee in the following manner: • There must be one from each category • It should at least one form purchase department • The charted accountant must be in the committee Solution: • (3C1*4C1*2C1*1C1)/(10C4)=4/35 • 1-(6C4/10C4)=1-1/14=13/14 • (1C1*9C3)/10C4=2/5

21. 4. In a random arrangement of letter of the word ‘COMMERCE’, find the probabilities that all the vowels are together. Solutions: Exhaustive events: n = 8!/(2!2!2!) Consider ‘OEE’ as one letter to find the probability for the vowels are together, so that total no of letters in ‘COMMERCE’ may be taken as 6. No of favorable cases m = (6! 3!)/(2!2!2!) p= m/n = 3/28

22. 5. Twenty five books are placed at random in the self. Find the probability that a particular pair of books shall be: i) always together ii) never together Solution: No of exhaustive events: n = 25! • Let us now regard that two particular books are tagged together so that we shall regard them as a single book. Thus, now we have 24 books, which can be arranged 24! ways. But the two books which are fastened together can be arranged 2! Ways. m= 2!*24! p=m/n=2/25 ii) 1 – p = 23/25

23. 6. A car is parked among N cars in a row, not at either end. On his return the owner finds that exactly r of the N places are still occupied. What is the probability that both neighboring places are empty? Solution: n = (N-1)C(r-1) m= (N-3)C(r-1) p=m/n = {(N-r)(N-r-1)}/{(N-1)(N-2)}

24. 7. Each coefficients in the equation , is determined by throwing an ordinary die three times. Find the probability that the equation will have real roots. Solution: n = 6*6*6=216 (no of combination for die) p = 43/216

25. 8. Out of (2n+1) tickets consequentively numbered, three are drawn at random. Find the chance that the nos. on them are in A.P. Solution n = (2n+1)C3 To find favorable cases: d=1: (1,2,3)----,(2,3,4),-----,(2n-1,2n,2n+1) : (2n-1) d=2: (1,3,5)----,(2,4,6),----,(2n-3,2n,2n+1 ) : (2n-3) . . d=n-1: (1,n,2n-1), (2,n+1,2n),(3,n+2,2n+1) : 3 d=n: (1,n+1,2n+1):1 m= 1+3+5+…+(2n-1)=(n/2)[1+(2n-1)]=n^2 p=m/n

26. Statistical (or Empirical ) Probability • Definition: If an experiment is performed repeatedly under essentially homogeneous and identical conditions, then the limiting values of the ratio of the number of times the event occurs to the number of trials, as the number of trials becomes indefinitely large, is called the probability of happing of the event, it being assumed that the limit is finite and unique. i.e.

27. e.g. J. E. Kerrich conducted coin tossing experiment with 10 sets of 1,000 tosses each during his confinement in World War II. The number of heads found by him were: 502, 511, 497, 529, 504, 476, 507, 520, 504, 529. This gives the probability of getting a head in a toss of a coin as: P(E)= 5079/10000=0.5079≈1/2.

28. Limitations: • If an experiment is repeated a large number of times, the experimental conditions may not remain identical and homogeneous. • The limit in (definition) may not attain a unique value

29. Axiomatic approach to Probability • This approach closely relets the theory of probability with the set theory. It includes both the classical and the statistical approach as particular cases and overcome the deficiencies of each of them. • Terminology: Random Experiment, Sample Space and Elementary Event

30. Definition: P(A) is the probability function defined on a σ – field B of events if the following properties or axioms hold. • For each A , P(A) >=0 • P(S)=1

31. Theorems: • Addition theorem of Probability If A and B are any two events and are not disjoint then 2. P(Φ) = 0 3. P(A’)=1 - P(A) 4.

32. Examples: 1.

33. 2. An investment consultant predicts that the price of a certain stock will go up during the next week are 1:2 and the price remaining the same are 1:3 what is the probability that the price of the stock will go down during the next week? Solution: A: ‘stock price will go up’ => P(A)=1/3 B: ‘stock price will remain same’ => P(B)=1/4 P(stock price will go down)=1-P(AUB)=1-[P(A)+P(B)]=1-7/12=5/12.

34. 3. An integer is chosen at random from two hundred digits. What is the probability that the integer is divisible by 6 or 8? Solution: Sample space: S = {1,2,3,…,200}=>n(S)=200 A: integer is divisible by 6 = {6,12,…, 198} => n(A)=198/6=33=>P(A)=33/200 B: integer is divisible by 8 = {8,16,…,200} => n(B)=200/8=25=>P(B)=25/200 A∩B: integer is divisible by 6 and 8 = {24,48,…192} => n(A∩B)=192/24=8=>P(A∩B)=8/200 A U B: integer is divisible y 6 or 8 P(A U B)= P(A)+P(B)-P(A∩B)=1/4

35. 4. Three newspapers A,B and C are published in a certain city. It is estimated from a survey that of the adult population: 20% read A, 16% read B, 14% read C, 8% read both A and B, 5% read both A and C, 4% read both B and C, 2% read all three. Find what percentage read at least one of the papers? Solution: P(A)=0.2, P(B) = 0.16, P(C) = 0.14, P(A ∩ B)= 0.08, P(A ∩ C)= 0.5 , P(B∩C)=0.04, P(A∩B∩C)=0.02. P(AUBUC)= P(A)+P(B)+P(C)-P(A∩B)-P(A∩C)-P(B ∩C)+P(A ∩B ∩C)=0.35 Answer: 35%

36. Conditional Probability However, quite often, while evaluating probability of some event, we already have some information stemming from the experiments. For example, if we have prior information that the outcome of the random experiment must be in a set B of S, then this information must be used to re-appraise that likelihood that the outcome will be also be in B. This re-appraised probability is denoted by P(A/B) and is read as the conditional probability of the event A, given that the event B has already happed. P(A/B)= P(A∩B)/P(B) (conditional probability of happening of A, given that B has already happened) Independent Events: P(A/B)=P(A) => P(A ∩ B) = P(A)*P(B)

37. Examples: • Data on the readership of a certain magazine show that the proportion of ‘male reader under 35 is 0.40 and over 35 is 0.20. If the proportion of readers under 35 is 0.70, find (i) the proportion of subscribers that are ‘female over 35 years’ (ii) calculate the probability that a randomly selected male subscriber is under 35 years of age. Solution: A: Reader of the magazine is male. B: Reader of the magazine is over 35 years of age. P(A∩B)=0.20, P(A∩B’)=0.40, P(B’)=0.70 i) P(A’∩B)=P(B)-P(A∩B)=0.10 ii) P(B’/A)=P(B’∩A)/P(A)=0.4/0.6=2/3

38. 2. A bag contains 17 counters marked with the no. 1 to 17. A counter is drawn and replaced; a second drawing is then made. What is the probability that the first number drawn is even and the second odd? How will your results be effected if the first counter drawn is not replaced? Solution: A: getting even numbered counter, P(A)= 8/17 B: getting odd numbered counter, P(B)=9/17 First even and second odd numbered counter= A ∩ B, P(A∩B)=P(A)*P(B)=72/289, (A and B are independent) However, if the first counter is not replaced then, P(A∩B)=P(A)*P(B/A)=(8/17)*(9/16)=9/34.

39. 3. A consignment of 15 record players contains 4 defectives. The record players are selected at random one by one, and examined. Those examined are not put back. What is the probability that the 9th one examined is the last defective? Solution: A: 3 defective in examination of 8 record, P(A)=(4C3*11C5)/15C8 B/A: the 9th piece examined is a defective given that there 3 defective in first 8 examination, P(B/A)=1C1/7C1=1/7 Hence, P(A∩B)= P(A)*P(B/A)=8/195

40. 4. In the following figure, assume that the probability of a relay being closed is ‘p’ and that a relay open or closed independently of any other. Find the probability that current from L to R.

41. Solution: Ai = the relay i (i = 1,2,3,4,5) is closed. E = Current will flow from L to R. In the figure, the current will flow from L to R if at least one the circuit from L to R is closed. That is i) A1∩A2=B1 ii) A4∩A5=B2 iii)A1∩A3∩A5=B3 iv) A4∩A3∩A2=B4 P(B1 U B2 U B3 U B4)=?, P(B1)=P(A1)P(A2)=p*p=P(B2), P(B3)=p*p*p=P(B4) P(B1∩B2)=P(A1∩A2∩A3∩A4)=p*p*p*p,… p= 2p^2+2p^3-5p^4+2p^5

42. Bayes’ Theorem Prior Probability---New Information---Bayes’ theorem----Posterior Probability Theorem: If E1, E2,…,En are mutually disjoint events with P(Ei)>0, then for any arbitrary event A which is a subset of such that P(A)>0, we have

43. Examples: • A letter is known to have come either from TATANAGAR or from CALCUTTA. On the envelope just two consecutive letters TA are visible. What is the probability that the letter came from CALCUTTA? Solution: E1: letter came from TATANAGAR, P(E1)=1/2 E2: letter came from CALCUTTA, P(E2)=1/2 A: two consecutive visible letter on the envelope are ‘TA’ P(A/E1) = 2/8, P(A/E2) = 1/7 The probability that the letter came from CALCUTTA,

44. 2. From a vessel containing 3 white and 5 black balls, 4 balls are transferred into an empty vessel. From this vessel a ball is drawn and is fond to be white. What is the probability that out of four balls transferred 3 are white and 1 is black? Solution: E1: Transfer of 0 white and 4 black balls, P(E1)=5C4/8C4 E2: 1 3 , P(E2)= (3C1*5C3)/8C4 E3: 2 2 , P(E3)=(3C2*5C2)/8C4 E4: 3 1 ,P(E4)=(3C3*5C1)/8C4 E: Drawing of a white ball from the second vessel, P(E/E1)=0,P(E/E2)=1/4,P(E/E3)=2/4, P(E/E4)=3/4 P(E4/E)=1/7

45. 3.A speaks truth 4 out of 5 times. A die is tossed. He reports that there is a six. What is the chance that actually there was six? Solution: E1: A speaks truth, E2: A tells a lie, A: A reports six P(E1)= 4/5, P(E2)=1/5, P(A/E1)=1/6, P(A/E2)=5/6 P(E1/A)= 4/9

46. 4. A and B are two weak students of statistics and their chances of solving a problem in statistics correctly are 1/6 and 1/8 respectively. If the probability of their making a common error is 1/525 and they obtain the same answer, find the probability that their answer is correct. Solution: E1: Both A and B solve the problem correctly E2: Exactly one of the solves the problem correctly E3: Neither of them solves the problem correctly E: They get the same answer P(E1)=(1/6)*(1/8)=1/48, P(E/E1)=1; P(E2)=[(1/6)(7/8)+(5/6)(1/8)]=1/4, P(E/E2)=0; P(E3)=(5/6)(7/8)=35/38 P(E/E3)=1/525. P(E1/E)=15/16

47. Markov Chain/Markov Process It is a sequence of experiments with the following properties • An experiment has a finite number of discrete outcomes, called states. The process, or experiment, is always in one of these states. • With each additional trial the experiment can move from its present state to any other state or remain in the same state. • The probability of going from one state to a other on the next state depends only on the present state and not on past states. • The Probability of moving from any one state to another in one step is presented in a transition matrix • The transition matrix is square, since all possible states are used for rows and columns • Each entry is between o and 1, inclusive • The entries in each row add to 1. • The state matrix times the transition matrix give the next-state matrix.

48. e.g. A rental firm has three locations. A truck rented at one location may be returned to any of the locations. The company’s records show the probability of a truck rental at one location being returned to another. From these records the transition matrix is formed as follows: Rented to I II III I Rented from II III

49. Steady State: A state matrix X = [p1, p2, …,pn] is a steady state or equilibrium matrix for a transition matrix T if XT = X. • Regular Transition Matrix: A transition matrix T of a Markov process is called regular if some power of T has only positive entries.

50. Examples: • Initial-state and transition matrix are given as bellow. Find the following two next-state matrices. Solution: