Probability Theory: Rules, Models, and Applications

1. Appendix II – Probability Theory Refresher Leonard Kleinrock, Queueing Systems, Vol I: Theory Nelson Fonseca, State University of Campinas, Brazil

2. Appendix II – Probability Theory Refresher

3. Random event: statistical regularity • Example: If one were to toss a fair coin four times, one expects on the average two heads and two tails.There is one chance in sixteen that no heads will occur. If we tossed the coin a million times, the odds are better than 10 to 1 that at least 490.000 heads will occur. 88

4. II.1 Rules of the game • Real-world experiments: • A set of possible experimental outcomes • A grouping of these outcomes into classes called results • The relative frequency of these classes in many independent trials of the experiment Frequency = number of times the experimental outcome falls into that class, divided by number of times the experiment is performed

5. Mathematical model: three quantities of interest that are in one-to-one relation with the three quantities of experimental world • A sample space is a collection of objects that corresponds to the set of mutually exclusive exhaustive outcomes of the model of an experiment. Each object  is in the set S is referred to as a sample point • A family of events  denoted {A, B, C,…}in which each event is a set of samples points { }

6. Venn Diagram • A Venn diagram is a diagram that shows all possible logical relations between a finite collection of different sets

7. Venn Diagram

8. Sample Space

9. A probability measure P which is an assignment (mapping) of the events defined on S into the set of real numbers. The notation is P[A], and have these mapping properties: • For any event A,0 <= P[A] <=1 (II.1) • P[S]=1 (II.2) • If A and B are “mutually exclusive” events then P[A U B]=P[A]+P[B] (II.3)

10. Probability Space

11. Tree Diagram • tree diagram may be used to represent a probability space

12. Notation • Exhaustive set of events: a set of events whose union forms the sample space S • Set of mutually exclusive exhaustive events , which have the properties

13. Mutually exclusive events

14. Mutually exclusively event

15. The triplet (S, , P) along with Axioms (II.2)-(II.3) form a probability system • Conditional probability • The event B forces us to restrict attention from the original sample space S to a new sample space defined by the event B, since B must now have a total probability of unity. We magnify the probabilities associated with conditional events by dividing by the term P[B]

18. Two events A, B are said to be statistically independent if and only if • If A and B are independent • Theorem of total probability If the event B is to occur it must occur in conjunction with exactly one of the mutually exclusive exhaustive events A i

20. Conditional probability

21. The second important form of the theorem of total probability • Instead of calculating the probability of some complex event B, we calculate the occurrence of this event with mutually exclusive events

22. Bayes’ theorem Where {A }are a set of events mutually exclusive and exhaustive • Example: You have just entered a casino and gamble with a twin brother, one is honest and the other not. You know that you lose with probability=½ if you play with the honest brother, and lose with probability=P if you play with the cheating brother i

26. The question is: what is the probability of your being playing with the cheating brother since you lost?

27. II.2 Random variables • Random variable is a variable whose value depends upon the outcome of a random experiment • To each outcome, we associate a real number, which is in fact the value the random variable takes on that outcome • Random variable is a mapping from the points of the sample space into the (real) line

28. Random variable

30. Discrete x Continuous

31. Random variable

33. Discrete x Continuous

34. Random variable

35. S • Example: If we win the game we win $5, if we lose we win -$5 and if we draw we win $0. L (3/8) D (1/4) W (3/8)

36. Probability distribution function (PDF), also known as the cumulative distribution function

37. 1 x -5 +5 0

38. Cumulative Distribution Function

39. Probability density function (pdf) • The pdf integrated over an interval gives the probability that the random variable X lies in that interval

40. Probability Density Function

41. Cumulative - Density

42. -5 0 +5 • Distributed random variable

43. Impulse function (discontinuous) • Functions of more than one variable • “Marginal” density function • Two random variables X and Y are said to be independent if and only if

44. We can define conditional distributions and densities • Function of one random variable • Given the random variable X and its PDF, one should be able to calculate the PDF for the variable Y

45. y y 0

47. II.3 Expectation • Stieltjes integrals deal with discontinuities and impulses Let

48. The Stieltjes integral will always exist and therefore it avoids the issue of impulses • Without impulses the pdf may not exist • When impulses are permitted we have

49. Function of random variable