Introduction to Basic Probability Concepts and Methods

1. Lecture 5. Basic Probability David R. Merrell 90-786 Intermediate Empirical Methods For Public Policy and Management

2. I think that the team that wins game five will win the series...Unless we lose game five. -- Charles Barkley

3. Regularity: Empirical Rule contains 68% of data contains 95% of data contains 99.9% of data

4. How to Verify? • Try Monte Carlo simulations • Easy to use Minitab • Let’s do that!

5. Terminology • Probability trial: a process giving observations with uncertain values • Repeated probability trials: independently repeated under the same conditions • Outcome: a most basic happening • Event: set of outcomes

6. Assignment of Probabilities 1. Symmetry--Classical 2. Relative Frequency 3. Betting Odds--Subjective

7. Classical Approach • Elementary outcomes are equally likely • Probability is defined to be the proportion of times that an event can theoretically be expected to occur • Used in standard games of chance • We can determine the probability of an event occurring without any experiments or trials ever taking place

8. Example 1 - Rolling a die • Experiment: Roll a die • Sample space: S = {1, 2, 3, 4, 5, 6} • Number of possible outcomes: 6 • P(4) = 1/6 • P(even) = 3/6 • P(number < 3) = 2/6

9. Example 2 - Flipping a coin • Experiment: Flip 2 coins • Sample space: S = {HH, TH, HT, TT} • Number of possible outcomes: 4 • P(both heads) = 1/4 • P(at least one tail) = 3/4

10. Example 3 - Drawing a card • Experiment: Draw a card from a deck of 52 • Number of possible outcomes: 52 • P(ace) = 4/52 • P(diamond) = 13/52 • P(red and ace) = 2/52

11. Relative Frequency Approach • Used when classical approach is not applicable and repeated probability trials are possible • Probability is the proportion of times an event is observed to occur in a large number of trials

12. Example 4--Relative Frequencies • In 1985, 22.9% of whites were below the poverty level • In 1977, the percent urban in Iraq was 64. • In 1984, the divorce rate in Maine was 3.6 per 1000 population. (Problems here!)

13. Law of Large Numbers

14. “Law of Small Numbers” • Toss a coin 1000 times and it will show up heads 500 times???

15. “Law of Averages” • “I’ve lost money every time I bought a stock...I’m due!”

16. Subjective Approach • Used when repeated probability trials are not feasible. • Probability is subjective--an educated guess, a personal assessment

17. Well-Calibrated Probability Forecaster • Link subjective probability to repeated probability trials • P(MSFT goes up tomorrow) = .55 • Does it go up 55% of the time?

18. Example 5--Subjective Probability • What is the probability that the Pittsburgh Steelers will win next week? • What is the probability that Al Gore will be elected president in the year 2000?

19. Odds vs. Probabilities • Odds are a restatement of probability • If the probability that an event will occur is 3/5, then the odds in favor of the event occurring are 3:2 • Odds against an event occurring are the reverse of odds in favor of occurring. In this case 2:3. • To calculate the probability, given the odds 1:3 1 1 probability is 1/4 1 + 3 4

20. Odds Odds of a:b in favor of an event A Bet in Favor Bet Against b -b A Occurs A Does Not -a a

21. Probability Notation • P(A) - probability that event A occurs • P(A’) - probability that event A will not occur (A’ is the complement of A) • P(A B) - probability that A will occur or B will occur or both (Union of A and B) • P(A  B) - probability that A and B will occur simultaneously (Joint probability of A and B) • P(A | B) - probability of A, given that B is known to have occurred. (Conditional probability)

22. Probability Axioms 1. P(A) > 0 2. P(S) = 1 3. Ai mutually exclusive,

23. Addition Law for Probability P(A or B) = P(A) + P(B) - P(A and B) Example: A left engine functions B right engine functions

24. “Proof by Paint” A B 1 1 0 “paint and scrape” A B 2 1 1 1 2

25. If Mutually Exclusive ... P(A or B) = P(A) + P(B) Note simplification of Addition Rule

26. If Independent ... P(A and B) = P(A)P(B) Note simplification of Multiplication Rule

27. Some Connections ... Logic Set Arithmetic Simplification and x independence or + mutually exclusive Note: independence is NOT mutual exclusivity

28. Multiplication Law for Probability P(A and B) = P(A B) = P(A)P(B|A) = P(A|B)P(B) Example Sell cocaine and go to jail A B

29. Example 6--Probability Calculations P(adult male is a Democrat) = 0.6, P(belongs to a labor union) = 0.5 P(Democrat and labor union) = 0.35, Find the probability that an adult male chosen at random: • is a Democrat or belongs to a labor union • does not belong to a labor union • is a Democrat given that he belongs to a labor union

30. Conditional Probability Events A, B P(A and B) = P(B |A)P(A) = P(A|B)P(B) Definition:

31. Example 7--Conditional Probability

32. Contingency Table • Help determine probabilities when we have two variables • Joint and conditional probabilities are in the cells • Marginal probabilities are on the “margins” of the table

33. Educational Achievement: Coding of Ordinal Variable • 1 if grade 4 or less • 2 if grades 5-7 • 3 if grade 8 • 4 if high school incomplete (9-11) • 5 if high school graduate (12) • 6 if technical, trade, or business after high school • 7 if college/ university incomplete • 8 if college/university graduate or more

34. Educational Achievement Table

35. Count--Absolute Frequency

36. Joint Probability

37. Marginal Probability

38. Conditional Probabilities: P(Ed =4|F) P(F|Ed=4)

39. Conditional Probabilities Marginal Probability Joint Probability Absolute Frequencies

40. Example 8--More Probability Calculations Find the probability that the individual: • is a high school graduate • is female • is male or has incomplete high school • is female and did not complete college • graduated from college given that he is a male • is male given that he graduated from college

41. Next Time ... • Bayes Rule • Total Probability Rule • Applications