Topic 4: Discrete Random Variables and Probability Distributions

1. Topic 4: Discrete Random Variables and Probability Distributions CEE 11 Spring 2002 Dr. Amelia Regan These notes draw liberally from the class text, Probability and Statistics for Engineering and the Sciences by Jay L. Devore, Duxbury 1995 (4th edition)

2. Definition • For a given sample space S of some experiment, A random variable is any rule that associates a number with each outcome in S • Random variables may take on finite or infinite values • Examples: {0,1}, the number of sequential tosses of a coin in which the outcome “head” is observed • A set is discrete either if it consists of a finite number of elements or if its elements may be listed in sequence so that there is a first element, a second element, a third element, and so on, in the list • A random variable is said to be discrete if its set of possible values is a discrete set.

3. Definition • The probability distribution or probability mass function (pmf) of a discrete random variable is defined for every number x by • The cumulative distribution function (cdf) F(x) of a discrete rv X with pmf p(X) is defined for every number x by

4. Examples

5. Definition • Let X be a discrete rv with set of possible values D and pmf p(x). The expected value or mean value of X, denoted E(x) or mx is given by: • If the rv X has a set of possible values D and pmf p(x), then the expected value of any function h(X), denoted by E[h(x)] or mh(x) is computed by: • Note: according to Ross(1988) p. 255 this is known as the law of the unconscious statistician

6. Example • Let X be a discrete rv with the following pmf and corresponding cdf • Now let h(x) = x2+2

7. Class exercise • Let X be a discrete rv with the following pmf • Calculate the cdf of X • Calculate the E(X) • Now let h(x) = 3x3-100 -- Calculate h(x)

8. Definition • For linear functions of x we have simple rules

9. Variance of a random variable • If X is a random variable with mean m, then the variance of X denoted by Var(X) is defined by • Recall that we previously defined the variance of a population as the average of the squared deviations from the mean. The expected value is nothing other than the average or mean so this form corresponds exactly to the one we used earlier.

10. Variance of a random variable • Its often convenient to use the a different form of the variance which, applying the rules of expected value which we just learned and remembering the E[X] = m, we derive in the following way.

11. The Bernoulli distribution • Any random variable whose only two possible values are 0 and 1 is called a Bernoulli random variable. • Example: Suppose a set of buildings are examined in a western city for compliance with new stricter earthquake engineering specifications. After 25% of the cities buildings are examined at random and 12% are found to be out of code while 88% are found to conform to the new specifications it is supposed that buildings in the region have a 12% likely hood of being out of code. • Let X = 1 if the next randomly selected building is within code and X = 0 otherwise. • The distribution of buildings in and out of code is a Bernoulli random variable with parameters p= 0.88 and 0.12.

12. The Bernoulli distribution • We write this as follows

13. The Bernoulli distribution • In general form, a is the parameter for the Bernoulli distribution but we usually refer to this parameter as p, the probability of success • The mean of the Bernoulli distribution with parameter p is • The variance of the Bernoulli distribution with parameter p is

14. The Binomial distribution • Now consider a random variable that is made up of successive (independent) Bernoulli trials and define the random variable X as the number of successes among n trials. • The Binomial distribution has the following probability mass function • Remembering what we learned about combinations this makes intuitive sense. The binomial coefficient represents the number of ways to distribution the x successes among n trials. px represents the probability that we have x successes in the n trials, while (1-p)(n-x) represents the probability that we have n-x failures in the n trials.

15. The Binomial distribution • Computing the mean and the variance of the binomial distribution is straightforward. First remember that the binomial random variable is the sum of the number of successes in n consecutive Bernoulli trials. Therefore

16. The Binomial distribution

17. The Binomial distribution

18. The Binomial distribution

19. Class exercise • A software engineer has historically delivered completed code to clients on schedule, 40% of the time. If her performance record continues, the probability of the number of on schedule completions in the next 6 jobs can be described by the binomial distribution. • Calculate the probability that exactly four jobs will be completed on schedule • Calculate the probability that at most 5 jobs will be completed on schedule • Calculate the probability that at least two jobs will be completed on schedule • Calculate the probability that at most 5 jobs will be completed on schedule

20. The Hypergeometric distribution • The binomial distribution is made up of independent trials in which the probability of success does not change. • Another distribution in which the random variable X represents the number of successes among n trials is the hypergeometric distribution. • The hypergeometric distribution assumes a fixed population in which the proportion or number of successes is known. • We think of the hypergeometric distribution as involving trials without replacement and the binomial distribution involving trials with replacement. • The classic illustration of the differences between the hypergeometric and the binomial distribution is that of black and white balls in a urn. Assume the proportion of black balls is p. The distribution of the number of black balls selected in n trials is binomial(x;n;p) if put the balls back in the urn after selection and hypergeometric(x;n;M,N) if we set them aside after selection. (engineering examples to come)

21. The Hypergeometric distribution • The probability mass function of the hypergeometric distribution is given by the following, where M is the number of possible successes in the population, N is the total size of the population, x is number of successes in the n trials.

22. The Hypergeometric distribution • Example: Suppose out of 100 bridges in a region, 30 have been recently retrofitted to be more secure during earthquakes. Ten bridges are selected randomly from the 100 for inspection. What is the probability that at least three of these bridges will be retrofitted?

23. The Hypergeometric distribution • The mean and variance of the hypergeometric distribution is given below:

24. The Hypergeometric and the Binomial distributions • Note the following: • Sometimes we refer to M/N, the proportion of the population with a particular characteristic as p. • This is very close to E[X] = np and Var(x) = pn(1-p) which are the mean and variance of the Binomial distribution. In fact we can think of • as a correction term which accounts for the fact that in the hypergeometric distribution we sample without replacement. • Question: Should the variance of the Hypergeometric distribution with proportion p be greater than or less than the variance of the Binomial with parameter p? Can you give some intuitive reason why this is so?

25. The Hypergeometric and the Binomial distributions • Example: Binomial, p = 0.40, three trials vs. • Hypergeometric with M/N = 2/5 = 0.40, three trials

26. The Hypergeometric and the Binomial distributions • If the ratio of the n, the number of trials is small and N, the number in the population is large then we can approximate the hypergeometric distribution with the binomial distribution in which p = M/N. • This used to be very important. As recently as five years ago computers and hand calculators could not calculate large factorials. The calculator I use can not calculate 70!. A few years ago 50! was out of the question. • Despite improvements in calculators its still important to know that if the ratio of n to N is less than 5% (we are sampling 5% of the population) then we can approximate the hypergeometric distribution with the binomial. • Check your calculators now -- what is the maximum factorial they can handle?

27. Class exercise • The system administrator in charge of a campus computer lab has identified 9 machines out of 80 with defective motherboards. • While he is on vacation the lab is moved to a different room, in order to make room for graduate student offices. • The administrator kept notes about which machines were bad, based on their location in the old lab. During the move the computers were places on their new desks in a random fashion so all of the machines must be checked again. • If the administrator checks three of the machines for defects, what is the probability that one of the three will be defective? • Calculate using both the hypergeometric distribution and the binomial approximation to the hypergeometric.

28. The Geometric distribution • The geometric distribution refers to the random variable represented by the number of consecutive Bernoulli trials until a success is achieved. • Suppose that independent trials, each having probability p, • 0 < p < 1, of being a success are performed until a success occurs. If we let X equal the number of trials prior to the success, then • The above definition is the one used in our textbook. A more common definition is the following: Let X equal the number of trials including the last trial, which by definition is a success. Then we get the following:

29. The geometric distribution • The expected value and variance of the geometric distribution is given for the first form by: • For the second form, the expected value has more intuitive appeal. Can you convince yourself that the value is correct? • Please note that the variance is the same in both cases. • Explain why this is so.

30. The geometric distribution • We can derive E(X) in the following way:

31. The geometric distribution These steps are so that we can work with the geometric series 1+x+x2+x3+…=1/(1-x) so p(1+q+q2+…)=p(1/1-q) Here we just substitute p for 1-q

32. The geometric distribution Try deriving the variance of the geometric distribution by finding E(X2)

33. Poisson Distribution • One of the most useful distributions for many branches of engineering is the Poisson Distribution • The Poisson distribution is often used to model the number of occurrences during a given time interval or within a specified region. • The time interval involved can have a variety of lengths, e.g., a second, minute, hour, day, year, and multiples thereof. • Poisson processes may be temporal or spatial. The region in question can also be a line segment, an area, a volume, or some n-dimensional space. • Poisson processes or experiments have the following characteristics:

34. Poisson Distribution • 1. The number of outcomes occurring in any given time interval or region is independent of the number of outcomes occurring in any other disjoint time interval or region. • 2. The probability of a single outcome occurring in a very short time interval or very small region is proportional to the length of the time interval or the size of the region. This value is not affected by the number of outcomes occurring outside this particular time interval or region. • 3. The probability of more than one outcome occurring in a very short time interval or very small region is negligible. • Taken together, the first two characteristics are known as the “memoryless” property of Poisson processes. • Transportation engineers often assume that the number of vehicles passing by a particular point on a road is approximately Poisson distributed. • Do you think that this model is more appropriate for a rural hiqhway or a city street?

35. Poisson Distribution • The pdf of the Poisson distribution is the following: • The parameter l is equal to a t, where a is the intensity of the process the average number of events per time unit and t is the number of time units in question. In a spatial Poisson process a represents the average number of events per unit of space and t represents the number of spatial units in question. • For example, the number of vehicles crossing a bridge in a rural area might be modeled as a Poisson process. If the average number of vehicles per hour, during the hours of 10:00 AM to 3:00 PM is 20 we might be interested in the probability that fewer than three vehicles cross on from 12:30 to 12:45 PM. In this case l = (20 per hour)(0.25hours) = 5. • The expected value and the variance of the Poisson distribution are identical and are equal to l.

36. Class exercise • An urban planner believes that the number of gas stations in an urban area is approximately Poisson distributed with parameter a = 3 per square mile. Lets assume she is correct in her assumption. • Calculate the expected number of gas stations in a four square mile region of the urban area as well as the variance of this number. • Calculate the probability that this region of four square miles has less than six gas stations. • Calculate the probability that in four adjacent regions of one square mile each, that at least two of the four regions contains more than three gas stations. • Do you think the situation is accurately modeled by a Poisson process? • Why or why not?

37. Some random variables that typically obey the Poisson probability law (Ross, p.130) • The number of misprints on a page (or group of pages) of a book • The number of people in a community living to 100 years of age • The number of wrong telephone numbers that are dialed in a day • The number of packages of dog biscuits sold in a particular store each day • The number of customers entering a post office (bank, store) in a give time period • The number of vacancies occurring during a year in the supreme court • The number of particles discharged in a fixed period of time from some radioactive material • WHAT ABOUT ENGINEERING? • Poisson processes are the heart of queueing theory --which is one of the most important topics in transportation and logistics. Lots of other applications too -- water, structures, geotech etc.

38. The Poisson Distribution as an approximation to the Binomial • When n is large and p is small, the Poisson distribution with parameter np is a very good approximation to the binomial (the number of successes in n independent trials when the probability of success is equal to p for each trial). • Example --Suppose that the probability that an idem produced by a certain machine will be defective is 0.10. Find the probability that a sample of 10 items will contain at most one flaw. np = 0.10*10=1.0

39. References • Ross, S. (1988), A first course in probability, Macmillian Press.