1 / 40

Multiple Discrete Random Variables

Multiple Discrete Random Variables. Introduction. Consider the choice of a student at random from a population of college students. We wish to know his/her height, weight, blood pressure, pulse rate, etc.

farrah
Download Presentation

Multiple Discrete Random Variables

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Multiple Discrete Random Variables

  2. Introduction • Consider the choice of a student at random from a population of college students. We wish to know his/her height, weight, blood pressure, pulse rate, etc. • The mapping from sample space of students to measurements of height and weight, would be H(si) = hi, W(si) = wi, of the student selected. • The table is a two-dimensional array that lists the probability P[H = hi and W =wj].

  3. Introduction • The information can also be displayed in a three-dimensional format. • These probabilities were termed join probabilities. The height and weight could be represented as 2 x 1 random vector. • We will study dependencies between the multiple RV. For example: “Can we predict a person’s height from his weight?”

  4. Jointly Distributed RVs • Consider two discrete RVs X and Y. They represent the functions that map an outcome of an experiment si to a value in the plane. for all • The experiment consists of the simultaneous tossing of a penny and a nickel. • Two random variable that are defined on the same sample space S are said to be jointly distributed.

  5. Jointly Distributed RVs • There are four vectors that comprise the sample space • The values of the random vector (multiple random variables) are denoted either by (x,y) which is an ordered pair/point in the plane or [x y]T a 2D vector. • The size of the sample space for discrete RV can be finite of countably infinite. • If X can take on 2 values Nx = 2, and Y can take on 2 values NY =2, the total number of elements in SX,Y is NXNY = 4. • Generally, if SX = {x1, x2,…,xNx} and SY = {y1, y2,…,yNy}, then the random vector can take on values in

  6. Jointly Distributed RVs • The notation A × B, where A and B are sets, denotes a Cartesian product set. • The joint PMF (bivariate PMF) as

  7. Properties of joint PMF • Property 1. Range of values of joint PMF • Property 2. Sum of values of joint PMF Similarly for a countably infinite sample space. • For two fair coins that do not interact as they are tossed we might assign pX,Y[i,j] = ¼.

  8. The procedure to determine the joint PMF from the probabilities defined on S • The procedure depends on whether the RV mapping is one-to-one or many-to-one. • For a one-to-one mapping from S to SX,Y we have It is assumed that sk is the only solution to X(s) = xi and X(s) = yj. • For a many-to-one transformation the joint PMF is found as

  9. Two dice toss with different colored dice • A red die and a blue die are tossed. The die that yields the larger number of dots is chosen. • If both dice display the same number of dots, the red die is chosen. The numerical outcome of the experiment is defined to be 0 if the blue die is chosen and 1 if the red die is chosen, along with its corresponding number of dots. • What is pX,Y[1,3] for example?

  10. Two dice toss with different colored dice • To determine the desired value of the PMF,we assume that each outcome in S is equally likely and therefore is equal to 1/36. • Since there are three outcomes that map into (1,3). • In general, we can use the joint PMF, to find probability of event A defined on SX,Y = SX × SY.

  11. Marginal PMFs and CDFs • If pX,Y[x,y] is known, then marginal probabilitiespX[xi] and pY[yi] can be determined. • Consider the general case find calculating the probability of an event of interest A oncountably infinite sample space . • Let A = {xk} × SY. Then, with i= k only with j = k only

  12. Example: Two coin toss • A penny (RV X) and a nickel (RV Y) are tossed and the outcomes are mapped into a 1 for a head and a 0 for a tail. Consider the joint PMF • The marginal PMFs are given as =1 =1

  13. Joint PMF cannot be determined from marginal PMFs • It is not possible in general to obtain joint PMF from marginal PMFs. • Consider the following joint PMF The marginal PMFs are the same as the ones before. • There are an infinite number of joint PMFs that have the same marginal PMFs. joint PMF  marginal PMFs marginal PMFs  joint PMF

  14. Joint cumulative distribution function • A joint cumulative distribution function (CDF) can be defined for a random vector as • and can be found explicitly by summing the joint PMFs as • The PMF can be recovered as

  15. Properties of Cumulative distribution functions • The marginal CDFs can be easily found from the joint CDF as • Property 1. Range of values • Property 2. Values of “endpoints” • Property 3. Monotonically increasing Monotonically increases as x and/or y increases. • Property 4. “Right” continuous • The joint CDF takes the value after the jump.

  16. Independence of Multiple RV • Consider the experiment of tossing a coin and then a die. • The outcome of the coin X = {0,1} and the outcome of a die Y = {1,2,3,4,5,6} are independent, hence the probability of the random vector (X,Y) taking on a value Y = yi does not depend on X = xi. • X and Y are independent random variables if all the joint events on SX,Y are independent. • The probability of joint events may be reduced to probabilities of “marginal events”. • If A = {xi} and B = {yj}, then and

  17. Independence of Multiple RV • If the joint PMF factors, then X and Y are independent. • To prove it, assume the joint PMF factors, the for all A and B • Example: Two coin toss – independence • Assume we toss a penny and nickel. If all outcomes are equivalently the joint PMF is given by marginal probability

  18. Independence of Multiple RV • Example: Two coin toss – dependence • Consider the same experiment but with a joint PMF given by Then pX,Y[0,0] = 1/8 ≠ (1/4)(3/8) = pX[0]pY[0] and hence X and Y cannot be independent. • If two random variables are not independent, they are said to be dependent.

  19. Independence of Multiple RV • Example: Two coin toss – dependent but fair coins • Consider the same experiment again but with joint PMF given by • Since pX,Y[0,0] = 3/8 ≠(1/2)(1/2), X and Y are dependent. However, by examining the marginal PMFs we see that the coins are in some sense fair since P[heads] = 1/2, thus we might conclude that the RVs were independent. This is incorrect. • If the RVs are independent, the joint CDF factors as well.

  20. Transformations of Multiple Random Variables • The PMF of Y = g(x) if the PMF of X is known is given by • In the case of two discrete RVs X and Y that are transformed into W = g(X, Y) and Z = h(X, Y), we have • Sometimes we wish to determine the PMF of Z = h(X, Y) only. Then we can use auxiliary RV W = X, so thatpZ is the marginal PMF and can be found form the formula above as

  21. Example: Independent Poisson RVs • Assume that the joint PMF is give as the product of the marginal PMFs, and each PMF is Poisson PMF. • Consider the transformation • We need to determine all (k,l) so that • But xk, yl and wi, zj can be replaced by k,l and i,j each with 0,1,….

  22. Example: Independent Poisson RVs • Apply the given transformation we get • Solving for (k, l) for the given (i, j), we have • We must have l ≥ 0 so that l = j – I ≥ 0. discrete unit step

  23. Use the discrete unit step sequence to avoid mistakes • The discrete unit step sequence was introduced to designate the region of w-z plane over which pW,Z[i,j] is nonzero. • The transformation will generally change the region over which the new joint PMF is nonzero. • A common mistake is to disregard this region and assert that the joint PMG is nonzero over i = 0,1,…; j = 0,1,…. • To avoid possible errors unit steps are applied

  24. Example: Independent Poisson RVs • To find the PMF of Z = X + Y from the joint PMF obtained earlier we set W = Xso we have SW = SX = {0,1,…} and • Since u[i] = 1 for I = 0,1,… and u[j - i] = 1 for i = 0,1,…,j and u[j - i] = 0 for I > j, we drop u[i]u[j – i] multipliers. • Not that Z can take on values j = 0,1,… since Z = X + Y.

  25. Connection to characteristic function • Generally the formula for the PMF of the sum of any two discrete RV X and Y, dependent or independent is given by • If the RV are independent, then since the joint PMF must factor, we have the result • This summation is a discrete convolution. Taking the Fourier transformation (defined with a +j) of both sides produces 

  26. Example: Independent Poisson RVs using CF approach • We showed that if X ~ Pois(λ), then • Thus using the above Fourier property we have • But the CF in the braces is that of a Poisson RV and corresponds to • The use of CF for determination of PMF for a sum of independent RV has considerably simplified the derivation. • In summary, if X and Y are independent RV with integer values, then the PMF of Z = X + Y is given by

  27. Transformation of a fine sample space • It is possible to obtain the PMF of Z = g(X,Y) by a direct calculation if the sample SX,Yis finite. • To first obtain the transformed joint PMFpw,z we • determine the finite sample space SZ. • determine which sample points (xi,yj) in SX,Y map into each • sum the probabilities of those (xi,yj) sample points to yield pz[zk]. • Mathematically this is equivalent to

  28. Direct computation of PMF for transformed RV • Consider the transformation of the RV (X,Y) into the scalar RV Z = X2+ Y2. The joint PMF is given by • To find the PMF for Z first note that (X,Y) takes on the values (i,j) = (0,0),(1,0),(0,1),(1,1). Therefore, Z must take on the values zk = i2 + j2 = 0,1,2. Then

  29. Expected Values • If Z = g(X, Y), then by definition its expected value Or using a more direct approach EX. Expected value of a sum of random variables: Z = g(X,Y) = X + Y

  30. Expected value of a product of RV • If Z = g(X, Y) = XY, then • If X and Y are independent, then since the joint PMF factors, we have • More generally,

  31. Variance of a sum of RVs • Consider the calculation of var(X + Y). Then, letting Z = g(X,Y)= (X + Y – EX,Y[X + Y])2, we have • The last term is called the covariance and defined as • or alternatively

  32. Joint moments • The questions of interest • If the outcomes of one RV is a given value, what can we say about the outcome of the other RV? • Will it be about the same or have the same have no relationship to other RV? There is clearly a relationship between height and weight.

  33. Joint moments • To quantify these relationships we form the product XY, which can take on the values +1, -1, and ±1 for the joint PMF above. • To determine the value of XY on the average we define the joint moment as EX,Y[XY]. For the case (a)

  34. Joint moments • In previous example EX[X] = EY[Y] = 0. If they are not zero, the joint moment will depend on the values of the means. • To nullify this effect it is convenient to use the joint central moments. That will produce the desired +1 for the joint PMF above.

  35. Independence implies zero covariance but zero covariance does not imply independence • Consider the joint PMF which assigns equal probability ½ to each of the four points point. • For the joint PMF the covariance is zero since and thus • However, X and Y are dependent because pX,Y[1,0] = 1/4 but pX[1]pY[0]=(1/4)(1/2) = 1/8.

  36. The joint k-lth moment More generally the joint k-lth moment is defined as For k = 1,2,…; l=1,2,…, when it exists. The joint k-lth moment central moment is defined as For k = 1,2,…; l =1,2,…, when it exists.

  37. Prediction of a RV outcome • The covariance between two RVs is useful for predicting Y based on knowledge of the outcome of X. • We seek a predictor Y that is linear in X or • The constants a and b are to be chose so that “on the average” the observed value of aX + bis close to the observed value of Y. • The solution is given by

  38. Prediction of a RV outcome • The the optimal linear prediction of Y given the outcome X = x is • Example: Predicting one RV outcome from knowledge of second RV outcome. We found from marginals Regression line

  39. Practice problems • Two coins are tossed in succession with a head being mapped into a +1 and a tail mapped into a -1. If a RV is defined as (X,Y) with X representing the mapping of the first toss and Y representing the mapping of the second toss, draw the mapping. Also, what is SX,Y? (Hint: see slides 4,5). • Two dice are tossed. The number of dots observed on the dice are added together to form the random variable X and also difference to form Y. Determine the possible outcomes of the random vector (X,Y)and plot them I the plane. How many possible outcomes are there? • Is a valid joint PMF? • . Find the marginal probability

  40. Homework 1 2) The values of a joint PMF are given below. Determine the marginal PMFs 3)

More Related