Brief Review

1. Brief Review Probability and Statistics

2. Probability distributions Continuous distributions

3. Defn (density function) Let x denote a continuous random variable then f(x) is called the density function of x 1) f(x) ≥ 0 2) 3)

4. Defn (Joint density function) Let x = (x1 ,x2 ,x3 , ... , xn) denote a vector of continuous random variables then f(x) = f(x1 ,x2 ,x3 , ... , xn) is called the joint density function of x = (x1 ,x2 ,x3 , ... , xn) if 1) f(x) ≥ 0 2) 3)

5. Note:

6. Defn (Marginal density function) The marginal density of x1 = (x1 ,x2 ,x3 , ... , xp) (p < n) is defined by: f1(x1) = = where x2 = (xp+1 ,xp+2 ,xp+3 , ... , xn) The marginal density of x2 = (xp+1 ,xp+2 ,xp+3 , ... , xn) is defined by: f2(x2) = = where x1 = (x1 ,x2 ,x3 , ... , xp)

7. Defn (Conditional density function) The conditional density of x1 given x2 (defined in previous slide) (p < n) is defined by: f1|2(x1 |x2) = conditional density of x2 given x1 is defined by: f2|1(x2 |x1)=

8. Marginal densities describe how the subvector xi behaves ignoring xj Conditional densities describe how the subvector xi behaves when the subvector xj is held fixed

9. Defn (Independence) The two sub-vectors (x1 and x2) are called independent if: f(x) = f(x1, x2) = f1(x1)f2(x2) = product of marginals or the conditional density of xi given xj : fi|j(xi |xj) = fi(xi) = marginal density of xi

10. Example (p-variate Normal) The random vector x (p× 1) is said to have the p-variate Normal distribution with mean vector m(p× 1) and covariance matrix S(p×p) (written x ~ Np(m,S)) if:

11. Example (bivariate Normal) The random vector is said to have the bivariate Normal distribution with mean vector and covariance matrix

15. Theorem (Transformations) Let x = (x1 ,x2 ,x3 , ... , xn) denote a vector of continuous random variables with joint density function f(x1 ,x2 ,x3 , ... , xn) = f(x). Let y1 =f1(x1 ,x2 ,x3 , ... , xn) y2 =f2(x1 ,x2 ,x3 , ... , xn) ... yn =fn(x1 ,x2 ,x3 , ... , xn) define a 1-1 transformation of x into y.

16. Then the joint density of y is g(y) given by: g(y) = f(x)|J| where = the Jacobian of the transformation

17. Corollary (Linear Transformations) Let x = (x1 ,x2 ,x3 , ... , xn) denote a vector of continuous random variables with joint density function f(x1 ,x2 ,x3 , ... , xn) = f(x). Let y1 = a11x1 + a12x2 + a13x3 , ... + a1nxn y2 = a21x1 + a22x2 + a23x3 , ... + a2nxn ... yn = an1x1 + an2x2 + an3x3 , ... + annxn define a 1-1 transformation of x into y.

18. Then the joint density of y is g(y) given by:

19. Corollary (Linear Transformations for Normal Random variables) Let x = (x1 ,x2 ,x3 , ... , xn) denote a vector of continuous random variables having an n-variate Normal distribution with mean vector m and covariance matrix S. i.e. x ~ Nn(m, S) Let y1 = a11x1 + a12x2 + a13x3 , ... + a1nxn y2 = a21x1 + a22x2 + a23x3 , ... + a2nxn ... yn = an1x1 + an2x2 + an3x3 , ... + annxn define a 1-1 transformation of x into y. Then y = (y1 ,y2 ,y3 , ... , yn) ~ Nn(Am,ASA')

20. Defn (Expectation) Let x = (x1 ,x2 ,x3 , ... , xn) denote a vector of continuous random variables with joint density function f(x) = f(x1 ,x2 ,x3 , ... , xn). Let U = h(x)= h(x1 ,x2 ,x3 , ... , xn) Then

21. Defn (Conditional Expectation) Let x = (x1 ,x2 ,x3 , ... , xn) = (x1 , x2 ) denote a vector of continuous random variables with joint density function f(x) = f(x1 ,x2 ,x3 , ... , xn) = f(x1 , x2 ). Let U = h(x1)= h(x1 ,x2 ,x3 , ... , xp) Then the conditional expectation of U given x2

22. Defn (Variance) Let x = (x1 ,x2 ,x3 , ... , xn) denote a vector of continuous random variables with joint density function f(x) = f(x1 ,x2 ,x3 , ... , xn). Let U = h(x)= h(x1 ,x2 ,x3 , ... , xn) Then

23. Defn (Conditional Variance) Let x = (x1 ,x2 ,x3 , ... , xn) = (x1 , x2 ) denote a vector of continuous random variables with joint density function f(x) = f(x1 ,x2 ,x3 , ... , xn) = f(x1 , x2 ). Let U = h(x1)= h(x1 ,x2 ,x3 , ... , xp) Then the conditional variance of U given x2

24. Defn (Covariance, Correlation) Let x = (x1 ,x2 ,x3 , ... , xn) denote a vector of continuous random variables with joint density function f(x) = f(x1 ,x2 ,x3 , ... , xn). Let U = h(x)= h(x1 ,x2 ,x3 , ... , xn) and V = g(x)=g(x1 ,x2 ,x3 , ... , xn) Then the covariance of U and V.

25. Properties • Expectation • Variance • Covariance • Correlation

26. E[a1x1 + a2x2 + a3x3 + ... + anxn] = a1E[x1] + a2E[x2] + a3E[x3] + ... + anE[xn] or E[a'x] = a'E[x]

27. E[UV] = E[h(x1)g(x2)] = E[U]E[V] = E[h(x1)]E[g(x2)] if x1 and x2 are independent

28. Var[a1x1 + a2x2 + a3x3 + ... + anxn] or Var[a'x] = a′Sa

29. Cov[a1x1 + a2x2 + ... + anxn , b1x1 + b2x2 + ... + bnxn] or Cov[a'x, b'x] = a′Sb

31. Multivariate distributions

32. The Normal distribution

33. Normal distribution with m = 50 and s =15 Normal distribution with m = 70 and s =20 1.The Normal distribution – parameters mands(or s2) Comment:If m = 0 and s = 1 the distribution is called the standard normal distribution

34. The probability density of the normal distribution If a random variable, X, has a normal distribution with mean mand variance s2 then we will write:

35. The multivariate Normal distribution

36. Let = a random vector Let = a vector of constants (the mean vector)

37. Let = a p ×p positive definite matrix

38. Definition The matrix Ais positive semi definite if Further the matrix Ais positive definite if

39. Suppose that the joint density of the random vector The random vector, [x1, x2, … xp]is said to have a p-variate normal distribution with mean vector and covariance matrix S We will write:

40. Example: the Bivariate Normal distribution with and

41. Now and

43. Hence where

44. Note: is constant when is constant. This is true when x1, x2 lie on an ellipse centered at m1, m2 .

46. Surface Plots of the bivariate Normal distribution

47. Contour Plots of the bivariate Normal distribution

48. Scatter Plots of data from the bivariate Normal distribution

49. Trivariate Normal distribution - Contour map x3 mean vector x2 x1

50. Trivariate Normal distribution x3 x2 x1