Understanding Random Variables and Distributions in Statistics

1. Chapter 2 Statistical Background

2. 2.3 Random Variables and Probability Distributions • A variable X is said to be a random variable (rv) if for every real number a there exists a probability P(X ≦ a) that X takes on a value less than or equal to a • Thus P(X = x) is the probability that the random variable X takes the value x. • P( x1≦ X ≦ x2 ) is the probability that the random variable X takes values between x1 and x2, both inclusive.

3. 2.3 Random Variables and Probability Distributions • A formula giving the probabilities for different values of the random variable X is called a probability distribution in the case of discrete random variables. • Probability density function (denoted by p.d.f.) for continuous random variables. This is usually denoted by f(x)

4. 2.3 Random Variables and Probability Distributions • In general, for a continuous random variable, the occurrence of any exact value of X may be regarded as having a zero probability. • Hence probabilities are discussed in terms of some ranges. • These probabilities are obtained by integrating f(x) over the desired range.

5. 2.3 Random Variables and Probability Distributions • For instance, if we want Prob (a≦ X ≦b), this is given by

6. 2.4 The Normal Probability Distribution andRelated Distributions • There are some probability distributions for which the probabilities have been tabulated and which are considered suitable descriptions for a wide variety of phenomena. • These are the normal distribution and the x2, t, and F distributions

7. 2.4 The Normal Probability Distribution andRelated Distributions • There is also a question of whether the normal distribution is an appropriate one to use to describe economic variables. • However, even if the variables are not normally distributed, one can consider transformations of the variables so that the transformed variables are normally distributed.

8. 2.4 The Normal Probability Distribution andRelated Distributions The Normal Distribution (an example) • The normal distribution is a bell-shaped distribution which is used most extensively in statistical applications in a wide variety of fields. • Its probability density function is given by

9. 2.4 The Normal Probability Distribution andRelated Distributions and the correlation between x and x is , then In particular, and

10. 2.4 The Normal Probability Distribution andRelated Distributions ~ Distribution • If x1 , x2 , …, xn are independent normal variable with mean zero and variance 1, that is, , then is said to have the - distribution with degrees of freedom (d.f.) n, and we will write this as .

11. 2.4 The Normal Probability Distribution andRelated Distributions • The subscript n denotes the d.f. • The - distribution is the distribution of the sum of squares of n independent standard normal variables.

12. 2.4 The Normal Probability Distribution andRelated Distributions • If ,then Z should be defined as • The -distribution also has an “additive property,” although it is different from the property of the normal distribution and is much more restrictive.

13. 2.4 The Normal Probability Distribution andRelated Distributions • The property is: If and and and are independent, then

14. 2.4 The Normal Probability Distribution andRelated Distributions t - Distribution • If and and x and y are independent, has a t-distribution with d.f. n. • We write this as . • The subscript n again denotes the d.f.

15. 2.4 The Normal Probability Distribution andRelated Distributions • Thus the t-distribution is the distribution of a standard normal variable divided by the square root of an independent averaged variable ( variable divided by its degrees of freedom). • The t-distribution is a symmetric probability distribution like the normal distribution but is flatter than the normal and has longer tails. • As the d.f n approached infinity, the t-distribution approaches the normal distribution.

16. 2.4 The Normal Probability Distribution andRelated Distributions F ~Distribution • If and and and are independent, has the F-distribution with d.f. n1 and n2. • We write this as

17. 2.4 The Normal Probability Distribution andRelated Distributions F ~Distribution • The first subscript, n1, refers to the d.f. the numerator, and the second subscript, n2 ,refers to the d.f. of the denominator. • The F -distribution is thus the distribution of the ratio of two independent averaged variables.

18. 2.5 Classical Statistical Inference • Statistical inference is the area that describes the procedures by which we use the observed data to draw conclusions about the population from which the data came or about the process by which the data were generated.

19. 2.5 Classical Statistical Inference • Broadly speaking, statistical inference can be classified under two headings: • Classical inference • Bayesian inference.

20. 2.5 Classical Statistical Inference • In Bayesian inference we combine sample information with prior information. • Suppose that we draw a random sample y1 , y2 , …, yn of size n from a normal population with mean and variance (assumed known), and we want to make inferences about .

21. 2.5 Classical Statistical Inference • In classical inference we take the sample mean as our estimate of .Its variance is . • The inverse of this variance is known as the sample precision. Thus the sample precision is .

22. 2.5 Classical Statistical Inference • In Bayesian inference we have prior information on . • This is expressed in term of a probability distribution known as the prior distribution. • Suppose that the prior distribution is normal with mean and variance , that is, precision .

23. 2.5 Classical Statistical Inference • We now combine this with the sample information to obtain what is known as the posterior distribution of . • This distribution can be shown to be normal. • Its mean is a weighted average of the sample.

24. 2.5 Classical Statistical Inference • Mean and the prior mean , weighted by the sample precision and prior precision, respectively. Thus Where • Also, the precision (or inverse of the variance) of the posterior distribution of is , that is, the sum of sample precision and prior precision.

25. 2.5 Classical Statistical Inference • For instance, if the sample mean is 20 with variance 4 and the prior mean is 10 with variance 2, we have • The posterior mean will lie between the sample mean and the prior mean. • The posterior variance will be less than both the sample and prior variance.

26. 2.5 Classical Statistical Inference • 1. Point estimation. • 2. Interval estimation. • 3. Testing of hypotheses.

27. 2.6 Properties of Estimators • There are some desirable properties of estimators that are often mentioned in the book. • These are: • 1. Unbiasedness. • 2. Efficiency. • 3. Consistency. • The first two are small-sample properties. The third is a large-sample property.

28. 2.6 Properties of Estimators Unbiasedness • An estimator g is said to be unbiased for if ,that is, the mean of the sampling distribution of g is equal to . • What this says is that if we calculate g for each sample and repeat this process infinitely many times, the average of all these estimates will be equal to .

29. 2.6 Properties of Estimators • If , then g is said to be biased and we refer to as the bias. • Unbiasedness is a desirable property but not at all costs. • Suppose that we have two estimator g1 and g2 can assume values far away from and yet have its mean equal to ,whereas g2 always ranges close to but has its mean slightly away from .

30. 2.6 Properties of Estimators • Then we might prefer g2 to g1 because it has smaller variance even though it is biased. • If the variance of the estimator is large, we can have some unlucky samples where our estimate is far from the true value. • Thus the second property we want our estimators to have is a small variance. • One criterion that is often suggested is the mean-squared error (MSE), which is defined by

31. 2.6 Properties of Estimators Efficiency • The property of efficiency is concerned with the variance of estimators. • Obviously, it is a relative concept and we have to confine ourselves to a particular class. • If g is an unbiased estimator and it has the minimum variance in the class of unbiased estimators, g is said to be an efficient estimators. • We say that g is an MVUE (a minimum-variance unbiased estimator).

32. 2.6 Properties of Estimators • If we confine ourselves to linear estimators, that is, • where the c’s are constants which we choose so that g is unbiased and has minimum variance, g is called a BLUE (a best linear unbiased estimator).

33. 2.6 Properties of Estimators Consistency • Often it is not possible to find estimators that have desirable small-sample properties such as properties. • In such cases, it is customary to look at desirable properties in large samples. • These are called asymptotic properties.

34. Three such properties often mentioned are consistency, asymptotic unbiasedness ,and asymptotic efficiency

35. 2.6 Properties of Estimators • Suppose that is the estimator of base on a sample of size n. • Then the sequence of estimators is called a consistent sequence if for any arbitrarily small positive numbers and there is a sample size such that

36. 2.6 Properties of Estimators • That is, by increasing the sample size n the estimator can be made to lie arbitrarily close to the true value of with probability arbitrarily close to 1. • This statement is also written as And more briefly we write it as

37. 2.6 Properties of Estimators • A sufficient condition for to be consistent is that the bias and variance should both tend to zero as the sample size increase. • This condition is often useful to check in practice, but it should be noted that the condition is not necessary. • An estimator can be consistent even if the bias does not tend to zero • An example: unbiased and consistent estimate