Estimation

1. Estimation

2. Estimators & Estimates • Estimators are the random variables used to estimate population parameters, while the specific values of these variables are the estimates. • Example: the estimator of m is often but if the observed values of X are 1, 2, 3, and 6, the estimate is 3. So the estimator is a formula; the estimate is a number.

3. Properties of a Good Estimator • Unbiasedness • Efficiency • Sufficiency • Consistency

4. Unbiasedness • An estimator (“theta hat”) is unbiased if its expected value equals the value of the parameter (theta) being estimated. That is, In other words, on average the estimator is right on target.

5. Examples If we divided by n instead of by n-1, we would not have an unbiased estimator of s2. That is why s2 is defined the way it is.

6. Bias • The bias of an unbiased estimator is • zero.

7. Mean Squared Error (MSE)

8. Efficiency • The most efficient estimator is the one with the smallest MSE.

9. Efficiency for unbiased estimators (where the bias is zero), MSE = s2. So if you are comparing unbiased estimators, the most efficient one is the one with the smallest variance. If you have two estimators, one of which has a small bias & a small variance and the other has no bias but a large variance, the more efficient one may be the one that is just slightly off on average, but that is more frequently in the right vicinity.

10. Example: sample mean & median • As we have found, the sample mean is an unbiased estimator of m. • It turns out that the sample median is also an unbiased estimator of m. • We know the variance of the sample mean is s2/n. • The variance of the sample median is (p/2)(s2/n). • Since p is about 3.14, p/2 >1. • So the variance of the sample median is greater than s2/n, the variance of the sample mean. • Since both estimators are unbiased, the one with the smaller variance (the sample mean) is the more efficient one. • In fact, among all unbiased estimators of m, the sample mean is the one with the smallest variance.

11. Sufficiency • An estimator is said to be sufficient if it uses all the information about the population parameter that the sample can provide.

12. Examples • Example 1: The sample median is not a sufficient estimator because it uses only the ranking of the observations, and not their numerical values [with the exception of the middle one(s)]. • Example 2: The sample mean, however, uses all the information, and therefore is a sufficient estimator.

13. Consistency • An estimator is said to be consistent if it yields estimates that converge in probability to the population parameter being estimated as n approaches infinity. • In other words, as the sample size increases, The estimator spends more and more of its time closer and closer to the parameter value. • One way that an estimator can be consistent is for its bias and its variance to approach zero as the sample size approaches infinity.

14. Example of a consistent estimator distribution of estimator when n = 500 • distribution of estimator when n = 50 As the sample size increases, the bias & the variance are both shrinking. distribution of estimator when n = 5 m

15. Example: Sample Mean • _ • We know that the mean of X is m. • So its bias not only goes to zero as n approaches infinity, its bias is always zero. • The variance of the sample mean is s2/n. • As n approaches infinity, that variance approaches zero. • So, since both the bias and the variance go to zero, as n approaches infinity, the sample mean is a consistent estimator.

16. A great estimator: the sample mean We have found that the sample mean is a great estimator of the population mean m. It is unbiased, efficient, sufficient, & consistent.

17. Point Estimators versus Interval Estimators • Up until now we have considered point estimators that provide us with a single value as an estimate of a desired parameter. • It is unlikely, however, that our estimate will precisely equal our parameter. • We, therefore, may prefer to report something like this: We are 95% certain that the parameter is between “a” and “b.” • This statement is a confidence interval.

18. 0.4750 0 1.96 Z Building a Confidence Interval • We know that Pr(0 < Z < 1.96) = 0.4750 • Then Pr(-1.96 < Z < 1.96) = 0.95 -1.96 We also know that is distributed as a standard normal (Z). So there is a 95% probability that

19. Continuing from: with 95% probability, Multiplying through by , Subtracting off , Multiplying by -1 and flipping the inequalities appropriately, Flipping the entire expression,

20. So we have a 95% Confidence Interval for the Population Mean m

21. Example: Suppose a sample of 25 students at a university has a sample mean IQ of 127. If the population standard deviation is 5.4, calculate the 95% confidence interval for the population mean. We are 95% certain that the population mean is between 124.88 & 127.12 .

22. When we say we are 95% certain that the population mean m is between 124.88 & 127.12, it means this: • The population mean m is a fixed number, but we don’t know what it is. • Our confidence intervals, however, vary with the random sample that we take. • Sometimes we get a more typical sample, sometimes a less typical one. • If we took 100 random samples and from them calculated 100 confidence intervals, 95 of the intervals should contain the population mean that we are trying to estimate.

23. What if we want a confidence level other than 95%? In our formula, the 1.96 came from our the fact that the Z distribution will be between -1.96 and 1.96 95% of the time. To get a different confidence level, all we need to do is find the Z values such that we are between them the desired percent of the time. Using that Z value, we have the general formula for the confidence interval for the population mean m :

24. Notice: In our confidence interval formula, we used “less than” symbols: Your textbook uses “less than or equal to” symbols: Either of these is acceptable. Recall that the formula is built upon the concept of the normal probability distribution. The probability that a continuous variable is exactly equal to any particular number is zero. So it doesn’t matter whether you include the endpoints of the interval or not.

25. Determining Z values for confidence intervals 0.9800 0.4900 -k 0 k Z -2.33 2.33 • Suppose we want a 98% confidence interval. • We need to find 2 values, call them –k and k, such that Z is between them 98% of the time. • Then Z will be between 0 and k with probability half of 0.98, which is 0.49 . • Look in the body of the Z table for the value closest to 0.49, which is 0.4901 . • The number on the border of the table corresponding to 0.4901 is 2.33. • So that is your value of k, and the number you use for Z in your confidence interval.

26. Sometimes 2 numbers in the Z table are equally close to the value you want. • For example, if you want a 90% confidence interval, you look for half of 0.90 in the body of the Z table, that is, 0.45. • You find 0.4495 and 0.4505. Both are off by 0.0005. • The number on the border of the table corresponding to 0.4495 is 1.64. • The number corresponding to 0.4505 is 1.65. • Usually in these cases, we use the average of 1.64 and 1.65, which is 1.645. • Similarly for the 99% confidence interval, we usually use 2.575. (Draw your graph & work through the logic of this number.)

27. Which interval is wider: One with a higher confidence level (such as 99%) or one with a lower confidence level (such as 90%)? • Let’s think it through using an unrealistic but slightly entertaining example.

28. You have the misfortune of being stranded on an island, with a cannibal & a bunch of bears. • It gets worse… • You get captured by the cannibal. • The cannibal, who knows the island well, decides to give you a chance to avoid being dinner. • He says if you can correctly estimate the number of bears, he’ll let you go. • To give you a fighting chance, he’ll let you give him an interval estimate.

29. You think that there are probably about a hundred bears on the island. • Would you be more confident of not being dinner if you gave the cannibal a narrow interval like 90 to 110 bears, or a wider one like 75 to 125 bears? • You would definitely be more confident with the wider interval. • Thus, when the confidence level needs to be very high (such as 99%), the interval needs to be wide.

30. Let’s redo the IQ example with a different confidence level. We had a sample of 25 students with a sample mean IQ of 127. The population standard deviation was 5.4 . Calculate the 99% confidence interval for the population mean. Our general formula is: We said that the Z value for 99% confidence is 2.575. Putting in our values, or 124.22 < m < 129.78

31. We had for the 95% confidence interval: 124.88 < m < 129.12We just got for the 99% confidence interval:124.22 < m < 129.78The 99% confidence interval starts a little lower & ends a little higher than the 95% interval. So the 99% interval is wider than the 95% interval, as we said it should be.

32. What do we do if we want to compute a confidence interval for m, but we don’t know the population standard deviation s? • We use the next best thing, the sample standard deviation s. • But with s, instead of a Z distribution, we have a t (with n-1 degrees of freedom). So, becomes

33. Example: From a large class of normally distributed grades, sample 4 grades: 64, 66, 89, & 77. Calculate the 95% confidence interval for the class mean grade m. is the appropriate formula. So we need to determine the sample mean, sample standard deviation, and the t-value.

34. 4 grades: 64, 66, 89, & 7795% confidence interval for m Adding our X values,we get 296.

35. 4 grades: 64, 66, 89, & 7795% confidence interval for m Dividing by 4, we findour sample mean is 74.

36. 4 grades: 64, 66, 89, & 7795% confidence interval for m Keep in mind that the sample standard deviation is So, next we subtract our sample mean 74 from each of our X values,

37. 4 grades: 64, 66, 89, & 7795% confidence interval for m square the differencesand add them up.

38. 4 grades: 64, 66, 89, & 7795% confidence interval for m Then we divide by n-1 (which is 3) to get the sample variance s2,

39. 4 grades: 64, 66, 89, & 7795% confidence interval for m and take the square root to get the sample standard deviation s.

40. 0.95 0.025 0.025 0 3.182 t3 So we have and s = 11.5 Since n = 4, dof = n-1 = 3 Since we want 95% confidence, we want 0.95 as the middle area of our graph, and .025 in each of the 2 tails. We find the 3.182 in our t table. Our formula is Putting in our numbers we have So our 95% confidence interval is 56 < m < 92. The interval is very wide, because we only have 4 observations. If we had more information, we’d be able to get a narrower interval.

41. std . dev. or estimate of the std. dev. of our pt. estimate std . dev. or estimate of the std. dev. of our pt. estimate z or t point estimate z or t Desired parameter point estimate From our previous confidence intervals, we can see that we have a basic format, which can be used when the point estimator is roughly normal.

42. Calculating confidence intervals for the binomial proportion parameter p • When the number events of interest (X) and the number of events not of interest (n-X) are each at least five, the binomial distribution can be approximated by the normal and we can develop a confidence interval for the binomial proportion parameter p. • That is, we can develop a confidence interval for p ,if X ≥ 5 and n-X ≥ 5 .

43. We need a point estimate for p, & the standard deviation of our point estimate. • For the point estimate we will use the binomial proportion variable X/n or p . • Its standard deviation was . • Since we don’t know p, we will use our sample proportion p in the standard deviation formula.

44. std . dev. or estimate of the std. dev. of our pt. estimate std . dev. or estimate of the std. dev. of our pt. estimate z or t point estimate z or t Desired parameter point estimate Use our format to get the confidence interval for the binomial proportion p.

45. We have our confidence interval for the binomial proportion p.

46. 0.95 0.4750 0 1.96 Z Example: Consider a random sample of 144 families; 48 have 2 or more cars. Compute the 95% confidence interval for the population proportion of families with 2 or more cars. n = 144 Our z value is 1.96 .

47. We now have n = 144, z = 1.96, So our 95% confidence interval for p is: 0.256 < p < 0.410 .

48. Suppose we want a confidence interval not for a mean but for the difference in two means (m1-m2). • For example, we may be interested in • the difference in the mean income for two counties, or • the difference in the mean exam scores for two classes.

49. std . dev. or estimate of the std. dev. of our pt. estimate std . dev. or estimate of the std. dev. of our pt. estimate z or t point estimate z or t Desired parameter point estimate We will use the same basic format, but it will be a bit more complicated. Our “desired parameter” is m1 – m2 . Our point estimate is . Initially, we will assume that we have the population standard deviations, so we will use a z. We need the standard deviation of the point estimate, . To get that we will first determine the variance of .

50. Recall:V(aX + bY) = a2V(X) + b2V(Y) + 2ab[C(X,Y)] • Letting a = 1, b = -1, If our samples are independent, the covariance term is zero, and the expression becomes or