1 / 34

Robust Estimators

Robust Estimators. Estimating Population Parameters. The mean (X) and variance (s 2 ) are the descriptive statistics that are most commonly used to represent the data points of some sample.

red
Download Presentation

Robust Estimators

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Robust Estimators

  2. Estimating Population Parameters • The mean (X) and variance (s2) are the descriptive statistics that are most commonly used to represent the data points of some sample. • The real reason that they are the preferred measures of central tendency and variance is because of certain properties they have as estimators of their corresponding population parameters; and 2.

  3. Estimating Population Parameters (cont.) • Four properties are considered desirable in a population estimator; sufficiency, unbiasedness, efficiency, & resistance. • Both the mean and the variance are the best estimators in their class in terms of the first three of these four properties. • To understand these properties, you first need to understand a concept in statistics called the sampling distribution

  4. Sampling Distribution Demo • Basically, the idea is this – in order to examine the properties of a statistic we often want to take repeated samples from some population of data and calculate the relevant statistic on each sample. We can then look at the distribution of the statistic across these samples and ask a variety of questions about it. • Check out this demonstration

  5. Properties of a Statistic • 1) Sufficiency A sufficient statistic is one that makes use of all of the information in the sample to estimate its corresponding parameter. For example, this property makes the mean more attractive as a measure of central tendency compared to the mode or median.

  6. Estimating Population Parameters • 2) Unbiasedness A statistic is said to be an unbiased estimator if its expected value (i.e., the mean of a number of sample means) is equal to the population parameter it is estimating. • Using the resampling procedure, the mean can be shown to be an unbiased estimator.

  7. Assessing the Bias of an Estimator Explanation of N-1 in s2 formula. • If the more intuitive formula for s2 is used: • it turns out to underestimate 2

  8. Assessing the Bias of an Estimator (cont.) • This bias to underestimate is caused by the act of sampling and it can be shown that this bias can be reduced if N-1 is used in the denominator instead of N. • Note that this is only true when calculating s2, if you have a measurable population and you want to calculate 2, you use N in the denominator, not N-1.

  9. Degrees of Freedom • The mean of 6, 8, & 10 = 8. • If I allow you to change as many of these numbers as you want BUT the mean must stay 8, how many of the numbers are you free to vary?

  10. Degrees of Freedom • The point of this exercise is that when the mean is fixed, it removes a degree of freedom from your sample -- this is like actually subtracting 1 from the number of observations in your sample. • It is for this reason that we use N-1 in the denominator when we calculate s2 • The calculation requires that the mean be fixed first which effectively removes -- fixes -- one of the data points.

  11. Estimating Population Parameters • 3) Efficiency The efficiency of a statistic is reflected in the variance that is observed when one examines the statistic over a bunch of independently chosen samples. The smaller the variance, the more efficient the statistic is said to be.

  12. Estimating Population Parameters • 4) Resistance The resistance of an estimator refers to the degree to which that estimate is effected by extreme values. As mentioned previously, both X and s2 are highly sensitive to extreme values.

  13. Estimating Population Parameters • 4) Resistance Despite this, they are still the most commonly used estimates of the corresponding population parameters, mostly because of their superiority over other measures in terms sufficiency, unbiasedness, & efficiency. However…

  14. J. W. Tukey “Statistics is a science in my opinion, and it is no more a branch of mathematics than are physics, chemistry and economics; for if its methods fail the test of experience – not the test of logic – they will be discarded”

  15. Outline • Different measures • trimmed mean, winsorized mean, M-estimators • Median absolute deviation • Estimation • Bootstrap (a re-sampling procedure) • End result • Increase your chances of getting significant results

  16. Two (sometimes) related questions • What is the best measure to estimate for the scientific purpose? • Mean v median • What is the best way to measure the estimate?

  17. These question both concern robustness, but each concern something else: Which measure: measurement level and meaningfulness of a statistic. How measured: the distribution of the data

  18. Robust Estimation • What is Robust? • Influence plots, breakdown points (resistance) • Robust Estimators • Trimmed, winsorized, and the M-Estimators • Computer intensive methods • The bootstrap • Mostly on central tendency, but briefly on others.

  19. What to measure: Central Tendency How many hours of TV watched yesterday

  20. Trimmed Mean 1 2 2 2 3 3 3 3 3 3 4 4 4 4 4 5 5 6 8 10 2 2 2 3 3 3 3 3 3 4 4 4 4 4 5 5 6 8 • Simply cut off the ends • Example: 10% trim • N=20 • Cut out 2 values (10%), lowest and highest • Mean = 3.95 • Median = 3.50 • 10% trimmed mean = 3.78 

  21. Winsorized Mean 1 2 2 2 3 3 3 3 3 3 4 4 4 4 4 5 5 6 8 10 2 2 2 2 3 3 3 3 3 3 4 4 4 4 4 5 5 6 8 8 • Make some percentage of most extreme values the same as the previous value • Doing this to 10% of our data… • = 3.90 

  22. M-Estimators • More mathematically complex, but that’s what computers are good for • Calculate the center of the distribution and give less weight to values that are further away from center • Different M-Estimators give different weights for deviating values

  23. All together now • Mean = 3.95 • Median = 3.50 • 10% trim mean = 3.78 • 10% Winsorized = 3.90 • Huber’s = 3.51 • Tukey’s s = 3.33 • Hampel’s = 3.37 • Andrew’s = 3.33

  24. S vs. AD vs. Median Absolute Deviation • MAD = the median absolute deviation from the median • Efficiency: MAD > s > AD • s vs. AD • For normal distribution AD = .8(s) • Turns out the efficiency advantage of s over AD is nullified and even reversed for even slightly less than normal data.

  25. MAD • For normal distribution .67(s) or .85(AD) • More efficient than either s or AD • Also more resistant than either • Tukey • MAD “the best of an inferior lot”

  26. Classical Statistics • Parameter Estimates (“Point” Estimates) • Sample mean and sample standard deviation • Sample correlation and covariance estimates • Linear least squares model fits • Statistical Inference • t-statistic and t-interval for an unkown mean • Standard errors and t-values for regression coefficients • F-tests for regression model hypotheses

  27. Classical stats are often not robust Outliers have “unbounded influence” on classical statistics, resulting in: • Inaccurate parameter estimates and predictions • Inaccurate statistical inference • Standard errors are too large • Confidence intervals are too wide • t-statistics lack power • In short, unreliable when outliers are present

  28. Resistance • A Fundamental Continuity Concept - Small changes in the data result in only small changes in estimate • “Change a few, so what” J.W. Tukey • Valuable Consequence - A good fit to the bulk of the data - Reliable, automatic outlier detection

  29. J. W. Tukey (1979) “… just which robust/resistant methods you use is not important – what is important is that you use some. It is perfectly proper to use both classical and robust/resistant methods routinely, and only worry when they differ enough to matter. But when they differ, you should think hard.”

  30. Bootstrap • More voodoo • The basic idea involves sampling with replacement to produce random samples of size n from the original data • Each of these samples provides an estimate of the parameter of interest. • Repeating the sampling a large number of times provides information on the variability of the estimator.

  31. Bootstrap • From previous data example • 1000 samples • Distribution of Means of each sample  • Mean = 3.94

  32. Bootstrap • With smaller data sets we should be concerned about biasedness • Does our statistic reliably estimate the population parameter? • Bootstrapped estimates and subsequent inferential use may be preferable in such low N situations

More Related