Understanding Analysis of Variance (ANOVA) in Statistical Analysis

1. Analysis of Variance Introduction

2. Analysis of Variance • The Analysis of Variance is abbreviated as ANOVA • Used for hypothesis testing in • Simple Regression • Multiple Regression • Comparison of Means

3. Sources • There is variation anytime that all of the data values are not identical • This variation can come from different sources such as the model or the factor • There is always the left-over variation that can’t be explained by any of the other sources. This source is called the error

4. Variation • Variation is the sum of squares of the deviations of the values from the mean of those values • As long as the values are not identical, there will be variation • Abbreviated as SS for Sum of Squares

5. Degrees of Freedom • The degrees of freedom are the number of values that are free to vary once certain parameters have been established • Usually, this is one less than the sample size, but in general, it’s the number of values minus the number of parameters being estimated • Abbreviated as df

6. Variance • The sample variance is the average squared deviation from the mean • Found by dividing the variation by the degrees of freedom • Variance = Variation / df • Abbreviated as MS for Mean of the Squares • MS = SS / df

7. F • F is the F test statistic • There will be an F test statistic for each source except for the error and total • F is the ratio of two sample variances • The MS column contains variances • The F test statistic for each source is the MS for that row divided by the MS of the error row

8. F • F requires a pair of degrees of freedom, one for the numerator and one for the denominator • The numerator df is the df for the source • The denominator df is the df for the error row • F is always a right tail test

9. The ANOVA Table • The ANOVA table is composed of rows, each row represents one source of variation • For each source of variation … • The variation is in the SS column • The degrees of freedom is in the df column • The variance is in the MS column • The MS value is found by dividing the SS by the df

10. ANOVA Table • The complete ANOVA table can be generated by most statistical packages and spreadsheets • We’ll concentrate on understanding how the table works rather than the formulas for the variations

11. The ANOVA Table The explained* variation has different names depending on the particular type of ANOVA problem

12. Example 1 The Sum of Squares and Degrees of Freedom are given. Complete the table.

13. Example 1 – Find Totals Add the SS and df columns to get the totals.

14. Example 1 – Find MS Divide SS by df to get MS.

15. Example 1 – Find F F = 6.30 / 4.50 = 1.4

16. Notes about the ANOVA • The MS(Total) isn’t actually part of the ANOVA table, but it represents the sample variance of the response variable, so it’s useful to find • The total df is one less than the sample size • You would either need to find a Critical F value or the p-value to finish the hypothesis test

17. Example 2 Complete the table

18. Example 2 – Step 1 SS / df = MS, so 106.6 / df = 21.32. Solving for df gives df = 5. F = MS(Source) / MS(Error), so 2.60 = 21.32 / MS. Solving gives MS = 8.20.

19. Example 2 – Step 2 SS / df = MS, so SS / 26 = 8.20. Solving for SS gives SS = 213.2. The total df is the sum of the other df, so 5 + 26 = 31.

20. Example 2 – Step 3 Find the total SS by adding the 106.6 + 213.2 = 319.8

21. Example 2 – Step 4 Find the MS(Total) by dividing SS by df. 319.8 / 31 = 10.32

22. Example 2 – Notes • Since there are 31 df, the sample size was 32 • Since the sample variance was 10.32 and the standard deviation is the square root of the variance, the sample standard deviation is 3.21

23. Example 3 The sample size is n = 20. Work this one out on your own!

24. Example 3 - Solution How did you do?