1 / 57

ANOVA: Analysis of Variation

ANOVA: Analysis of Variation. The basic ANOVA situation. Two variables: 1 Categorical, 1 Quantitative Main Question: Do the (means of) the quantitative variables depend on which group (given by categorical variable) the individual is in?. ANOVA Example. 比較各省籍 ( 台灣、大陸、客家人 ) 人士在收入及教育年數上的差異。

emery-mccall
Download Presentation

ANOVA: Analysis of Variation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ANOVA:Analysis of Variation

  2. The basic ANOVA situation Two variables: 1 Categorical, 1 Quantitative Main Question: Do the (means of) the quantitative variables depend on which group (given by categorical variable) the individual is in?

  3. ANOVA Example • 比較各省籍(台灣、大陸、客家人)人士在收入及教育年數上的差異。 • 大學中各年級的同學智商是否有別？ • 三種不同的教學方法對於學生的成績是否有影響？

  4. An example ANOVA situation • Subjects: 25 patients with blisters • Treatments: Treatment A, Treatment B, Placebo • Measurement: # of days until blisters heal • Data [and means]: • A: 5,6,6,7,7,8,9,10 [7.25] • B: 7,7,8,9,9,10,10,11 [8.875] • P: 7,9,9,10,10,10,11,12,13 [10.11] • Are these differences significant?

  5. Informal Investigation • Graphical investigation: • side-by-side box plots • multiple histograms • Whether the differences between the groups are significant depends on • the difference in the means • the standard deviations of each group • the sample sizes

  6. Side by Side Boxplots

  7. At its simplest (there are extensions) ANOVA tests the following hypotheses: H0: The means of all the groups are equal. Ha: Not all the means are equal doesn’t say how or which ones differ. Can follow up with “multiple comparisons” Note: we usually refer to the sub-populations as “groups” when doing ANOVA. What does ANOVA do?

  8. Why ANOVA? 在比較多組母體的平均值時，我們通常不採用兩兩比較的方式，主要的原因有二： 一、這種做法太浪費時間，因為比較幾個母體可能產生很多的比較組，例如比較五個母體的平均值差異，如果以兩兩比較的方式，我們必須進行C52=10次的t-test。 二、如果每組的顯著水準皆為α，則全體比較的顯著水準會高於α。

  9. Why ANOVA? 假設我們在.05的顯著水準下要檢定下列虛擬假設: H0: u1=u2=u3 如果拆成下列三組虛擬假設： H0: u1=u2 , H0: u1=u3 , H0: u2=u3 每個假設被「接受」的機率為.95，三個假設全部被接受的機率為.953=.857，也就是說當假設為真但被推翻的機率為(1 - 0.857) = 0.143 > 0.05 遠高於顯著水準。

  10. Why ANOVA? 因此我們需要在共同的顯著水準α下，同時考量多個平均值得差異，我們以F分配來進行檢定，稱之為變異數分析(ANOVA,ANalysis Of VAriance) 。

  11. Assumptions of ANOVA • each group is approximately normal • check this by looking at histograms and/or normal quantile plots, or use assumptions • can handle some nonnormality, but not severe outliers • standard deviations of each group are approximately equal • rule of thumb: ratio of largest to smallest sample st. dev. must be less than 2:1

  12. Normality Check • We should check for normality using: • assumptions about population • histograms for each group • normal quantile plot for each group • With such small data sets, there really isn’t a really good way to check normality from data, but we make the common assumption that physical measurements of people tend to be normally distributed.

  13. Standard Deviation Check Variable treatment N Mean Median StDev days A 8 7.250 7.000 1.669 B 8 8.875 9.000 1.458 P 9 10.111 10.000 1.764 • Compare largest and smallest standard deviations: • largest: 1.764 • smallest: 1.458 • 1.458 x 2 = 2.916 > 1.764 • Note: variance ratio of 4:1 is equivalent.

  14. ANOVA Example • 消費者很想知道哪種車最省油，比較A, B, C三種車款每加崙可以行駛的里數如下：

  15. ANOVA Example 三種汽車每單位汽油的里數皆相同 Q:我們所觀察到的樣本平均數差異是否大到足以推翻上面的虛擬假設?

  16. ANOVA Example Q:各組平均值的差異是來自於抽樣誤差還是母體差異？

  17. ANOVA Example 例如A車與Ｂ車的平均值差異為1.4里，這個差異是否大到我們可以有信心的說u1與u2也有差異？ 這個問題決定於x1, x2是否為母體平均值的精確估計值。

  18. ANOVA Example 如果標準差很小，則兩個樣本平均值一點點的差距都可能是母體平均值不同的訊號。 同理，如果標準差過大，則即使我們觀察到樣本平均值之間有很大的差距，我們也不太有信心能夠宣稱母體的平均數真的有別

  19. ANOVA Example 樣本標準差或變異數測量各個樣本內，各觀察值之間的變異程度。 如果樣本內的變異數很小，則各樣本之間平均數的差距若過大，為母體平均數不同的有力證據 反之，如果樣本內的變異數過大，則即使樣本平均值之間有差異，我們仍然很難下斷論說母體的平均值不同。

  20. ANOVA Example 因此檢定各樣本的平均值是否相同的問題涉及比較樣本內的變異（組內差異）及樣本間的變異（組間差異）。所以通常稱之為變異數分析。

  21. ANOVA Example 樣本內的變異數很小 C B A 18 19 20 21 22 23

  22. ANOVA Example 樣本內的變異數很大 C B A 15 17 19 21 23 25 27

  23. Variables in ANOVA • 我們經常設計研究來了解造成某種現象變化的原因，例如我們想要了解為什麼有時候種植西瓜會甜有時候不會甜（甜度變動)，這種我們欲了解的變動稱為依變項(dependent variable)、被解釋變項、或反應變項(response variable)。 • 我們懷疑西瓜的甜度與栽種過程中是否施肥有關，將某些西瓜種籽加以施肥處理，其他西瓜保持自然生長，這種造成依變項產生變化的變數稱之為因子(factor)或獨立變項、 自變項(independent variable)。

  24. Dependent and Independent Variables • 在上面的例子中，比較各種汽車的里程數，何者為依變項？何者為獨立變項？ • 依變項： • 自變項：

  25. 因子水準(Factor level)與處理(Treatment) • 因子水準為某因子（自變數）之特殊形式或不同狀態，例如我們可以將「施肥」細分成三個水準：完全不施肥、施輕肥、施重肥。 • 如果解釋的因子為單一(施肥與否），稱為單因子分析，如果解釋因子在兩個以上（施肥與否＋栽種溫度），稱為多因子分析。

  26. The Logical of ANOVA • 假設從K個母體中抽取大小分別為n1, n2, n3…nk的K個獨立隨機樣本。我們對母體有下列的假設： • 各母體皆為常態分配，且有共同相同的變異數σ2。 • 以u1, u2, …uk來表示母體的平均數，單因子分析檢證下虛擬假設 • H0: u1=u2…=uk vs. H1: 至少有兩組平均數不同

  27. Notation for ANOVA • n = number of individuals all together • I = number of groups • = mean for entire data set is • Group i has • ni= # of individuals in group i • xij = value for individual j in group i • = mean for group i • si = standard deviation for group i

  28. How ANOVA works (outline) 總差異 = 由因子所引起的差異＋ 隨機差異 因子的影響 隨機差異的影響 總平均

  29. How ANOVA works (outline) 兩邊取平方和

  30. How ANOVA works (outline) • 變異數分析是透過各組樣本內的變異與組間變異之比較來檢證各組平均值是否相等，全體樣本資料的總變異量為: • 即個別觀察值與總平均數差距的平方和，稱為總變異量或總平方和。

  31. How ANOVA works (outline) • 變異數分析將總變異量分解成下列兩部分: 總變異 =組內變異(或未解釋變異) + 組間變異(或已解釋變異) = Within-group Sum of Squares or Sum of Squares Within (SSW) + Between-Group Sum of Squares or Sum of Squares Between (SSB) Total Sum of Squares (TSS)

  32. How ANOVA works (outline) • ANOVA measures two sources of variation in the data and compares their relative sizes • variation BETWEEN groups • for each data value look at the difference between its group mean and the overall mean • variation WITHIN groups • for each data value we look at the difference between that value and the mean of its group

  33. ANOVA test statistic F The ANOVA F-statistic is a ratio of the Between Group Variaton divided by the Within Group Variation: A large F is evidence againstH0, since it indicates that there is more difference between groups than within groups.

  34. ANOVA test statistic F • 當H0為真時， • 當H0為不真時， 因此MSB及MSW的比率提供我們判斷虛擬假設是否無真的訊息。

  35. ANOVA test statistic F • In One-way ANOVA, the test statistics is 如果H0為真，分子分母皆為母體變異數的不偏估計式，因此兩者的比率會十分接近1。 如果H0為不真，則MSG會高估母體變異數，F值會大於1。F愈大，H0愈不可能為真。 如果假設為真，則F統計量依循自由度為(K-1)及(n-K)的F 分配。

  36. F Distribution • 欲比較兩母體變異數是否相等時，我們可以計算樣本變異數的比值： • 如果比率很接近1，則我們相信母體變異數很有可能一樣，如果此比值很大或很小，則母體變異數相等的機率不高。 • 究竟此比值要多大或多小才能推翻母體變異數相等的虛擬假設？

  37. F Distribution • 為了回答此問題，我們必須知道S21/S22此一隨機變數的抽樣分配。設有兩常態分配的母體： • 且X1與X2互相獨立，自X1, X2中分別取獨立隨機樣本n1, n2，令：

  38. Minitab ANOVA Output Analysis of Variance for days Source DF SS MS F P treatment 2 34.74 17.37 6.45 0.006 Error 22 59.26 2.69 Total 24 94.00 Df Sum Sq Mean Sq F value Pr(>F) treatment 2 34.7 17.4 6.45 0.0063 ** Residuals 22 59.3 2.7 R ANOVA Output

  39. F Distribution • 若虛無假設為真，即σ21=σ22，則檢定量為： • 若將變異數較大者視為來自母體1，則統計檢定量的值會大於1。此時單尾檢定都是右尾檢定

  40. How are these computations made? • We want to measure the amount of variation due to BETWEEN group variation and WITHIN group variation • For each data value, we calculate its contribution to: • BETWEEN group variation: • WITHIN group variation:

  41. An even smaller example • Suppose we have three groups • Group 1: 5.3, 6.0, 6.7 • Group 2: 5.5, 6.2, 6.4, 5.7 • Group 3: 7.5, 7.2, 7.9 • We get the following statistics:

  42. number of data values - number of groups (equals df for each group added together) 1 less than number of groups 1 less than number of individuals (just like other situations) Excel ANOVA Output

  43. Computing ANOVA F statistic overall mean: 6.44 F = 2.5528/0.25025 = 10.21575

  44. Minitab ANOVA Output Analysis of Variance for days Source DF SS MS F P treatment 2 34.74 17.37 6.45 0.006 Error 22 59.26 2.69 Total 24 94.00 # of data values - # of groups (equals df for each group added together) 1 less than # of groups 1 less than # of individuals (just like other situations)

  45. Minitab ANOVA Output Analysis of Variance for days Source DF SS MS F P treatment 2 34.74 17.37 6.45 0.006 Error 22 59.26 2.69 Total 24 94.00 • SS stands for sum of squares • ANOVA splits this into 3 parts

  46. Minitab ANOVA Output Analysis of Variance for days Source DF SS MS F P treatment 2 34.74 17.37 6.45 0.006 Error 22 59.26 2.69 Total 24 94.00 MSG = SSG / DFG MSE = SSE / DFE P-value comes from F(DFG,DFE) F = MSG / MSE (P-values for the F statistic are in Table E)

  47. So How big is F? • Since F is • Mean Square Between / Mean Square Within • = MSG / MSE • A large value of F indicates relatively more • difference between groups than within groups (evidence against H0) • To get the P-value, we compare to F(I-1,n-I)-distribution • I-1 degrees of freedom in numerator (# groups -1) • n - I degrees of freedom in denominator (rest of df)

  48. Connections between SST, MST, and standard deviation If ignore the groups for a moment and just compute the standard deviation of the entire data set, we see So SST = (n-1) s2, and MST = s2. That is, SST and MST measure the TOTAL variation in the data set.

  49. Connections between SSE, MSE, and standard deviation Remember: So SS[Within Group i] = (si2) (dfi ) This means that we can compute SSE from the standard deviations and sizes (df) of each group:

More Related