1 / 44

Statistical Analysis

Statistical Analysis. Professor Lynne Stokes Department of Statistical Science Lecture 6 Solving Normal Equations and Estimating Estimable Model Parameters. Regression Models. Model. Residuals. Least Squares. Sum of Squared Residuals. Solution: Solve the Normal Equations.

jorryn
Download Presentation

Statistical Analysis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Statistical Analysis Professor Lynne Stokes Department of Statistical Science Lecture 6 Solving Normal Equations and Estimating Estimable Model Parameters

  2. Regression Models Model Residuals Least Squares Sum of Squared Residuals Solution: Solve the Normal Equations

  3. Regression Solution • Under usual assumptions, the least squares estimator is • Unique • Unbiased • Minimum Variance • Consistent • Known sampling distribution • Universally used

  4. Analysis of Completely Randomized Designs Fixed Factor Effects Factor levels specifically chosen Inferences desired only on the factor levels included in the experiment Systematic, repeatable changes in the mean response

  5. Flow Rate Experiment Fixed or Random ? MGH Fig 6.1

  6. Flow Rate Experiment 0.35 0.30 Average Flow Rate Conclusion ? 0.25 0.20 A B C D Filter Type

  7. Statistical Model for Single-Factor, Fixed Effects Experiments Model yij = m + ai + eij i = 1, ..., a; j = 1, ..., ri Response Overall Mean (Constant) Main Effect for Level i Error ai: Effect of Level i = change in the mean response

  8. Effects Model yij = m + ai + eij i = 1, ..., a; j = 1, ..., ri Fixed Effects Models Connection: i =  + i Statistical Model for Single-Factor, Fixed Effects Experiments Cell Means Model yij = mi + eij i = 1, ..., a; j = 1, ..., ri

  9. Solving the Normal Equations Single-Factor, Balanced Experiment yij = m + ai + eij i = 1, ..., a j = 1, ..., r n = ar Matrix Formulation y = Xb + e y = (y11 y12 ... y1r ... ya1 ya2 ... yar)’

  10. Solving the Normal Equations Residuals Least Squares Solution: Solve the Normal Equations

  11. Solving the Normal Equations Normal Equations Check

  12. Solving the Normal Equations Normal Equations Linearly Dependent a + 1 Parameters, a Linearly Independent Equations Infinite Number of Solutions Check

  13. Solving the Normal Equations Normal Equations One Solution

  14. Solving the Normal Equations Normal Equations Another Solution

  15. Solving the Normal Equations Normal Equations Another Solution

  16. Solving the Normal Equations All solutions to the normal equations produce the same estimates of “estimable functions” of the model means • Solutions are not estimates • Estimable Functions • All solutions provide one unique estimator • Estimators are unbiased

  17. Solving the Normal Equations Two-Factor, Balanced Experiment yijk = mij + eijk = m + ai + bj + (ab)ij + eijk i = 1, ..., a j = 1, ..., b k = 1, ..., r Matrix Formulation y = Xb + e n = abr X = [ 1 : XA : XB : XAB ] b = ( m , a1 , ... , aa , b1 , ... , bb , (ab)11 , ... , (ab)ab)

  18. Solving the Normal Equations Two-Factor, Balanced Experiment yijk = mij + eijk = m + ai + bj + (ab)ij + eijk i = 1, ..., a j = 1, ..., b k = 1, ..., r Matrix Formulation y = Xb + e n = abr X = [ 1 : XA : XB : XAB ] b = ( m , a1 , ... , aa , b1 , ... , bb , (ab)11 , ... , (ab)ab) Number of Parameters 1 + a + b + ab rank( X ) < 1+a+b+ab

  19. Solving the Normal Equations Normal Equations Check

  20. Solving the Normal Equations Matrix Linear Dependencies One Solution 1n None XA1 : Columns of XA Sum to 1n aa= 0 Eliminates a column From XA a – 1 “degrees of freedom”

  21. Solving the Normal Equations Matrix Linear Dependencies One Solution 1n None XA 1 : Columns Sum of XA to 1n aa= 0 XB1 : Columns Sum of XB to 1nbb = 0 Eliminates a column From XB b – 1 “degrees of freedom”

  22. Solving the Normal Equations Matrix Linear Dependencies One Solution 1n None XA 1 : Columns sum to 1n aa= 0 XB 1 : Columns sum to 1nbb = 0 XAB1 + (a - 1) + (b - 1) : Sum over all columns = 1n (ab)ab= 0 Eliminates a column from XAB

  23. Solving the Normal Equations Matrix Linear Dependencies One Solution 1n None XA 1 : Columns Sum to 1n aa= 0 XB 1 : Columns Sum to 1nbb = 0 XAB 1 + (a - 1) + (b - 1) : Sum over all columns = 1n (ab)ab= 0 Sums of columns over each i = 1,...,a-1 & each j = 1,...,b-1 (ab)ib= 0 equal one of the remainingi=1,...,a-1 columns of XA and XB (ab)aj= 0 j=1,...,b-1 (a – 1)(b – 1) “degrees of freedom”

  24. Solving the Normal Equations Matrix Linear Dependencies One Solution XA 1 : Columns sum to 1n aa= 0 XB 1 : Columns sum to 1nbb = 0 XAB 1 + (a - 1) + (b - 1) : Sum over all columns = 1n (ab)ab= 0 Sums of columns over each i = 1,...,a-1 & each j = 1,...,b-1 (ab)ib= 0 equal one of the remaining i=1,...,a-1 columns of XA and XB (ab)aj= 0 j=1,...,b-1 Constraints : 1 + 1 + {1 + (a - 1) + (b - 1)} = a + b + 1 Degrees of Freedom : (1 + a + b + ab) - (a + b + 1) = ab = 1 + (a - 1) + (b - 1) + (a - 1)(b - 1)

  25. Solving the Normal Equations Check

  26. Solving the Normal Equations Another Solution Check

  27. Flow Rate Experiment Fixed or Random ? MGH Fig 6.1

  28. Quantifying Factor Effects Effect Change in average response due to changes in factor levels Factor Level 1 2 3 k Overall Average . . . . . . Average Effect of Level t : -

  29. Quantifying Factor Effects Effect Change in average response due to changes in factor levels Factor Level 1 2 3 k Overall Average . . . Average . . . Effect of changing from Level s to Level t :

  30. Main Effects for Factor B Change in average response due to changes in the levels of Factor B Interaction Effects for Factors A & B Effect of Level i of Factor A at Level j of Factor B Effect of Level i of Factor A Quantifying Factor Effects Main Effects for Factor A Change in average response due to changes in the levels of Factor A

  31. Quantifying Factor Effects Main Effects for Factor A Change in average response due to changes in the levels of Factor A Main Effects for Factor B Change in average response due to changes in the levels of Factor B Interaction Effects for Factors A & B Change in average response due joint changes in Factors A & B in excess of changes in the main effects

  32. Two-Level Factors Effect of Level 1: Effect of Level 2: Common to Use Note: If r1 = r2 ,

  33. Factors at Two Levels • Most common choice for designs involving many factors • Many efficient fractional factorial and screening designs available • Can use p two-level factors in place of factors whose number of levels is 2p

  34. M(Temp) = Average @ 180o - Average @ 160o = 75.8 - 52.8 = 23.0 M(Conc) = Average @ 40% - Average @ 20% = 61.8 - 66.8 = -5.0 M(Catalyst) = Average @ C2 - Average @ C1 = 65.0 - 63.5 = 1.5 Calculating Two-Level Factor Effects: Pilot Plant Study Main Effect Difference between the average responses at the two levels BHH Section 10.3 MGH Section 5.3

  35. M(Conc @ C2) = Average @ 40%&C2 - Average @ 20%&C2 = 62.5 - 67.5 = -5.0 M(Conc @ C1) = Average @ 40%&C1 - Average @ 20%&C1 = 61.0 - 66.0 = -5.0 I(Conc,Cat) = {M(Conc @ C2) - M(Conc @ C1)} / 2 = 0 Calculating Two-Level Factor Effects Two-Factor Interaction Effect Half the difference between the main effects of one factor at each level of the second factor BHH Section 10.4 MGH Section 5.3

  36. Calculating Two-Level Factor Effects Two-Factor Interaction Effect Half the difference between the main effects of one factor at each level of the second factor M(Temp @ C2) = Average @ 180o&C2 - Average @ 160o&C2 = 81.5 - 48.5 = 33.0 M(Temp @ C1) = Average @ 180o&C1 - Average @ 160o&C1 = 70.0 - 57.0 = 13.0 I(Temp,Cat) = {M(Temp @ C2) - M(Temp @ C1)} / 2 = (33.0 - 13.0) / 2 = 10.0

  37. Cell Means and Effects Model Estimability Three-Factor Balanced Experiment yijkl = mijk + eijkl i = 1 , ... , a ; j = 1 , ... , b ; k = 1, ... , c ; l = 1 , ... , r mijk = m + ai + bj + gk + (ab)ij + (ag)ik + (bg)jk + (abg)ijk

  38. Cell Means Models: Estimable Functions All cell means are estimable

  39. Cell Means Models: Estimable Functions All cell means are estimable All linear combinations of cell means are estimable Does not depend on parameter constraints

  40. Cell Means Models: Estimable Functions All cell means are estimable Some linear combinations of cell means are uninterpretable Some linear combinations of cell means are essential

  41. Cell Means and Effects Models Imposing parameter constraints simplifies the relationships; makes the parameters more interpretable

  42. Means and mean effects Parameter Equivalence:Effects Representation & Cell Means Model Parameter constraints

  43. Contrasts Contrasts often eliminate nuisance parameters; e.g., m

  44. Contrasts Main Effects Interactions Show

More Related