1 / 68

Regression / Calibration

This text provides an overview of regression and calibration methods including Multiple Linear Regression (MLR), Ridge Regression (RR), Principal Component Regression (PCR), and Partial Least Squares (PLS). It discusses their advantages, limitations, and prediction diagnostics.

jacquelinesmith
Download Presentation

Regression / Calibration

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Regression / Calibration MLR, RR, PCR, PLS

  2. Paul Geladi Head of Research NIRCE Unit of Biomass Technology and Chemistry Swedish University of Agricultural Sciences Umeå Technobothnia Vasa paul.geladi@btk.slu.sepaul.geladi@syh.fi

  3. Univariate regression

  4. y Slope a Offset x

  5. y e y = a + bx + e Slope b a Offset a x

  6. y x

  7. Linear fit Underfit y x

  8. y Overfit x

  9. y Quadratic fit x

  10. Multivariate linear regression

  11. y = f(x) Works sometimes y = f(x) Works only for a few variables Measurement noise! ∞ possible functions

  12. K X y I

  13. y = f(x) y = f(x) Simplified by: y = b0 + b1x1 + b2x2 + ... + bKxK + f Linear approximation

  14. Nomenclature y = b0 + b1x1 + b2x2 + ... + bKxK + f y : response xk : predictors bk : regression coefficients b0 : offset, constant f : residual

  15. K X y I X, y mean-centered b0 out

  16. y = b1x1 + b2x2 + ... + bKxK + f y = b1x1 + b2x2 + ... + bKxK + f y = b1x1 + b2x2 + ... + bKxK + f } I samples y = b1x1 + b2x2 + ... + bKxK + f y = b1x1 + b2x2 + ... + bKxK + f

  17. y = b1x1 + b2x2 + ... + bKxK +f y = b1x1 + b2x2 + ... + bKxK +f y = b1x1 + b2x2 + ... + bKxK +f y = b1x1 + b2x2 + ... + bKxK +f y = b1x1 + b2x2 + ... + bKxK +f

  18. K y X f + = b I y=Xb+f

  19. X, yknown, measurableb,funknownNo solutionfmust be constrained

  20. The MLR solutionMultiple Linear RegressionOrdinary Least Squares (OLS)

  21. b= (X’X)-1X’y Least squares Problems?

  22. 3b1 + 4b2 = 1 4b1 + 5b2 = 0 One solution

  23. 3b1 + 4b2 = 1 4b1 + 5b2 = 0 b1 + b2 = 4 No solution

  24. 3b1 + 4b2 + b3 = 1 4b1 + 5b2 +b3 = 0 ∞ solutions

  25. b= (X’X)-1X’y -K > I ∞ solutions -I > K no solution -error in X -error in y -inverse may not exist -inverse may be unstable

  26. 3b1 + 4b2 + e = 1 4b1 + 5b2 + e = 0 b1 + b2 + e = 4 Solution

  27. Wanted solution • - I ≥ K • No inverse • No noise in X

  28. Diagnostics y=Xb+f SS tot = SSmod + SSres R2 = SSmod / SStot = 1- SSres / SStot Coefficient of determination

  29. Diagnostics y=Xb+f SSres = f’f RMSEC = [ SSres / (I-A) ] 1/2 Root Mean Squared Error of Calibration

  30. Alternatives to MLR/OLS

  31. Ridge Regression (RR) b= (X’X)-1X’y I easiest to invert b= (X’X + kI)-1X’y k (ridge constant) as small as possible

  32. Problems - Choice of ridge constant - No diagnostics

  33. Principal Component Regression (PCR) • I ≥ K • Easy inversion

  34. Principal Component Regression (PCR) A K X T PCA • - A ≤ I • T orthogonal • Noise in X removed

  35. Principal Component Regression (PCR) y=Td+f d = (T’T)-1T’y

  36. Problem How many components used?

  37. Advantage - PCA done on data - Outliers - Classes - Noise in X removed

  38. Partial Least SquaresRegression

  39. X t u Y

  40. X t u Y w’ q’ Outer relationship

  41. X t u Y w’ q’ Inner relationship

  42. A A X t u Y w’ q’ A A p’

  43. Advantages - X decomposed - Y decomposed - Noise in X left out - Noise in Y left out

  44. PCR, PLS are one component at a time methodsAfter each component, a residual is calculatedThe next component is calculatedon the residual

  45. Another view y=Xb+f y=XbRR+fRR y=XbPCR+fPCR y=XbPLS+fPLS

  46. Prediction

  47. K Xcal ycal I Xtest yhat ytest J

  48. Prediction diagnostics yhat = Xtestb ftest = ytest -yhat PRESS = ftest’ftest RMSEP = [ PRESS / J ] 1/2 Root Mean Squared Error of Prediction

  49. Prediction diagnostics yhat = Xtestb ftest = ytest -yhat R2test = Q2 = 1 - ftest’ftest/ytest’ytest

More Related