Non-Linear Models

1. Non-Linear Models • Tractable non-linearity • Equation may be transformed to a linear model. • Intractable non-linearity • No linear transform exists

2. Tractable Non-Linear Models • Several general Types • Polynomial • Power Functions • Exponential Functions • Logarithmic Functions • Trigonometric Functions

3. Polynomial Models • Linear • Parabolic • Cubic & higher order polynomials • All may be estimated with OLS – simply square, cube, etc. the independent variable.

4. Power Functions • Simple exponents of the Independent Variable • Estimated with

5. Exponential and Logarithmic Functions • Common Growth Curve Formula • Estimated with • Note that the error terms are now no longer normally distributed!

6. Logarithmic Functions

7. Trigonometric Functions • Sine/Cosine functions • Fourier series

8. Intractable Non-linearity • Occasionally we have models that we cannot transform to linear ones. • For instance a logit model • Or an equilibrium system model

9. Intractable Non-linearity • Models such as these must be estimated by other means. • We do, however, keep the criteria of minimizing the squared error as our means of determining the best model

10. Estimating Non-linear models • All methods of non-linear estimation require an iterative search for the best fitting parameter values. • They differ in how they modify and search for those values that minimize the SSE.

11. Methods of Non-linear Estimation • There are several methods of selecting parameters • Grid search • Steepest descent • Marquardts algorithm

12. Grid search estimation • In a grid search estimation, we simply try out a set of parameters across a set of ranges and calculate the SSE. • We then ascertain where in the range (or at which end) the SSE was at a minimum. • We then repeat with either extending the range, or reducing the range and searching with smaller grid around the estimated SSE • Try the spreadsheet • Try this for homework!

13. Regression Diagnostics • Some cases are very influential in regression models • There are two ways to describe the influence that case may exert • Residual • Leverage • Examination of these particular cases may lead us to theoretical insight.

14. Regression Diagnostics • Residual • Outliers – extreme measures weaken the goodness of fit indexes and hypothesis tests • Studentized residuals • where hii is the hat diagonal

15. Residual Diagnostics (cont.) • RStudent

16. Residual Diagnostics (cont.) • Leverage – influential observation • Hat diagonal – a measure of the observations “remoteness” in X-space. • Hat diagonal;s greater than 2 times the number of coefficients in the model divided by the number of observations are considered significant.

17. Residual Diagnostics (cont.) • Leverage (cont.) • Cook’s D • If D is greater than 1, then the observation is influential.

18. Residual Diagnostics (cont.) • Leverage (cont.) • dffits • If dffits is greater than 1, then the observation is influential.

19. Residual Diagnostics (cont.) • Leverage (cont.) • Dfbetas – an indicator of how much a given observation influences each regression coefficient