Intrinsically Linear Regression

1. 9 - 1 Intrinsically Linear Regression Chapter 9

2. 9 - 2 Introduction In Chapter 7 we discussed some deviations from the assumptions of the regression model. One of the assumptions was that the residuals are normally distributed. If this assumption does not hold, then we may have to transform the data into a form that will make it appear linear, so that regression analysis can be used.

3. 9 - 3 Non-linear Models � Introduction To model non-linear relationships with OLS regression, the data must first be transformed in a way that makes the relationship linear All the steps for linear regression may then be performed on the transformed data The most common forms of non-linear models are: Logarithmic Exponential Power To apply linear regression to non-linear models, the data which exhibit an underlying non-linear relationship must be transformed so as to reveal a linear relationship instead. The transformations are performed on the x-coordinates, y-coordinates, or both x- and y-coordinates of the data. Once this is accomplished, the steps for linear regression analysis may be performed exactly as seen before on the transformed data. The most common transformation is to take natural logarithms � this function, when properly applied, causes Logarithmic, Exponential, or Power models to become linear models. These functions and the specific transformations needed to make them linear are shown on the next slide.To apply linear regression to non-linear models, the data which exhibit an underlying non-linear relationship must be transformed so as to reveal a linear relationship instead. The transformations are performed on the x-coordinates, y-coordinates, or both x- and y-coordinates of the data. Once this is accomplished, the steps for linear regression analysis may be performed exactly as seen before on the transformed data. The most common transformation is to take natural logarithms � this function, when properly applied, causes Logarithmic, Exponential, or Power models to become linear models. These functions and the specific transformations needed to make them linear are shown on the next slide.

4. 9 - 4 Linear transformations Logarithmic y = a + b ln x Exponential y = a e b x ln y = ln a + b x Power y = a x b ln y = ln a + b ln x This slide shows the three aforementioned types of non-linear models and how they can be transformed to be made linear. The unit space curves are on the left, illustrating the shape(s) to look for in your scatter plot or residuals plot. Note that some of the curves have a variety of shapes � the shapes differ for certain values of the parameter b (all parameters are shown in red). As indicated, both the exponential and power curves increase (decrease) with positive (negative) values of b, which becomes the slope of the line in log space. The linear transformations of the curves are on the right. The variable that must be transformed to make the data linear can be seen by observing the transformed equation and/or the axes of the plots. The logarithmic curve becomes linear when you take the log of x, the exponential curve becomes linear when you take the log of y. The graphs in these cases are called semi-log plots since one of the variables is transformed with a logarithm and the other is not. The power curve becomes linear when you take the log of x and the log of y. The graph in this case is called a log-log plot since both variables are transformed with logarithms. (This power transformation was already seen in Module 7 Learning Curve Analysis.)This slide shows the three aforementioned types of non-linear models and how they can be transformed to be made linear. The unit space curves are on the left, illustrating the shape(s) to look for in your scatter plot or residuals plot. Note that some of the curves have a variety of shapes � the shapes differ for certain values of the parameter b (all parameters are shown in red). As indicated, both the exponential and power curves increase (decrease) with positive (negative) values of b, which becomes the slope of the line in log space. The linear transformations of the curves are on the right. The variable that must be transformed to make the data linear can be seen by observing the transformed equation and/or the axes of the plots. The logarithmic curve becomes linear when you take the log of x, the exponential curve becomes linear when you take the log of y. The graphs in these cases are called semi-log plots since one of the variables is transformed with a logarithm and the other is not. The power curve becomes linear when you take the log of x and the log of y. The graph in this case is called a log-log plot since both variables are transformed with logarithms. (This power transformation was already seen in Module 7 Learning Curve Analysis.)

5. 9 - 5 The data is plotted in unit space (left) then trans-formed and plotted on a semi-log graph (right) The next step is to conductlinear regression analysison the data in semi-logspace After the analysis is complete, we will transform the parameters of the linear equation back to unit space Example: Exponential Model To show this process through an example, we will continue where we left off with the exponential example in Module 6 Basic Data Analysis Principles (see inset slide). In this example, the data points are plotted in unit space on the left. To review, it was observed that the example curve had an exponential shape, therefore we took the log of all the y-data and plotted that transformation against x. This achieved the linear shape shown on the right. The next steps are to run OLS linear regression on the transformed data (the plot on the right) and then transform the parameters from the derived linear equation to those in the exponential equation. This is accomplished using the formulas shown in the box at the bottom of the slide.To show this process through an example, we will continue where we left off with the exponential example in Module 6 Basic Data Analysis Principles (see inset slide). In this example, the data points are plotted in unit space on the left. To review, it was observed that the example curve had an exponential shape, therefore we took the log of all the y-data and plotted that transformation against x. This achieved the linear shape shown on the right. The next steps are to run OLS linear regression on the transformed data (the plot on the right) and then transform the parameters from the derived linear equation to those in the exponential equation. This is accomplished using the formulas shown in the box at the bottom of the slide.

6. 9 - 6 The Multiplicative Model

7. 9 - 7 The Multiplicative Model However, in order to produce a cost estimating relationship using the method of least squares, we must transform the multiplicative model into a linear model (at least temporarily). The solution is to create a log-linear equation. Now we can perform a linear regression on ln(Y) and ln(X), then transform the results of the linear regression into an exponential equation.

8. 9 - 8 Interpreting the Results A linear regression of transformed data will provide exactly the same type of results as a linear regression of raw data. The computer doesn�t know you�ve given it transformed data. So you need to re-transform the results into an exponential model. The Intercept corresponds to ln(b0), and the slope corresponds to the exponent, b1. Moreover, the statistics of the transformed data can be misleading.

9. 9 - 9 Interpreting the Results The Standard Error and the R2 reported for a log-linear model can not be compared to those for a linear model. This is because both are functions of SSE. Recall that SSE is the error sum of squares, and the standard error is expressed in terms of dollars. The Standard Error in log space has a different meaning than that in unit space. However, we can compare between log-linear models.

10. 9 - 10 Interpreting the Results When comparing linear and log-linear cost models, use section III in Cost$tat entitled Predictive Measures (unit space). Cost$tat provides measures in unit space in section III for log-linear models. These numbers can be compared directly to corresponding linear models. Standard Error Coefficient of Variation Adjusted R-squared Remember, you CANNOT compare these measures in log space to the same measures in unit space. All comparisons must be in unit space.