Chapter 13:

1. Chapter 13: SIMPLE LINEAR REGRESSION

2. SIMPLE LINEAR REGRESSION • Simple Regression • Linear Regression

3. Simple Regression • Definition • A regression model is a mathematical equation that describes the relationship between two or more variables. A simple regression model includes only two variables: one independent and one dependent. The dependent variable is the one being explained, and the independent variable is the one used to explain the variation in the dependent variable.

4. Linear Regression • Definition • A (simple) regression model that gives a straight-line relationship between two variables is called a linear regression model.

5. Figure 13.1 Relationship between food expenditure and income. (a) Linear relationship. (b) Nonlinear relationship. Linear Food Expenditure Food Expenditure Nonlinear Income Income (b) (a)

6. Figure 13.2 Plotting a linear equation. y y = 50 + 5x 150 100 x = 10 y = 100 50 x = 0 y = 50 5 10 15 x

7. Figure 13.3 y-intercept and slope of a line. y 5 1 Change in y 5 1 50 Change in x y-intercept x

8. SIMPLE LINEAR REGRESSION ANALYSIS • Scatter Diagram • Least Square Line • Interpretation of a and b • Assumptions of the Regression Model

9. SIMPLE LINEAR REGRESSION ANALYSIS cont. y = A + Bx Constant term or y-intercept Slope Independent variable Dependent variable

10. SIMPLE LINEAR REGRESSION ANALYSIS cont. • Definition • In the regression modely = A + Bx + Є, A is called the y-intercept or constant term, B is the slope, and Є is the random error term. The dependent and independent variables are y and x, respectively.

11. SIMPLE LINEAR REGRESSION ANALYSIS • Definition • In the model ŷ = a + bx, a and b, which are calculated using sample data, are called the estimates of A and B.

12. Table 13.1 Incomes (in hundreds of dollars) and Food Expenditures of Seven Households

13. Scatter Diagram • Definition • A plot of paired observations is called a scatter diagram.

14. Figure 13.4 Scatter diagram. First household Seventh household Food expenditure Income

15. Figure 13.5 Scatter diagram and straight lines. Food expenditure Income

16. Least Squares Line Figure 13.6Regression line and random errors. e Food expenditure Regression line Income

17. Error Sum of Squares (SSE) • The error sum of squares, denoted SSE, is • The values of a and b that give the minimum SSE are called the least square estimates of A and B, and the regression line obtained with these estimates is called the least square line.

18. The Least Squares Line • For the least squares regression line ŷ = a + bx,

19. The Least Squares Line cont. • where • and SS stands for “sum of squares”. The least squares regression line ŷ = a + bx us also called the regression of y on x.

20. Example 13-1 • Find the least squares regression line for the data on incomes and food expenditure on the seven households given in the Table 13.1. Use income as an independent variable and food expenditure as a dependent variable.

21. Table 13.2

22. Solution 13-1

23. Solution 13-1

24. Solution 13-1 Thus, ŷ = 1.1414 + .2642x

25. Figure 13.7 Error of prediction. ŷ = 1.1414+ .2642x Predicted = $1038.84 e Error = -$138.84 Food expenditure Actual = $900 Income

26. Interpretation of a and b Interpretation of a • Consider the household with zero income • ŷ = 1.1414 + .2642(0) = $1.1414 hundred • Thus, we can state that households with no income is expected to spend $114.14 per month on food • The regression line is valid only for the values of x between 15 and 49

27. Interpretation of a and b cont. Interpretation of b • The value of b in the regression model gives the change in y due to change of one unit in x • We can state that, on average, a $1 increase in income of a household will increase the food expenditure by $.2642

28. Figure 13.8 Positive and negative linear relationships between x and y. y y b < 0 b > 0 x x (a) Positive linear relationship. (b) Negative linear relationship.

29. Assumptions of the Regression Model Assumption 1: • The random error term Є has a mean equal to zero for each x

30. Assumptions of the Regression Model cont. Assumption 2: • The errors associated with different observations are independent

31. Assumptions of the Regression Model cont. Assumption 3: • For any given x, the distribution of errors is normal

32. Assumptions of the Regression Model cont. Assumption 4: • The distribution of population errors for each x has the same (constant) standard deviation, which is denoted σЄ.

33. Figure 13.11(a) Errors for households with an income of $2000 per month. Normal distribution with (constant) standard deviation σЄ E(ε) = 0 Errors for households with income = $2000 (a)

34. Figure 13.11 (b) Errors for households with an income of $ 3500 per month. Normal distribution with (constant) standard deviation σЄ E(ε) = 0 Errors for households with income = $3500 (b)

35. Figure 13.12 Distribution of errors around the population regression line. 16 Food expenditure 12 Population regression line 8 4 10 x = 20 30 x = 35 40 50 Income

36. Figure 13.13 Nonlinear relations between x and y. y y x x (a) (b)

37. Figure 13.14 Spread of errors for x = 20 and x = 35. 16 Food expenditure 12 Population regression line 8 4 10 x = 20 30 x = 35 40 50 Income

38. STANDARD DEVIATION OF RANDOM ERRORS • Degrees of Freedom for a Simple Linear Regression Model • The degrees of freedom for a simple linear regression model are • df = n – 2

39. STANDARD DEVIATION OF RANDOM ERRORS cont. • The standard deviation of errors is calculated as • where

40. Example 13-2 • Compute the standard deviation of errors se for the data on monthly incomes and food expenditures of the seven households given in Table 13.1.

41. Table 13.3

42. Solution 13-2

43. COEFFICIENT OF DETERMINATION • Total Sum of Squares (SST) • The total sum of squares, denoted by SST, is calculated as

44. Figure 13.15 Total errors. 16 12 Food expenditure 8 4 10 20 30 40 50 Income

45. Table 13.4

46. Figure 13.16 Errors of prediction when regression model is used. ŷ = 1.1414 + .2642x Food expenditure Income

47. COEFFICIENT OF DETERMINATION cont. • Regression Sum of Squares (SSR) • The regression sum of squares , denoted by SSR, is

48. COEFFICIENT OF DETERMINATION cont. • Coefficient of Determination • The coefficient of determination, denoted by r2, represents the proportion of SST that is explained by the use of the regression model. The computational formula for r2 is • and 0 ≤ r2 ≤ 1

49. Example 13-3 • For the data of Table 13.1 on monthly incomes and food expenditures of seven households, calculate the coefficient of determination.

50. Solution 13-3 From earlier calculations b = .2642, SSxx = 211.7143, and SSyy= 60.8571