Multivariate Data Analysis Chapter 4 – Multiple Regression

1. Multivariate Data AnalysisChapter 4 – Multiple Regression MIS 6093 Statistical Method Instructor: Dr. Ahmad Syamil

2. Chapter 4What is Multiple Regression Analysis? • An Example of Simple and Multiple Regression • Setting a Baseline: Prediction Without an Independent Variable • Prediction Using A Single Independent Variable – Simple Regression • The Role of the Correlation Coefficient • Specifying the Simple Regression Equation • Establishing a Confidence Interval for the Prediction • Assessing Prediction Accuracy

3. Chapter 4What is Multiple Regression Analysis? • Prediction Using Several Independent Variables – Multiple Regression • The Impact of Multicollinearity • The Multiple Regression Equation • Adding a Third Independent Variable • Summary

4. Chapter 4A Decision Process for Multiple Regression Analysis • Stage 1: Objectives of Multiple Regression • Research Problems Appropriate for Multiple Regression • Prediction with Multiple Regression • Explanation with Multiple Regression • Specifying a Statistical Relationship • Selection of Dependent and Independent Variables

5. Chapter 4A Decision Process for Multiple Regression Analysis Cont. • Stage 2: Research Design of a Multiple Regression Analysis • Sample Size • Statistical Power and Sample Size • Generalizability and Sample Size • Fixed Versus Random Effects Predictors • Creating Additional Variables • Incorporating Nonmetric Data with Dummy Variables • Representing Curvilinear Effects with Polynomials • Representing Interaction or Moderator Effects • Summary

6. Chapter 4A Decision Process for Multiple Regression Analysis Cont. • Stage 3: Assumptions in Multiple Regression Analysis • Assessing Individual Variables Versus the Variate • Linearity of the Phenomenon • Constant Variance of the Error Term • Independence of the Error Terms • Normality of the Error Term Distribution • Summary

7. Chapter 4A Decision Process for Multiple Regression Analysis Cont. • Stage 4: Estimating the Regression Model and Assessing Overall Fit • General Approaches to Variables Selection • Confirmatory Specification • Sequential Search Methods • Combinational Approach • Overview of the Model Selection Approaches • Testing the Regression Variate for Meeting the Regression Assumptions

8. Chapter 4A Decision Process for Multiple Regression Analysis Cont. • Stage 4: Estimating the Regression Model and Assessing Overall Fit (Cont.) • Examining the Statistical Significance of Our Model • Significance of the Overall Model: The Coefficient of Determination • Significance Tests of Regression Coefficients • Identifying Influential Observations

9. Chapter 4A Decision Process for Multiple Regression Analysis Cont. • Stage 5: Interpreting the Regression Variate • Using the Regression Coefficients • Standardizing the Regression Coefficients: Beta Coefficients • Assessing Multicollinearity • The Effect of Multicollinearity • Identifying Multicollinearity • Remedies for Multicollinearity

10. Chapter 4A Decision Process for Multiple Regression Analysis Cont. • Stage 6: Validation of the Results • Additional or Split Samples • Calculating the PRESS Statistics • Comparing Regression Models • Predicting with the Model

11. Chapter 4Illustration of a Regression Analysis • Stage 1: Objectives of the Multiple Regression • Stage 2: Research Design of the Multiple Regression Analysis • Stage 3: Assumptions of the Multiple Regression Analysis

12. Chapter 4Illustration of a Regression Analysis (Cont.) • Stage 4: Estimating the Regression Model and Assessing Overall Model Fit • Stepwise Estimation: Selecting the First Variable • Stepwise Estimation: Adding X3 • Stepwise Estimation: A Third Variable is Added ---- X6 • Evaluating the Variate for the Assumptions of Regression Analysis • Identifying Outliers as Influential Observations

13. Chapter 4Illustration of a Regression Analysis(Cont.) • Stage 5: Interpreting the Variate • Measuring the Degree and Impact of Multicollinearity • Stage 6: Validating the Results • Evaluating Alternative Regression Models • A Confirmatory Regression Models • Including a Nonmetric Independent Variable • A Managerial Overview of the Results

14. Chapter 4 • Summary • Questions • References …..to Chapter 4A

15. Chapter 4A • Assessing Multicollinearity • A Two-Part Process • An Illustration of Assessing Multicollinearity

16. Chapter 4aIdentifying Influential Observations • Step 1: Examining Residuals • Analysis of Residuals • Partial regression plots • Step 2: Identifying Leverage Points from the Predictors • Hat Matrix • Mahalanobis distance (D^2)

17. Chapter 4aIdentifying Influential Observations (Cont.) • Step 3: Single-Case Diagnostics Identifying Influential Observations • Influences on individual coefficients • Overall influence measures • Step 4: Selecting and Accommodating Influential Observations

18. Chapter 4aIdentifying Influential Observations (Cont.) • Example from the HATCO Database • Step 1: Examining the Residuals • Step 2: Identifying Leverage Points • Step 3: Single-Case Diagnostics • Step 4: Selecting and Accommodating Influential Cases • Overview

19. Chapter 4A • Summary • Questions • References …………end