150 likes | 169 Views
Advanced Statistical Methods: Continuous Variables http://statisticalmethods.wordpress.com. Multiple Regression – Part II tomescu.1@sociology.osu.edu. Major Types of Multiple Regression. Standard multiple regression Sequential (hierarchical) regression Statistical (stepwise) regression
E N D
Advanced Statistical Methods: Continuous Variableshttp://statisticalmethods.wordpress.com Multiple Regression – Part II tomescu.1@sociology.osu.edu
Major Types of Multiple Regression Standard multiple regression Sequential (hierarchical) regression Statistical (stepwise) regression R² = a + b + c + d + e R²= the squared multiple correlation; it is the proportion of variation in the DV that is predictable from the best linear combination of the IVs (i.e. coefficient of determination). R = correlation between the observed and predicted Y values (R = ryŶ ) a x2
Adjusted R2 Adjusted R2 = modification of R2 that adjusts for the number of terms in a model. R2 always increases when a new term is added to a model, but adjusted R2 increases only if the new term improves the model more than would be expected by chance.
Standard (Simultaneous) Multiple Regression • all IVs enter into the regression equation at once; each one is assessed as if it had entered the regression after all other IVs had entered. • each IV is assigned only the area of its unique contribution; • the overlapping areas (b & d) contribute to R² but are not assigned to any of the individual IVs
Table 1: Regression of (DV) Assessment of Socialism in 2003 on (IVs) Social Status, controlling for Gender and Age **p <0.001; *p < 0.05; Interpretation of beta (standardized) coefficients: for a one standard deviation unit increase in X, we get a Beta standard deviation change in Y; Since variables are transformed into z-scores (i.e. standradized), we can assess their relative impact on the DV (assuming they are uncorrelated with each other)
Sequential (hierarchical) Multiple Regression - researcher specifies the order in which IVs are added to the equation; • each IV/IVs are assessed in terms of what they add to the equation at their own point of entry; • If X1 is entered 1st, then X2, then X3: X1 gets credit for a and b; X2 for c and d; X3 for e. IVs can be added one at a time, or in blocks a
The Regression SUM of SQUARES, SS(regression) = SS(total) + SS(residual)SSregression = Sum (Ŷ – Ybar)² = portion of variation in Y explained by the use of the IVs as predictors; SStotal = Sum (Y- Ybar)²SSresidual = Sum (Y- Ŷ)² - the squared sum of errors in predictionsR² = SSreg/SStotal
ANOVA The Regression MEAN SQUARE : MSS(regression) = SS(regression) / df, df = k where k = no. of variables The MEAN square residual (error): MSS(residual) = SS(residual) / df, df= n - (k + 1) where n = no. of cases and k= no. of variables.
Hypothesis Testing with (Multiple) Regression F – test The null hypothesis for the regression model: Ho: b1 = b2 = … = bk = 0 MSS(model) • F = -------------- MSS(residual) The sampling distribution of this statistic is the F-distribution
t – test for the effect of each independent variable The Null Hypothesis for individual IVs The test of H0: bi = 0 evaluates whether Y and X are statistically dependent, ignoring other variables. We use the t statistic b • t = -------------- σB where σB is a standard error of B SS(residual) • σB = -------- n - 2
Assessing the importance of IVs • if IVs are uncorrelated w. each other: compare standardized coefficients (betas); higher absolute values of betas reflect greater impact; • if the IVs are correlated w. each other: compare total relation of the IV with the DV, and of IVs with each other using bivariate correlations; compare the unique contribution of an IV to predicting the DV = generally assessed through partial or semi-partial correlations In partial correlation (pr), the contribution of the other IVs is taken out of both the IV and the DV; In semi-partial correlation (sr), the contribution of the other IVs is taken out of only the IV (squared) sr shows the unique contribution of the IV to the total variance of the DV
Assessing the importance of IVs – continued In standard multiple regression, sr² = the unique contribution of the IV to R² in that set of IVs (for an IV, sr² = the amount by which R² is reduced, if that IV is deleted from the equation) If IVs are correlated: usually, sum of sri² < R² • the difference R² - sum of sri² for all IVs = shared variance (i.e. variance contributed to R² by 2/more variables) Sequential regression: sri² = amount of variance added to R² by each IV at the point that it is added to the model In SPSS output sri² is „R² Change” for each IV in „Model Summary” Table
Suppressor Variables = IV which helps predicting DV & increases R² due to its correlation with other IVs. • It suppresses variance that is irrelevant to prediction of DV • traditional/classical suppression; • cooperative/reciprocal suppression; • negative/net suppression Output: compare simple correlation btw. each IV & DV, with the standardized regression coefficient (beta weight) for the IV. If beta = significant, look if: • the absolute value of the simple correlation btw. IV and DV = much smaller than beta; • the simple correlation ceoff. & beta have opposite signs. If more than 2,3 IVs - difficult to identify suppressor.
Interaction Terms; centering • if reasonable to assume that the importance of IV1 varies over the range of IV2 interaction (compute IV1_2= IV1 * IV2). Centering:convert the IVs that form the interaction to deviation scores (Xi – Mean for X) each variable will have mean=0 Why do it? • possible problems w. multicollinearity • does not affect correlation w. other variables; • unstandardized regression coeff (bs) for the simple terms (b1, b2) are the same as when uncentered; • affects bs for interactions (& powers) of IVs included in the regression; • the betas (stadradized coeff) are different for all effects