PROC ROBUSTRET & Evaluating Regression analyses With The Help of PROC RSQUARE

PROC ROBUSTRET & Evaluating Regression analyses With The Help of PROC RSQUARE Animal Science 500 Lecture No. 10 October 5, 2010

PROC ROBUSTREG • The purpose of robust regression is to detect outliers and provide stable results in the presence of outliers. • In order to achieve this stability, robust regression limits the influence of outliers. • Outliers can be classified as: • Problems with outliers in the y-direction (response direction) • Problems with multivariate outliers in the x-space (i.e., outliers in the covariate space, which are also referred to as leverage points) • Problems with outliers in both the y-direction and the x-space

PROC ROBUSTREG • Two types of estimations methods • M Estimation - is the method for outlier detection and robust regression when contamination is mainly in the response direction (y) • LTS Estimation - the method used when data contamination occurs in the x space.

PROC ROBUSTREGM-estimation • The following ROBUSTREG statements analyze the data: Proc Robustregdata=stack; model y = x1 x2 x3 / diagnostics leverage; id x1; test x3; run; quit;

PROC ROBUSTREG M-estimation Proc Robustregdata=stack; model y = x1 x2 x3 / diagnostics leverage; id x1; test x3; run; quit; The procedure does M estimation with the bisquare weight function (default), and it uses the median method for estimating the scale parameter. The MODEL statement specifies the covariate effects. The DIAGNOSTICS option requests a table for outlier diagnostics, The LEVERAGE option adds leverage point diagnostic results to this table for continuous covariate effects. The ID statement specifies that variable x1 is used to identify each observation in this table. If the ID statement is missing, the observation number is used to identify the observations (might even be better this way in some cases). Tests of significance for the covariate effects are obtained using the test line with a variable(s) listed with the test term. http://support.sas.com/onlinedoc/913/getDoc/en/statug.hlp/rreg_sect3.htm

PROC ROBUSTREG example outputM-estimation The ROBUSTREG Procedure Model Information Data Set WORK.STACK Dependent Variable y Number of Covariates 3 Number of Observations 21 Method M Estimation Summary Statistics Variable Q1 Median Q3 Mean Standard MAD Deviation x1 53.0000 58.0000 62.0000 60.4286 9.1683 5.9304 x2 18.0000 20.0000 24.0000 21.0952 3.1608 2.9652 x3 82.0000 87.0000 89.5000 86.2857 5.3586 4.4478 y 10.0000 15.0000 19.5000 17.5238 10.1716 5.9304 http://support.sas.com/onlinedoc/913/getDoc/en/statug.hlp/rreg_sect3.htm

PROC ROBUSTREG example output http://support.sas.com/onlinedoc/913/getDoc/en/statug.hlp/rreg_sect3.htm

PROC ROBUSTREG example outputM-estimation The ROBUSTREG Procedure http://support.sas.com/onlinedoc/913/getDoc/en/statug.hlp/rreg_sect3.htm

PROC ROBUSTREG example outputM-estimation http://support.sas.com/onlinedoc/913/getDoc/en/statug.hlp/rreg_sect3.htm

PROC ROBUSTREGLTS-estimation • The following statements invoke the ROBUSTREG procedure with the LTS estimation method. Proc Robustregdata=hbkfwls method=lts; model y = x1 x2 x3 / diagnostics leverage; Id index; run; quit;

PROC ROBUSTREGLTS-estimation • The following statements invoke the ROBUSTREG procedure with the LTS estimation method. Proc Robustregdata=hbkfwls method=lts; model y = x1 x2 x3 / diagnostics leverage; Id index; run; quit; http://support.sas.com/onlinedoc/913/getDoc/en/statug.hlp/rreg_sect4.htm

PROC ROBUSTREG Output LTS-estimation http://support.sas.com/onlinedoc/913/getDoc/en/statug.hlp/rreg_sect4.htm

PROC ROBUSTREG Output LTS-estimation The final weighted least squares estimates are shown. These estimates are least squares estimates computed after deleting the detected outliers. http://support.sas.com/onlinedoc/913/getDoc/en/statug.hlp/rreg_sect4.htm

PROC RSQUARE • The RSQUARE procedure selects optimal subsets of independent variables in a multiple regression analysis. • Regression coefficients and a variety of statistics useful for model selection can be printed or output to a SAS data set. • In SAS Version 6+, the RSQUARE procedure is subsumed by PROC REG.

PROC RSQUARE • General Form • PROC RSQUARE options; • MODEL dependents=independents/options; • FREQ variable; • WEIGHT variable; • BY variables; • Run; • Quit;

PROC RSQUARE • There must be one or more MODEL statements. • The FREQ, WEIGHT, and BY statements can appear only once. • The MODEL, FREQ, WEIGHT, and BY statements can appear in any order.

PROC RSQUARE Options • The following options can be specified in the PROC statement; • DATA=SASdataset • names the SAS data set to be used. • The data set can be an ordinary SAS data set or a TYPE=CORR, COV, or SSCP data set. If the DATA= option is omitted, RSQUARE uses the most recently created SAS data set. • SIMPLE|S • Prints means and standard deviations for every variable listed in a MODEL statement.

PROC RSQUARE Options • The following options can be specified in the PROC statement; • CORR|C • Pprints the correlation matrix for all variables in the analysis. • NOINT • suppresses the intercept term from all models. • NOPRINT • suppresses the regression printout • OUTEST=SASdataset • creates a TYPE=EST data set containing model-selection statistics and parameter estimates for the selected models.

PROC RSQUARE Options • The options listed in the MODEL Statement section can also be used in the PROC RSQUARE statement. • Any option specified in the PROC statement applies to every MODEL statement except those in which you specify a different value of the option. • Optional statistics will appear in the OUTEST= data set only if the corresponding options are specified in the PROC statement

PROC RSQUARE Model Statement Options • MODEL dependents=independents/options; • Options are listed after a forward slash that follows the model statement • SELECT=n • Specifies the maximum number of subset models of each size to be printed or output to the OUTEST= data set. • If SELECT= is used without the B option, the variables in each MODEL are listed in order of inclusion instead of the order in which they appear in the MODEL statement. • If SELECT= is omitted and the number of regressors is less than 11, all possible subsets are evaluated. • If SELECT= is omitted and the number of regressors is greater than 10, the number of subsets selected is at most equal to the number of regressors. A small value of SELECT= greatly reduces the CPU time required for large problems.

PROC RSQUARE Model Statement Options • MODEL dependents=independents/options; • Options are listed after a forward slash that follows the model statement • INCLUDE=i • Requests that the first i variables after the equal sign in the MODEL statement be included in every regression model. • The default status = no variables are required to appear in every model. • START=n • Specifies the smallest number of regressors to be reported in a subset model. The default value is one more than the value specified by the INCLUDE= option, or one if INCLUDE= is omitted.

PROC RSQUARE Model Statement Options • MODEL dependents=independents/options; • Options are listed after a forward slash that follows the model statement • STOP=n • Specifies the largest number of regressors to be reported in a subset model. The default is the number of regressors listed in the MODEL statement. • ADJRSQ • Computes r-square adjusted for degrees of freedom for each model selected. • CP • Computes Mallows' Cp statistic for each model selected.

PROC RSQUARE Model Statement Options • MODEL dependents=independents/options; • Options are listed after a forward slash that follows the model statement • JP • Computes Jp, the estimated mean square error of prediction for each model selected assuming that the values of the regressors are fixed and that the model is correct. • The Jp statistic is also called the final prediction error (FPE). • MSE • Computes the mean square error for each model selected. • SSE • Computes the error sum of squares for each model selected. • B • Computes estimated regression coefficients for each model selected

PROC RSQUARE Model Statement Options • MODEL dependents=independents/options; • The FREQ Statement can also be used in this syntax • The use of FREQ in this sense treats the data set as if each observation appears n times where n is the value of the FREQ variable for the observation. • The total number of observations will be considered equal to the sum of the FREQ variable when the procedure determines the df when calculating significance probabilities. • PROC RSQUARE options; • MODEL dependents=independents/options; • FREQ variable; • WEIGHT variable; • BY variables; • Run; • Quit;

PROC RSQUARE Model Statement Options • MODEL dependents=independents/options; • The FREQ Statement can also be used in this syntax • If your data set includes a variable indicating the frequency of occurrence for other values in the observation, you would include this variables name beside the Freq statement. • PROC RSQUARE options; • MODEL dependents=independents/options; • FREQ variable; • WEIGHT variable; • BY variables; • Run; • Quit;

PROC RSQUAREModel Statement Options • MODEL dependents=independents/options; • The WEIGHT Statement can also be used in this syntax • The WEIGHT statement names a variable in the input data set whose values are relative weights for a weighted least-squares fit. If the weight value is proportional to the reciprocal of the variance for each observation, then the weighted estimates are the best linear unbiased estimates (BLUE). • The WEIGHT and FREQ statements have similar effects, except in the calculation of degrees of freedom. BY Statement • PROC RSQUARE options; • MODEL dependents=independents/options; • FREQ variable; • WEIGHT variable; • BY variables; • Run; • Quit;

PROC RSQUARE Model Statement Options • MODEL dependents=independents/options; • The BY variable can be used in this syntax • The BY statement can be used with PROC RSQUARE • Will result in separate analyses on observations in groups defined by the BY variables. • When a BY statement appears, the procedure expects the input data set to be sorted in order of the BY variables. • If the data has not been sorted previously in ascending order, • Use PROC SORT procedure with a similar BY statement to sort the data, • Or might be appropriate to use the option NOTSORTED • or DESCENDING if data was previous sorted in the largest to smallest value for some other reason previously. • Most likely you will need to sort the data

PROC RSQUARE Model Statement Options PROC SORT DATA=New by variable1; Run; Quit; PROC RSQUARE options; MODEL dependents=independents/options; FREQvariable; WEIGHT variable; BY variables; Run; Quit;

PROC RSQUARE • What we are building toward using PROC RSQUARE is building the best model or most predictive model. • Topic of next lecture Model Development and Selection of Variables

PROC ROBUSTRET & Evaluating Regression analyses With The Help of PROC RSQUARE