1 / 71

Topic 1

Topic 1. Binary Logit Models. Often variables in social sciences are dichotomous: Employed vs. unemployed; Married vs. unmarried; Guilty vs. innocent; Voted vs. didn’t vote. Social scientists frequently wish to estimate regression models with a dichotomous dependent variable

carrie
Download Presentation

Topic 1

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Topic 1 Binary Logit Models

  2. Often variables in social sciences are dichotomous: • Employed vs. unemployed; • Married vs. unmarried; • Guilty vs. innocent; • Voted vs. didn’t vote

  3. Social scientists frequently wish to estimate regression models with a dichotomous dependent variable • Most researchers are aware that There is something wrong with using OLS for a dichotomous dependent variable; But they do not know what makes dichotomous dependent variable problematic in standard linear regression; and What other methods are superior

  4. Focus of this topic is on logit analysis (or logistic regression analysis) for dichotomous dependent variable • Logit models have many similarities to OLS regression models • Examine why OLS regression run into problems when the dependent variable is dichotomous

  5. Example • Dataset: penalty.txt • Comprises 147 penalty cases in the state of New Jersey • In all cases the defendant was convicted of first-degree murder with a recommendation by the prosecutor that a death sentence be imposed • Penalty trial is conducted to determine if the defendant should receive a death penalty or life imprisonment

  6. The dataset comprises the following variables: DEATH 1 for a death sentence 0 for a life sentence BLACKD 1 if the defendant was black 0 otherwise WHITVIC 1 if the victim was white 0 otherwise SERIOUS – an average rating of seriousness of the crime evaluated by a panel of judges, ranging from 1 (least serious) to 15 (most serious)

  7. DATA PENALTY; INFILE ‘D:\TEACHING\MS4225\PENALTY.TXT; INPUT DEATH BLACKD WHITVIC SERIOUS CULP SERIOUS2; PROC REG; MODEL DEATH=BLACKD WHITVIC SERIOUS; RUN;

  8. Remarks on OLS regression output: • The coefficient for SERIOUS is positive and very significant • Neither of the two racial variables are significantly different from zero • R2 is very low • F-test indicates overall significance of the model • Should we trust these results?

  9. Assumptions of the linear regression model: • (homoscedasticity) • (absence of autocorrelation) • ’s are treated as fixed • ~ Normal • If assumptions 1-5 are satisfied, then OLS estimators of a and b are B.L.U. • If all assumptions are satisfied, then OLS estimators of a and b are M.V.U.

  10. Now, what if y is a dichotomy with possible values of 1 or 0? • It is still possible to claim that assumptions 1, 2, 4 and 5 are true • But if 1 and 2 are true then 3 and 6 are necessarily false!!

  11. Consider assumption 6 • Note that If • It is obvious that cannot be normally distributed. In fact, it follows a Binomial distribution • So and are also not normally distributed. Standard inference procedures are no longer valid as a consequence • But in large samples, Binomial distribution tends towards the Normal distribution

  12. Consider assumption 3: • Note that • But from Assumptions 1 and 2, • Therefore,  Linear probability model (LPM)

  13. Accordingly, from our previous output, a 1-point increase in the SERIOUS scale is associated with a 0.038 increase in the probability of a death sentence; the probability of a death sentence for black defendants is 0.12 higher than for non-black defendants, ceteris paribus

  14. So, must be heteroscedastic. The disturbance variance is at a maximum when

  15. So, what are the consequences? • Violation of assumptions 3 and 6 does not lead to biased estimation by OLS (only assumptions 1 and 2 are required for OLS to yield unbiased estimators) • If the sample size is large enough, the estimators will be approximately normal even when are not normally distributed. • Voilation of the homoscedasticity assumption makes the OLS estimators no longer efficient. In addition, the estimated standard errors are biased.

  16. Also, the model is implausible, because is a linear function of and therefore has no upper or lower bound. But it is impossible for the true values (which are probabilities) to be greater than 1 or less than 0!

  17. Odds of an event: the ratio of the expected number of times that an event will occur to the expected number of times it will not occur, (e.g. an odds of 4 means we expect 4 times as many occurrences as non-occurrences; an odds of 5/2 (or 5 to 2) means we expect 4 occurrences to 2 non-occurrences.

  18. Let p be the probability of an event and 0 the odds of the event, then or

  19. Relationship between Odds and Probability Probability Odds 0.1 0.11 0.2 0.25 0.3 0.43 0.4 0.67 0.5 1.00 0.6 1.50 0.7 2.33 0.8 4.00 0.9 9.00 • o<1 => p<0.5 • o>1 => p>0.5 • 0 < o < ∞

  20. Death Sentence by Race of Defendant for 147 Penalty Trials. Blacks non-blacks Death 28 22 50 Life 45 52 97 Total 73 74 147 ∴ Ratio of black-odds to non-black odds are: => The odds of death sentence for blacks are 47.6% higher than for non-blacks, or the odds of death sentence for non-blacks are 0.63 times the corresponding odds for non-blacks.

  21. Logit model: which is the cumulative logistic distribution function. Let , then Notes: • As Zi ranges from -∞ to +∞, Pi ranges between 0 and 1; • Pi is non-linearly related to Zi

  22. Also, (the odds of the event) Let Although Li is linear in Xi, the probabilities themselves are not. This is in contrast to LPM.

  23. Graph of logit model for a single explanatory variable Pi (produce a graph using a = 0 and b = 1) 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

  24. Now • As f(a+bXi) is always positive, the sign of b indicates the direction of relationship between pi and Xi

  25. For the LOGIT model, ∴ In other words, a 1-unit change in Xi does not produce a constant effect on pi

  26. Note that yi only takes on values of 0 and 1, so Li is not defined. Therefore, OLS is not an appropriate estimation technique. Maximum Likelihood (ML) estimation is usually undertaken. ML basic principle: to choose as estimates those parameter values which would maximize the probability of what we have in fact observed.

  27. Steps: • Write down an expression for the probability of the data as a function of the unknown parameters [construction of likelihood function] • Find the values of the unknown parameters that make the value of this expression as large as possible.

  28. Taking the derivatives of log L and setting them to zero give:

  29. The first order conditions are non-linear in and , so solutions are typically obtained by iterative methods.

  30. Newton-Raphson algorithm Let U(a,b) be the vector of first derivatives of log L with respect to a and b and let I(a,b) be the corresponding matrix of the second derivatives. i.e. gradient or score vector

  31. Hessian matrix

  32. The Newton-Raphson algorithm derives new estimates based on where is the inverse of • In practice, we need a set of starting values. [PROC LOGISTIC starts with all coefficients equal to zero]

  33. The process is repeated until the maximum change in each parameter estimate from one step to the next is less than some criterion. i.e. and

  34. Note that • This variance-covariance matrix can be obtained using the COVB option in the MODEL statement in SAS

  35. SAS Program DATA PENALTY; INFILE ‘D:\TEACHING\MS4225\PENALTY.TXT’; INPUT DEATH BLACKD WHITVIC SERIOUS CULP SERIOUS2; PROC LOGISTIC DATA=PENALTY DESCENDING; MODEL DEATH=BLACKD WHITVIC SERIOUS; RUN;

  36. Interpretation of results • Rather than a t-statistic SAS reports a Wald Chi-square value, which is the square of the usual t-statistic. Reason: the t-statistic is only an asymptotic one and has an “asymptotic” N(0,1) distribution under null. The square of a N(0,1) is a chi-square random variable with one df.

  37. Test of overall significance • Likelihood-Ratio test: • Score (Lagrange-Multipler) test • Wald test:

  38. Model Selection Criteria • Akaike’s Information Criterion AIC = -2 ln L + 2 *(k+1) • Schwartz Criterion SC = -2 ln L + (k+1)*ln(n) 3. Generalized R2= analogous to conventional R2 used in linear regression

  39. Optimization Technique • Fishers’ scoring (Iteratively reweighted least squares) – equivalent to Newton-Raphson algorithm.

  40. Odds ratio = eb • The (predicted) odds ratio of 1.813 indicates that the odds of a death sentence for black defendants are 81% higher than the odds for other defendants • The (predicted) odds of death are about 29% higher when the victim is white. (But note that the coefficient is insignificant)

  41. a 1-unit increase in the SERIOUS scale is associated with a 21% increase in the predicted odds of a death sentence.

  42. Association of predicted probabilities and observed responses Example: For the 147 observations in the sample, there are 147C2= 10731 ways to pair them up (without pairing an observation with itself). Of these, 5881 pairs have either both 1’s on the dependent variable or both 0’s. These we ignore, leaving 4850 pairs in which one case has a 1 and the other case has a zero. For each pair, we ask the question “Does the case with a 1 have a higher predicted value (based on the model) than the case with a 0?

  43. If yes, we call that pair concordant; if no, we call that pair discordant; if the two cases have the same predicted value, we call it a tie. Let C = number of concordant pairs; D = number of discordant pairs; T = number of ties N = total number of pairs (before eliminating any)

  44. All 4 measures vary between 0 and 1 with large values corresponding to stronger associations between the predicted and observed values

  45. An Illustrative example of LOGIT model Table 12.4 of Ramanathan (1995) presents information on the acceptance or rejection to medical school for a sample of 60 applicants, along with a number of their characteristics. The variables are as follows: Accept =1 if granted an acceptance, 0 otherwise; GPA = cumulative undergraduate grade point average BI0 = score in the biology portion of the Medical College Admission Test (MCAT); CHEM = score in the chemistry portion of the MACT; PHY = score in the physics portion of the MCAT; RED = score in the reading portion of the MCAT;

  46. PRB = score in the problem portion of the MCAT; QNT = score in quantitative portion of the MCAT; AGE = age of applicant; GENDER = 1 if male, 0 if female; • Estimate a LOGIT model for the probability of acceptance into medical school • Predict the probability of success of an individual with the following characteristics GPA = 2.96 BIO = 7 CHEM = 7

  47. PHY = 8 RED = 5 PRB = 7 QNT = 5 AGE = 25 GENDER = 0 • Calculate Cragg and Uhler’s pseudo R2 for the above model. How well does the model appear to fit the data? • AGE and GENDER represent personal characteristics. Test the hypothesis that AGE and GENDER jointly have no impact on the probability of success.

More Related