Steve Pudney Francesca Zantomio Institute for Social & Economic Research, University of Essex

1. Estimating the impact of a policy reform on welfare participation:The 2001 extension to the Minimum Income Guarantee for UK pensioners Steve Pudney Francesca Zantomio Institute for Social & Economic Research, University of Essex Ruth Hancock Department of Health & Human Sciences, University of Essex

2. Outline • Introduction • The MIG system and the 2001 reform • Data and entitlements simulation • Statistical Analysis • Results • Conclusions

3. “Non take-up” • People are observed NOT to claim means-tested welfare benefits to which they are entitled. • Take-up rate measures: by caseload: # claimants / # entitled benefit units by proportion of claimed government expenditure • The phenomenon seem to be particulary severe for pensioners in the UK

4. UK Pensioners Take-up • 2 million pensioners living in low income households • Government commitment to ensure a decent income in retirement • Welfare benefits (incl. state pension) make up the entire income of 15 % of pensioners • Still between a quarter and a third of entitled pensioners do not claim income support • Policy relevance

5. “Non take-up” rationale Qualitative research (Costigan et al.,1999; CAG, 2002) suggests that welfare participation involves claiming costs due to: Information and transaction costs • ignorance of the existence of the program • insufficient knowledge of the administrative procedures or entitlement criteria • language difficulties • troubles in filling the forms and gathering required information and documentation, • time, hassle… Intangible social stigma costs • distatste for welfare income • self-humiliating feeling • fear of stigmatisation from others • fear of losing independence

6. “Take-up” literature Social sciences literatures exploring various aspects of take-up behaviour (Kerr,1982; Hirsh and Rank 1999; Kayser and Frick,2001; Castranove et al., 2001) Economic literature (Moffit,1983; Blundell et al., 1988; Duclos, 1995; Pudney et al., 2006; Hernandez et al.,2006) • Claiming as utility maximizing choice: expected benefit vs claiming cost • Entitlement amount acts as positive incentive to take-up • Econometric approach:entitlemnts simulation and parametric modelling of take-up probability, estimation of stigma cost

7. Our approach Use of a policy change to pensioners income support system to identify theimpact of variation in entitlement on the take-up behaviour Non parametric analysis (misspecification) Comparison with traditional parametric approach Family Resorces Survey data ( samples of pensioners interviewed before and after the reform came into force)

8. The Minimum Income Guarantee (MIG)Income support for pensioners (60+), income and asset-tested • brings income up to a minimum level, B = G(X) – means • withdrawn at 100% as income rises • MIG in 2000/1: • assets between £3,000 & £8,000 converted to notional income • no eligibility if assets > £8,000 • system of age-related additions • Reformed system in 2001/2: • real increase in guarantee level • abolition of age additions • asset limits raised to (£6,000, £12,000) • Every eligible pensioner gained; some became newly-eligible

9. PRE-REFORM POST-REFORM

11. The evaluation problem • Rubin-type evaluation methods require an experimental implementation and a comparison group outside the experiments • Limitations: • assume a limited number of policy options • a non-universalist approach to policy that allows small-scale experiments • In the UK experiments are rare because these conditions aren’t fulfilled • lots of policy options are considered in advance of reform • there is a centralised or universalist approach to policy • policy is designed with a very short time horizon – by the time an experiment would be designed, implemented & evaluated, government has lost interest • The dominant approach remains “behavioural” modelling, which allows generalisation from past experience to hypothetical new policy environments • Raises issues of: • specification error • other modelling problems, such as measurement error

12. Our evaluation problem • want to identify the average treatment effect of the reform on the take-up of the treated • there can be no “control” group since: • all before-reform eligible people were affected • non-eligible people have no take-up behaviour to study • for any pensioner we observe the take-up behaviour only under one benefit regime (either before or after the reform) “Selection on observables” approach: Pre-reform interviewed pensionersare are matched in terms of pre-reform and post reform entitlements (and demographic characteristics) to post-reform interviewed pensioners and used as conterfactual

13. Family Resources Survey Data • Annual cross-section surveys for Department for Work & Pensions (DWP) • 2 years’ data, for t = 0 (FY 2000/1) & 1 (FY 2001/2) • Pensioner units at least 5 years over state retirement age, in single unit hholds, without mortgages/earned income • Extensive data cleaning of benefits data to minimize the scope for measurement error • Benefit entitlement is simulated under the pre-reform rules and post-reform rules • Final sample of entitled pensioners contains n0 = 845 in 2000/1 & n1 = 756 in 2001/2

14. Notation • Entitlement:Btr = amount of simulated MIG entitlement in circumstances of year t = 0, 1 under benefit system r = 0, 1 • Take-up outcome:Ttr = binary indicator of take-up status that would be observed in circumstances of year t = 0, 1 under benefit system r = 0, 1. Ttt is only meaningful if true Btt > 0 • Survey response:Rt = binary indicator of survey response that would be encountered in year t if selected for interviewing • We observe: {Xt , Ttt, Bt0, Bt1 | Rt = 1} for t = 0, 1 • Three population groups: • People ineligible before & after reform (Bt0 = 0 , Bt1 = 0) • People ineligible before & eligible after (Bt0 = 0 , Bt1 > 0) • People eligible before & after (Bt0 > 0 , Bt1 > 0) • Our concern is with response to change in generosity of the benefit system, so we concentrate on group 3

15. Change in take-up rate

16. Definitions & assumptions Causal impact of reform: Crude change in take-up rate Assumptions:A1 conditionally ignorable non-responce A1Rt Ttt | Wt = w, for t = 0, 1 and all w, where Wt = (Xt , Bt0, Bt1) A2 no counfunding macro level changes, conditional on Wt A2E(Ttr | Wt = w) independent of t for r = 0, 1 and all w Under these assumptions: = causal effect + effect of change in distribution of covariates

17. Matching approach • Matching on variables, not PSM: “selection” into year 0 & year 1 samples is random so no confounding except for non-response • stratification on Xt (sex, age, marital status, disability) • Nearest neighbour algorithm, distance minimization: • Mahalanobis distance measure on (Bt0 , Bt1) • matching with replacement • caliper options: reject matches breaching

18. Matching approach Stratum k estimator for change in take-up overall estimator of the reform-induced change

19. Results Impact with 2000/1 characteristics: Impact with 2001/2 characteristics:

20. Estimate of impact with 2000/1 characteristics: Estimate of impact with 2001/2 characteristics:

21. Matching quality Percentage of cases matched with D(i, j) < threshold

22. Parametric approach Probit model Predicted change in the take-up rate (eg When i is sampled in 2000/1)

24. Conclusions • Even without a real or natural experiment or a control group, it is worth trying to use nonparametric matching methods as a complement to parametric simulation models • The take-up of MIG was significantly increased by the reform • The effect was particularly large for those with largest potential gains from claiming • Overall, a conventional probit take-up model for the 2001 MIG reform for UK pensioners works OK compared to the nonparametric results; but there is some evidence of probit under-prediction of the response to reform by people facing large gains – with potentially important welfare implications • Next…

26. Take-up propensity Post-reform period Reform effect Publicity campaign Publicity effect Pre-reform period Time 2000 MIG publicity campaign

31. D(i, j) = (P0i − P1j)’V −1 (P0i − P1j)