1 / 50

Surveys: Collecting Policy Relevant Data

Surveys: Collecting Policy Relevant Data. Rachel Smith-Govoni September 17, 2007. Goals and Needs. Goals: Measure the poverty impact of economic policy Measure the distributional impact of economic policy Needs: Rely heavily on household survey data. Household Surveys - types.

spham
Download Presentation

Surveys: Collecting Policy Relevant Data

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Surveys: Collecting Policy Relevant Data Rachel Smith-Govoni September 17, 2007

  2. Goals and Needs Goals: • Measure the poverty impact of economic policy • Measure the distributional impact of economic policy Needs: • Rely heavily on household survey data

  3. Household Surveys - types • Single Topic • Labour Force Surveys( LFS) (ILO) BIH April 2006 and likely April 2008 • Housing Surveys • Census – national, 10 years – BIH 1991 • In-between • Multi-topic

  4. Household Surveys • Single Topic • In-between • Agricultural Surveys (FAO) • Demographic and Health (DHS) • Household Budget Surveys (HBS) BiH 2004 and 2007 • Multi-topic

  5. Household Surveys • Single Topic • In-between • Multi-topic • Multiple Indicator Cluster Survey UNICEF • BiH MICS2 1999, MICS3 2006 • Living Standards Measurement Study (LSMS) World Bank LSMS 2001 Living in BiH 2002, 2003, 2004 • Survey on Income and Living Conditions (SILC, EU)

  6. Census Purpose • Accurate measure of the population of a country • Geographic distribution of the population • Basic demographic information

  7. Census • Not a sample • Universal coverage • No sampling errors in estimates • Some corrections for non-response may be needed • Not many items

  8. Census Content • Demographic information: age, sex, race/ethnicity, family and household composition • Housing information • Others: basic education, labour, disability

  9. Census Albania: 2001 (1989) BiH 1991 (1981) Montenegro 2003 (1991) Serbia 2002 Kosovo 1981 Limited monitoring Limited use if looking at impact of policies affecting taxes, tariffs or pricing

  10. Census Uses • Sample frame • Link with household surveys for small area estimation

  11. BiH Master Samples • 2003 40,000 dwellings (HBS 2004) • 2006 40,000 dwellings • Total 80,000 used for LFS 2006….2008 MICS3 2006, HBS 2007

  12. Two types of errors: Sampling and non-sampling Time Cost Training Non-response

  13. Sampling vs. non-sampling errors Total error Sampling error Non-sampling error Sample size

  14. Labour Force Survey (Anketa o radnoj snazi – ARS) Purpose • Direct measurement of unemployment • General characteristics of the labour force

  15. Labour Force Survey Sample • Relatively large samples (9,000 2 weeks in April 2006) • Desire to disaggregate to different geographic areas • Individuals of working age

  16. Labour Force Survey Content • Characteristics of the labour force • Demographics • Education • Sectoral distribution of employment • Degree of formality • Seasonal • Income

  17. Labour Force Survey Limitations: • LFS typically capture partial, not total, income, under-estimate welfare • Measurement Error - Labour income measurement error at both ends of the distribution

  18. LFS in Latin AmericaItem non-response Source: Feres, 1998

  19. Household Budget Survey (Anketa o potrosnji domacinstava – APD, BiH 2004, 2007) • Inputs to National Accounts on consumer expenditures • Track changes in expenditures over time • Weights for the Consumer Price Index (Indeks Potrosackih Cijena)

  20. Non response rates (Eurostat Household Budget Surveys, 2003) • Bulgaria: 39.7% • Estonia, 44% • Hungary, 58.8% before replacement • Romania, 21.6 % Sample • Usually medium size sample – BiH larger than usual 7,500 interviews (700 a month) • High non-response rates

  21. Household Budget Surveys Content • Total Income • Total Consumption - diary • Short Demographics • Central Europe: agriculture • Limited health and education

  22. Household Budget Surveys Poverty Measurement • Consumption based welfare measure • Purpose of an HBS survey is NOT to measure welfare but to precisely measure mean expenditures on specific goods and services • These are conflicting goals

  23. Household Budget Surveys Poverty Measurement • Shortest possible reference periods • Minimize number of omitted expenditures • Good for precise measurement of regional or national means • Because of lumpy nature of purchases, not good for comparisons among households

  24. Multi-topic Household Surveys Those with a focus on measuring poverty • Survey on Income and Living Conditions (SILC) • Living Standards Measurement Study Surveys (LSMS)

  25. Multi-topic Household Surveys Purpose • Analysis of welfare levels and distribution • Study links between welfare levels and individual and household characteristics, economic, human and social capital • Social exclusion • Levels of access to, and use of, social services, government programs and spending

  26. Multi-topic Household Surveys Sample • Small sample sizes • Trade-off issue: Quality and cost considerations • Limits ability to assess programs or policies that affect small groups or small areas (over-sample) • Infrequent in many countries

  27. Zivjeti u BiH (Living in BiH)2001, 2002, 2003, 2004 Content 1 household composition 2 housing 3 individual demographics 4 health 5 labour 6 work history 7 social programs 8 migration 9 values and opinions 10 consumption 11 agriculture

  28. Multi-topic Household Surveys Poverty Measurement • Total consumption • Longer reference periods • Able to calculate use value of durables and housing • Total income • Suffers from standard measurement errors

  29. Designs for surveys across time Repeated cross sectional surveys (e.g. Household Budget Survey, Labour Force Survey) • Common design for large government surveys • New sample drawn for each survey • Carry similar questions each year • Used for trend analysis at aggregate level

  30. Designs for surveys across time Cohort Studies • Sample often based on an age group • Follow up same sample members at fairly long intervals • Developmental data as well as social and economic data • Data from parents, teachers associated with cohort member

  31. Designs for surveys across time e.g. Panel Study of Income Dynamics, USA – since 1968! Living in BiH 2001-2004, LSMS Albania 2002-2004, LSMS Serbia 2002-2003 • Draw a sample at one point in time and follow those sample members indefinitely (or as long as the funding continues) • Collect individual level data in household context • Repeated measures at fixed intervals (annual data collection)

  32. Advantages of Panel Data • Comparison of same individual over time - outcomes • Track of aspects of social change • Facilitates study of change and causal inference • Minimise the problem of inaccurate recall • Compare a person’s expectations with real change • Look at how changes in individuals’ behaviour affects their households • Identifies the co-variates of change and the relative risks of particular events for different types of people

  33. Net change - 0.1% unemployed 2001 2007 Changes in Employment Status A: CROSS-SECTIONAL INFORMATION Unemployed Employed

  34. 3.2% continuously unemployed 5.1% unemployed 2001 but employed 2007 5% employed 2001 but unemployed 2007 86.7% continuously employed Changes in Employment Status B: PANEL INFORMATION Still Unemployed Unemployed Employed Still Employed 2001 2007 Net change - 0.1% unemployed Actual change is 10.1

  35. Balkan Examples Albania - 15% of the unemployed in 2002 had made the transition to formal sector employment by 2004 BiH - About half who were poor in 2001 remained poor in 2004. Many individuals moved out of poverty. (Cross section headcount 18% for both years)

  36. Employment and the labour market • Unemployment duration and exit rates • Do the unemployed find stable employment? • The effect of non-standard employment on mental health • Temporary jobs: who gets them, what are they worth, and do they lead anywhere? • Family and Household • Patterns of household formation and dissolution • Breaking up - finances and well-being following divorce or split • The effect of parents’ employment on children's educational attainment

  37. Panel analysis Mobility, poverty and well-being among the informally employed – Peter Sanfey European Bank for Reconstruction and Development The origins of self employment, Leora Klapper et al, WB (soon to use Albania Panel also) The impact of health shocks on employment, earnings and household consumption, Kinnon Scott et al

  38. A Sample • Concept of ‘longitudinal household’ problematic for a panel - households change in composition over time or disappear altogether • Individual level sample

  39. Following rules • All members of households interviewed at Wave One • Children born to these original sample members • Original members are followed as they move house, and any new individuals who join with them are eligible to be interviewed • New sample members are followed if they split from the original member

  40. Questionnaire design • Core content carried every wave • Rotating core questions • One-off variable components • lifetime job history • marital and fertility history • Variable questions to respond to new research and policy agendas

  41. Attrition in panel surveys • Inevitable to some extent but can be minimised • Multiple sources of attrition in a panel • refusal to take part • respondents move and cannot be traced • non-contacts • Worry is potential bias if people who drop out differ significantly from those who stay in

  42. UK Panel Wave 1 RespondentsWave-on wave re-interview rates

  43. Fieldwork • respondent incentives as a ‘thank-you’ • extended fieldwork period for ‘tail-enders’ • refusal conversion programme • tracking procedures during fieldwork • panel maintenance between waves • Change of Address cards to update addresses • mailing of Respondent Report • details of contacts with respondents between waves

  44. The user database • Longitudinal data is complex • Provide users with database structure which enhances usability • Consistent record structure over time • Key variables for matching and linking data cross wave • Consistent variable naming conventions

  45. Conclusions • Longitudinal panel data allows us to answer research questions that cannot be answered with with cross-sectional data • Provides a different view of the world - see process through the life-course not just a static picture • Is complex (but so is the real world) - so needs to be well designed and conducted with sufficient resources to be successful

  46. Finalpoints • Welfare: household surveys- always missing the homeless, street children, institutionalized population • No one survey can meet all needs, review its purpose, coverage, content and quality before using • Need a system of surveys that meets the needs of data users

More Related