1 / 45

The Evolution of a Phoenix

This article provides an update on the reengineering of the Survey of Income and Participation (SIPP) by the US Census Bureau, including the development of the Event History Calendar (EHC) instrument and the use of administrative records data. It highlights the importance of SIPP in evaluating and improving government programs and its value as a source of information on income dynamics and program participation.

adamselizabeth
Download Presentation

The Evolution of a Phoenix

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Understanding Event History Calendars(Update on Reengineering SIPP) COPAFSMarch 5, 2010David JohnsonJason FieldsUS Census Bureau

  2. The Evolution of a Phoenix

  3. Congresswomen Maloney hails 25th Anniversary of SIPP, Census Bureau’s Survey of Income and Participation "October, 2008 marks 25 years of SIPP data collection. The vital data collected by career professionals at the Census Bureau allows for the evaluation of the efficiency and effectiveness of government programs and gives us a more robust picture of how well we are doing as a nation in helping families progress through tough economic challenges" “The SIPP allows Congress to allocate scarce government resources and save tax dollars. It’s fitting that during this national economic crisis we draw attention to this important diagnostic tool which helps us understand how we can best provide assistance to families in need.”

  4. The Unique Value of SIPP • To provide a nationally representative sample for evaluating: • annual and sub-annual dynamics of income • movements into and out of government transfer programs • family and social context of individuals and households • interactions between these items

  5. National Academy of SciencesNational Research CouncilCommittee on National StatisticsPanel ReportReengineering the Survey of Income and Program ParticipationConstance F. Citro and John Karl Scholz, Editors July 2009

  6. CNSTAT Report – Importance of SIPP Conclusion 2-1: The Survey of Income and Program Participation is a unique source of information for a representative sample of household members on the intrayear dynamics of income, employment, and program eligibility and participation, together with related demographic and socioeconomic characteristics. This information remains as vital today for evaluating and improving government programs addressed to social and economic needs of the U.S. population as it did when the survey began 25 years ago.

  7. Our work on SIPP Improvements • Improve Processing System and Collection Instrument • Develop Event History Calendar (EHC) Instrument • Examine use of administrative records data to supplement and evaluate survey data • Continue meetings with stakeholders, development of survey content, and use of reimbursable supplements

  8. Statements from the CNSTAT Report: On EHC methodology • As discussed in Belli (1998), in an event history calendar, “respondents are encouraged to consider various events that constitute their personal pasts as contained within broader thematic streams of events. Not only can respondents not the interrelationship of events within the same themes (top-down and sequential retrieval) but, depending on which themes are represented by the calendar, respondents can also not the interrelationships among events that exist with different themes (parallel retrieval).”

  9. Timeline for SIPP Development 2007 2008 2009 2010 2011 2012 Sep --- Jan --- May --- Sep --- Jan --- May --- Sep --- Jan --- May --- Sep --- Jan --- May --- Sep --- Jan --- May --- Sep --- Jan 2013 SIPP 2004 Data Gap SIPP 2008 Panel – Waves 1 – 10 collection Waves 11 – 13 SIPP 2004 Panel data release SIPP 2008 Panel – Waves 1 – 13 data release 2008 paper EHC Eval. Analysis 2009 SIPP Re-Engineering Instrument Dev. Systems Tests - Preparation Field Activities Processing and Evaluation 2009 Re-engineered SIPP automated Prototype Reference Period Field Act. 2nd automated prototype Reference Period 2012/13 SIPP Re-Engineering Instrument Refinement Systems Tests - Preparation Field Activities 2013 Reengineered SIPP Reference Period

  10. Re-engineered SIPP – Progress Update – COPAFS March 5, 2010

  11. Current SIPP Basics National panel survey – Since 1984 with sample size between about 11,000 and 45,000 interviewed households The duration of each panel varies from 2½ yrs to 4 yrs The SIPP sample is a multistage-stratified sample of the U.S. civilian noninstitutionalized population The survey uses a 4-month recall period – 3 interviews / year The sample is divided into 4 rotation groups for monthly interviewing Interviews are conducted by personal visit and by decentralized telephone

  12. EHC Interviewing Human Memory - structured/organized - links and associations EHC Exploits Memory Structure - links between the occurrence and timing of events EHC Encourages Active Assistance to Rs - flexible approach to help elicit an autobiographical “story”

  13. Evaluations of EHC Methods Many EHC vs. “Q-List” Comparisons - various methods - in general: positive data quality results BUT, Important Research Gaps - data quality for need-based programs? - comparison to 4-month reference period?

  14. SIPP Re-engineering Field Test Plans • Proof of concept test • - 2008 paper and pencil reinterview test • EHC CAPI test • - 2010 Integrated Blaise and C# instrument prototype • CAPI Revised test • - 2011 Test improvements to the wave 1 instrument, training, and expand sample to all regional offices. • - 2012 Test wave 2 concepts and instrument, examine movers and attrition issues, and refine training procedures.

  15. 2008 Paper Field Test Goals & Design Basic Goal: Can an EHC interview collect data of comparable quality to standard SIPP? - month-level data - one 12-month ref pd interview vs. three 4-month ref pd interviews - especially for need-based programs Basic Design: EHC re-interview of SIPP sample HHs Approximately 2000 HHs in IL and TX

  16. Results Summary 3 Patterns: 1. EHC = SIPP All Year equivalent data quality

  17. SSI -- % Participation for Each Month of CY2007 According to the SIPP and EHC Reports Analysis Summary - no “main effect” for method (SIPP = EHC) - no significant method difference in any month

  18. 3 Patterns: 1. EHC = SIPP All Year SSI; WIC (IL) Results Summary

  19. 3 Patterns: 1. EHC = SIPP All Year SSI; WIC (IL) 2. EHC < SIPP All Year Results Summary

  20. 3 Patterns: 1. EHC = SIPP All Year SSI; WIC (IL) 2. EHC < SIPP All Year Medicare; Social Security; WIC (TX); Food Stamps (IL) Results Summary

  21. 3 Patterns: 1. EHC = SIPP All Year SSI; WIC (IL) 2. EHC < SIPP All Year reduced EHC data quality, but not due to longer recall period Results Summary

  22. SOCIAL SECURITY -- % Covered in Each Month of CY2007 According to the SIPP and EHC Reports Analysis Summary - significant “main effect” for method (SIPP > EHC) - method difference is constant across months

  23. 3 Patterns: 1. EHC = SIPP All Year SSI; WIC (IL) 2. EHC < SIPP All Year Medicare; Social Security; WIC (TX); Food Stamps (IL) Results Summary

  24. 3 Patterns: 1. EHC = SIPP All Year SSI; WIC (IL) 2. EHC < SIPP All Year Medicare; Social Security; WIC (TX); Food Stamps (IL) 3. EHC < SIPP, Early in the Year Only Results Summary

  25. 3 Patterns: 1. EHC = SIPP All Year SSI; WIC (IL) 2. EHC < SIPP All Year Medicare; Social Security; WIC (TX); Food Stamps (IL) 3. EHC < SIPP, Early in the Year Only Food Stamps (TX); TANF (TX); employment; school enrollment Results Summary

  26. 3 Patterns: 1. EHC = SIPP All Year SSI; WIC (IL) 2. EHC < SIPP All Year Medicare; Social Security; WIC (TX); Food Stamps (IL) 3. EHC < SIPP, Early in the Year Only EHC data quality may suffer due to longer recall period Results Summary

  27. FOOD STAMPS (Texas Only) -- % Participation for Each Month of CY2007 According to the SIPP and EHC Reports Analysis Summary - no significant “main effect” for method - BUT significant variation by month -- JAN-MAY: SIPP > EHC later months: no difference (reversal?)

  28. FOOD STAMPS (Texas Only) -- % Participation for Each Month of CY2007 According to the SIPP and EHC Reports and ADRECS

  29. 3 Patterns: 1. EHC = SIPP All Year SSI; WIC (IL) 2. EHC < SIPP All Year Medicare; Social Security; WIC (TX); Food Stamps (IL) 3. EHC < SIPP, Early in the Year Only Food Stamps (TX); TANF (TX); employment; school enrollment Results Summary

  30. 2008 Paper Field Test Overall Summary Successful “Proof of Concept” Overwhelming Finding: SIPP-EHC Agreement Valuable Lessons to Inform Next Test - larger, broader sample - “correct” timing of field period - automated questionnaire Specific Data Comparisons are Instructive

  31. 2010 EHC CAPI Test Goals & Design Basic Goal: Can a CAPI Event History Calendar interview be: - developed in-house (new/unknown requirements) - integrated with survey management systems - administered by regular field staff interviewers Develop and test new processing system Determine the comparability of data collected: - month-level data - (1) 12-month ref pd intvw vs. (3) 4-month ref pd intvws - especially for need-based programs Basic Design: 8000 HHs interviewed in 10 states limited to high poverty strata that can be matched to areas in 2008 Panel SIPP data. Recording 400-500 of the completed interviews

  32. 2010 EHC CAPI Instrument

  33. Current Status Interviewing in 6 of 12 regional office areas (10 States) Interviews continue through March 13. Recording 400-500 of the completed interviews – transcription will begin shortly. Reviewing and making changes to content and design. Reviewing and revising training materials and methods Planning for 2013 Production implementation

  34. Mockup (example of possible changes) for 2011 EHC CAPI Instrument DEPENDENT DATA FILL UP TO LAST WAVES ‘INTV’ MONTH REFERENCE YEAR 20XX INTERVIEW YEAR 20XX+1 LANDMARKS INTV RESIDENCES FILL’S ARE USED TO LABEL INTERVIEW MONTH STATUS OR NAMES FROM JOBS/PROGRAM NAMES MARITAL STATUS – (CURRENT MARITAL STATUS fill) PRESENCE OF PARENT - MOM PRESENCE OF PARENT - DAD SCHOOL ENROLLMENT – (CURRENTLY ENROLLED fill) JOBS/BUSINESSES – (fill name of emp/bus from ehc as entered) + Job/Bus 2 – (fill name of emp/bus from ehc as entered) DATA ARE RECORDED UP TO THE MONTH OF INTERVIEW, WE ALLOW UP TO 6 MONTHS FROM THE END OF THE CALENDAR YEAR TO CONDUCT INTERVIEWS – MONTHS BEYOND THE INTERVIEW ARE GREY’ED OUT. + Job/Bus 3 – (fill name of emp/bus from ehc as entered) + Job/Bus 4 – (fill name of emp/bus from ehc as entered) + Job/Bus 5 – (fill name of emp/bus from ehc as entered) ALL OTHER WORK FOR PAY TIME NOT WORKING SSI - SUPPLEMENTAL SECURITY INCOME SNAP - FOOD STAMPS/SUPPLEMENTAL NUTRITION ASSISTANCE PRGM TANF - TEMPORARY ASSISTANCE FOR NEEDY FAMILIES GA – GENERAL ASSISTANCE PROGRAM WIC – WOMEN, INFANTS AND CHILDREN PROGRAM HEALTH INSURANCE – PRIVATE COVERAGE + ADDITIONAL PRIVATE COVERAGE MEDICARE COVERAGE MEDICAID COVERAGE – (fill interview state program name) MILITARY COVERAGE – (VA, CHAMPUS, CHAMPVA) OTHER HEALTH INSURANCE COVERAGE TIME WITHOUT HEALTH INSURANCE COVERAGE IF YES TO ‘NOW’ INITIALIZE FIRST PERIOD AND FILL INTERVIEW MONTH IN THE ‘TO’ BOX AND SET FOCUS ON THE ‘FROM’ BOX (FILL NAME OF PERSON FOR THIS EHC) Topic – (FILL TOPIC LABEL) * INTRO TO TOPIC QUESTION FILL – ASKS ABOUT STATUS NOW OR DIRECTS FR TO PROCEED [ ] (1) YES/PROCEED (2) NO (3) REFUSED * FOLLOW-UP ASKS ABOUT EVER DURING REFERENCE YEAR OR DIRECTS FR TO PROCEED [ ] (1) YES/PROCEED (2) NO (3) REFUSED * IDENTIFY NEW PERIOD OF TIME [ ] 0-NEW [F4 – EDIT SPELL] [CTRL-X DELETE SPELL] * WHEN DID THIS PERIOD OF (TOPIC FILL) END? [ ] CHOOSE MONTH * WHEN DID THIS PERIOD OF (TOPIC FILL) BEGIN? [ ] CHOOSE MONTH ANY MORE PERIODS OF (TOPIC FILL) DURING REFERENCE YEAR? [ ] (1) YES (2) NO RECORDING ‘NO’ ADDS STRIKEOUT TO ANY UNASSIGNED MONTHS IN THIS TOPIC – ANSWERING ‘YES’ CYCLES BACK TO NEW PERIOD – UPON SPELL COMPLETION YOU EXIT THE DETAILED QUESTIONS TO THE ‘ANY MORE’ ITEM.

  35. Assessing Users’ Needs URL: http://www.census.gov/sipp

  36. Comments: Jason Fields – Jason.M.Fields@Census.Gov

More Related