American Community Survey Multi-Year Estimates: Challenges and Opportunities

1. American Community SurveyMulti-Year Estimates:Challenges and Opportunities Discussant II: Mike Cohen Study Director, CNSTAT September 25th, 2008

2. Intro Remarks • Thanks to Bob Parker • Extremely important topic • Remarks are provocative, unfortunately with substantial ignorance of the implications --- • I am mainly playing a “devil’s advocate” role --- hoping to initiate a productive discussion • I have benefitted from hearing discussions of the CNSTAT ACS panel, and discussions with Connie Citro and Dan Cork, but these remarks are my own (and I doubt that any of the above even agree with these remarks).

3. Intro Remarks • Congratulations to the Census Bureau --- the ASA is a success! • VERY remarkable given the small staff • We are in a 15 year learning curve for the ACS in which we will gradually understand the local dynamics of various statistics, how to collect higher quality data, and what the data products should be (point due to Connie Citro).

4. Intro Remarks • There is a tension between the need to innovate and the need for stability to support period estimates • Learning about the ACS will benefit from stability • Innovation argues against stability • So there is a need to learn how to introduce change to the ACS while producing a useful series of estimates over time • This likely means parallel release of new and modified new numbers, or more than that • This is one of the key reasons an ACS methods panel would be useful to have.

5. Current Multi-Year Strategy: Advantages and Disadvantages • Advantages 1. Estimates are relatively easy to compute 2. Estimates are internally consistent--- things add up

6. Current Multi-Year Strategy: Advantages and Disadvantages • Disadvantages 1. Release of competing estimates is confusing 2. CB provides period estimates, which is not really what users need --- they want current point-in-time estimates 3. No estimates of change are provided other than for one-year estimates [I don’t have a lot to add here today, though]

7. RECONCEPTUALIZATION OF ACS PUBLISHED ESTIMATES • Continue to produce one-year estimates for areas > 65 K, do not produce 3- or 5-year estimates for those areas • For areas > 20 K, < 65K, produce a point-in time current estimate based on 3-years of data but do not produce 5-year estimates • For areas < 20 K, only produce a point-in-time current estimate based on 5 years of data

8. RECONCEPTUALIZATION OF ACS PUBLISHED ESTIMATES • Why the distinction between large, medium, and small areas? Why not join three or five years of data using weights for all areas to produce point-in-time current estimates? • Response: • a.) Least disruptive • b.) Might be better for change estimation since uses shorter time periods whenever possible.

9. RECONCEPTUALIZATION OF ACS PUBLISHED ESTIMATES • Problem: These estimates will not add up • Solution: It ought to be relatively mild degree of inconsistency • Controls can be implemented if needed.

10. Research Program: Current Point-in-Time Small-Area Estimation • NEED FOR INNOVATIVE AND ENERGETIC RESEARCH PROGRAM ON SMALL-AREA ESTIMATION

11. Research Program: Current Point-in-Time Small-Area Estimation • Stage One: Exploit Time Structure (Differently) • Replace 3-year and 5-year estimates with weighted moving averages to give additional weight to the data from the current time period: • E.G. .2 Yt-2 + .3 Yt-1 + .5 Yt • Choose weights by balancing reduced time bias against increased sampling variance • Three-year research program • Problem: Do you allow different weighting schemes for different estimates?

12. Research Program: Current Point-in-Time Small-Area Estimation • Stage Two: Exploit Geographic Structure of the ACS • BUT ONLY FOR IMPORTANT SERIES • Borrowing information from nearby jurisdictions (justification: if the poverty rate of a county is going up, the same will often be true for nearby counties) • It would be useful if estimates of level would add Not an easy problem, but a LOT of work has been done on spatio-temporal models (Cressie, others at Ohio State) Seven-year research program

13. Research Program: Current Point-in-Time Small-Area Estimation • Stage Three: Exploit the multivariate structure of the ACS • BUT ONLY FOR IMPORTANT SERIES • Various statistics from the ACS are correlated (justification: changes in the unemployment rate and poverty rate are likely correlated, so either include each in each other’s small-area model, or model them simultaneously) • HARD problem, but would be important to investigate • Ten-year research program

14. Research Program: Current Point-in-Time Small-Area Estimation • Across last two stages of modeling --- use of covariate information will help a lot in these regression-type small-area models • 1.) Administrative records • 2.) Census data • 3.) Household survey data • Justification: e.g., CPS should be helpful in estimating the poverty rate.

15. Research Program: Current Point-in-Time Small-Area Estimation • General approach to much of this was outlined by Chand and Alexander (1995, 1996, 1997) “The American Community Survey (ACS) component of the continuous measurement is designed to provide reliable direct estimates of the various population characteristics for substate areas. For small areas, such as census tracts, it is desirable to improve the ACS estimates by borrowing strength from other areas and other sources of data. In this project we will develop procedures to derive indirect estimates of characteristics of interest by integrating ACS data with administrative records and the previous census data.” I don’t know about tracts, but counties should work….

16. Research Program: Current Point-in-Time Small-Area Estimation • Basically they are proposing a Fay-Herriot type estimate: --- a linear combination of the ACS direct estimate and a regression-type indirect estimate, each weighted by their relative precisions. • Chand and Alexander did not explicitly add the time series part, but those models were not well-developed ten years ago, and they are still under development. • Chand and Alexander address the problem of things not adding up with a relatively straightforward weighting technique

17. SUMMING UP • Summing UP • 1.) Use weights that increase the impact of more recent years to produce point-in-time current estimates • 2.) For selected series, exploit geographic, multivariate, and covariate structure to provide best possible small-area estimates • 3.) Only produce one set of estimates for each area which corresponds to the shortest period estimate currently produced

18. SUMMING UP • 1. This is the Holy Grail of small-area estimation problems --- it is extremely important and extremely interesting • 2. Besides producing higher quality and more useful estimates --- which is not a small consideration --- it could also attract new Ph.D.s, develop research relationships with academics and their students, ASA research fellows, etc., etc. The Census Bureau would lead the way on what will soon be an international challenge to statistical agencies • The Census Bureau has made a wonderful start with the ACS --- this may point the direction to an equally wonderful next phase. • Thanks for listening.