Launching Adaptive Design at the US Census Bureau

1. Launching Adaptive Design at the US Census Bureau Presented by Michael Thieme, Chief, Center for Adaptive Design Peter Miller, Chief Scientist, Center for Adaptive Design Discussion with the Council of Professional Associations on Federal Statistics December 7, 2012

2. Center for Adaptive Design Overview This new Center works in three areas: Research Through research, experimentation, and testing, prove that adaptive design principles can be effective at increasing survey and census efficiency and timeliness Outreach and Education Using results from research and testing, communicate and demonstrate to survey sponsors and stakeholders that implementing adaptive design is a viable tool for increasing survey and census efficiency and timeliness Design and Build Develop a solution architecture, as well as a strategy for building and deploying the systems that enable near-real-time cost–error trade-off decisions in data collection operations

3. CAD Research Activities for 2012-2013 • 1. Assembling and assessing data requisite for adaptive design • Auxiliary frame data • Paradata • Survey response data

4. CAD Research Activities for 2012-2013 • 2. Test uses of Frame Enhancements and Paradata in Survey Management • Propensity models in simulation • Business rules in simulation • Business rules in the field • Testing grounds: National Health Interview Survey (NHIS), National Survey of College Graduates (NSCG), others

5. CAD Outreach and Education • Trainings and follow-up for survey teams • Development of custom dashboards to display key survey metrics • Work with Field on implementation issues • Publicize activities and lessons

6. Design and Build • Documenting our current architecture to inform where Adaptive Design can be applied • Developing adaptive design overall conceptual solution architecture, as well as initial baseline architecture (Spring 2013) • Target for initial baseline is to have a functioning system in place for the 2015 American Community Survey (ACS) and the 2014 Company Organization Survey/Annual Survey of Manufacturers (COS/ASM)

7. Communication About Adaptive Design • Need to show that adaptive design is not cutting corners • Need to show that it saves money • Need to show that it does not adversely affect estimates, time series, or the job of the analyst • Need to show a comparable product for the money expended

8. Precursors to Adoption • Flexibility to give adaptive design a chance to show whether it works or not • Openness to consideration of alternative quality measures (quality of estimates vs. response rate) • Collaborative strategies for testing adaptive design in a variety of surveysacross government

9. Adaptive Design Challenges • Legislative and political considerations • Use of administrative data and paradata • Alternative contact methods • Stopping data collection early • The Decennial Census, the American Community Survey, etc.

10. Adaptive Design Challenges • Transition between fixed and adaptive survey designs at Census: • Mode-switching based on analysis of paradata and modeling • 2015 – American Community Survey (ACS), Company Organization Survey/Annual Survey of Manufacturers (COS/ASM) • Maturing and implementing Adaptive Design business rules (including stopping rules) • 2016 – ACS, testing for 2020 Census, 2017 Economic Census • 2017 – Economic Census • 2020 – Decennial Census

11. Moving Forward • Take risks now • Adaptive design is a potential remedy for declining budgets and rising nonresponse • Explore the potential so that when budgets decline we have an understanding of how use it

12. Questions and Discussion