American Community Survey

1. American Community Survey ACS Content Review Webinar State Data Centers and Census Information Centers James Treat, ACSO Division Chief December 4,2013

2. Contents • Overview • Challenges • Survey Improvements • Content Review Process • Questions for Discussion

3. Overview • Budget • Original allocation $241 million • Sequestration level $225 million • Fully launched in 2005 • Annually sample 3.5 million addresses • Release schedule • 1-year – September • 3-year – October • 5-year – December • Annually release 11 billion estimates • 4 modes of data collection: Internet, Mail, Telephone, Person Follow-up • Internet launched December 2012

4. Challenges • Concerns from respondents/public about perceived intrusiveness and burden of survey have made its way to Congress • As a result, some in Congress want to make the ACS voluntary • Lack of general awareness of ACS

5. Survey Improvements • Added on-line response option • Developed and implemented an interactive survey form to improve respondent cognition and navigation • Modified existing questions on the survey form to reduce the number of response categories • Provided clarification of questions via various communication mechanisms • Reduced telephone contacts to respondents from 25 to 15 • Planned research to explore reducing person follow-up contacts with respondents • Developed an integrated communications plan includes strategies to raise awareness and importance of ACS • Improving usefulness of data products

6. Content Review Process Background: • In December 2010, the Census Bureau began a comprehensive assessment of the ACS program – including an initiative to examine and confirm the value of each question on the ACS. • In June 2012, OMB initiated a process to confirm and update information the statutory and regulatory authority for the questions on the ACS with select federal agencies. • In August 2012, the Interagency Council of Statistical Policy (ICSP) Subcommittee for the ACS was formed with a mission, in part, to ensure that the ACS would “provide the most useful information with the least amount of burden.” • To support the ICSP Subcommittee, the ACS Content Review is the next logical step in reviewing content beyond what was accomplished with the OMB effort.

7. Content Review Process • Issue: Determine how to evaluate each topic on the questionnaire • Steps completed: • Action Plan: Developed an action plan and sent to ESA for review, plan is in review at Census Bureau for signatures • Stability of estimates: Investigated but determined limited potential in assessing each questionnaire topic • Developed draft Communication Plan: Includes internal and external communication activities • Criteria: Drafted decision criteria for review process

8. Content Review Process Review Methodology: • In order to evaluate ACS content to determine which variables were most valuable, we decided on a cost-benefit analysis in which: • “cost” is the difficulty in obtaining quality information • “benefit” is the usefulness and quality of the data • The Census Bureau then identified and selected the decision criteria that fit into these two categories.

9. Content Review Process Decision Criteria: • The Census Bureau has identified 18 decision criteria in all: • 12 that measure the utility and quality of a variable • 6 that measure the difficulty of obtaining information • The Census Bureau will rely on federal agencies to provide information for about one-third of the criteria, and we will provide the information for the remaining criteria.

10. Content Review Process Utility Difficulty Cognitive Burden Sensitivity Overall Difficulty Number of Complaints Seconds to Answer Median Item Response Rate at County Level • Number of Statutory, Regulatory, and Programmatic Uses • Lowest Statutory, Regulatory, and Programmatic Geographic Levels • County-Level Interquartile Range in Values • Used to administer ACS • Used to select frame for a Federal Survey • Used to produce other Census Bureau products • Other data sources exist • Median County-level Coefficients of Variation

11. Content Review Process (Notional) Utility Difficulty 11

12. Content Review Process Process Challenges Ability to implement the assessment quickly Nonresponse to the initial request to participate Ability in having the right POCs and SMEs identified Develop a scoring method to adequately evaluate each topic • Leverage OMB information collected last year • Contact the federal agencies at the top level management • Federal agencies identify POC and SMEs • POCs/SMEs conduct investigation and documentation • ACS and POCs/SMEs engage in discussion and documentation • Conduct Question Assessment • Conduct external communication with federal and non-federal stakeholders • Make determination and document findings

13. Questions for Discussion • How would you like us to engage you in this process? • Can you provide us with additional thoughts on the criteria? • How can you help us to reduce respondent burden and perceived intrusiveness of the survey?