Assessing the Quality of Administrative Data

1. Assessing the Quality of Administrative Data 2012 FCSM Statistical Policy Seminar Session 7: New Perspectives on the Quality of Administrative Data December 5, 2012 Bill Iwig, USDA/NASS (on detail to OMB/SSP)

2. Development of the Data Quality Assessment Tool • Data Quality ~ Fitness for Use • Intended to facilitate decisions by a potential data user about the suitability of the data for identified purpose. • Intended to prompt additional questions. • Based on the information gathered, the data user may decide to: • Use the data as planned • Alter plans • Accommodate some quality weakness in the data • Not use the data

3. Development of the Data Quality Assessment Tool • Multi-Dimensional • Relevance • Accessibility • Interpretability • Coherence • Accuracy • Institutional Environment • Questions within each dimension are designed to address quality characteristics of micro-level administrative data. • Example answers provided based on characteristics of a Supplemental Nutrition Assistance Program (SNAP) data set.

4. Development of the Data Quality Assessment Tool

5. Putting the Phases and Dimensions Together • Discovery Phase: 12 questions covering Relevance, Accessibility, and Interpretability, including a request for a Data Dictionary • Initial Acquisition Phase: 29 questions covering Accessibility, Interpretability, Coherence, Accuracy, and Institutional Environment • Repeated Acquisition: 11 questions covering Interpretability (updated Data Dictionary), Coherence, Accuracy, and Institutional Environment

6. What it is and What it isn’t • Evaluates quality characteristics of an administrative data file. • Doesn’t address quality characteristics of linked data. • Doesn’t provide guidance for making fitness for use decisions. • Doesn’t provide quality improvement recommendations. • Not specifically designed to address quality of commercial data.

7. Status of Pilot Testing • Obtained feedback from three federal/state reviewers prior to pilot testing. • Pilot tested on: • Child Care data files from the Office of Child Care Information Systems at DHHS – Initial Acquisition Phase • Wisconsin Child Care Payment System files – All Phases • Expecting a response for two additional files. • Planning to conduct follow-up interview after completion of assessment tool.

8. General Comments • Completing the Data Quality Assessment Tool is perceived to be burdensome to the administrative agency. • Some agencies may be concerned about data quality implications. • Many administrative data files are compiled from various data sources. • Right scope and content to be incorporated into agency administrative record procedures?

9. Next Steps • Update based on feedback received. • Make available to the public. • Pursue opportunities to promote use.

10. Thank You Bill Iwig Detailee to the Statistical and Science Policy Office, OMB William_C_Iwig@omb.eop.gov 202-395-2293