P rinciples and Practices for a Federal S tatistical Agency — what and why —

1. Principles and Practices for a Federal Statistical Agency— what and why — Constance Citro, Director, CNSTAT COPAFS • June 7, 2013 Washington, DC

2. “P&P”—A Product of CNSTAT at the National Academies 1992 • 2001 • 2013 • 2005 • 2009

3. What Are the National Academies? National Academy of Sciences—independent, nonprofit honorific society; 1863 congressional charter to advise government on science and “art” National Academy of Engineering—est. 1964 Institute of Medicine – est. 1970 National Research Council – est. 1916 as operating arm of NAS About 60 standing units like CNSTAT

4. What Is CNSTAT? Established in 1972 as a standing unit of the NAS, recommended by the President’s Commission on Federal Statistics to provide an independent, objective resource for evaluation and improvement of federal statistical methods and operations. CNSTAT’s mission is to improve the statistical methods and information on which public policy decisions are based; it also serves as a coordinating force in the decentralized U.S. statistical system. Over its 40-year history, CNSTAT has produced over 240 consensus, interim, and workshop reports.

5. Who Serves on CNSTAT? Lawrence Brown (chair), statistics John Abowd, economics Lisa Lynch, economics David Card, economics Sally Morton, biostatistics Alicia Carriquiry, statistics Ruth Peterson, criminal justice Constantine Gatsonis, Ted Shortliffe, bioinformatics biostatistics Hal Stern, statistics James House, survey research John Thompson, survey mgt. Michael Hout, sociology (public/private sectors)* Sallie Keller, statistics Roger Tourangeau, survey *on leave of absence research Serve pro bono to oversee CNSTAT’s project portfolio Supported by 14 FTE staff from variety of backgrounds

6. What Does CNSTAT Do and Who Pays? Types of work (members of panels et al., are volunteers, reimbursed for travel) Consensus panels (10–14 members)—review data collection programs, entire agencies (BJS, BTS, NCES, NCSES, OIS), cross-cutting issues, key measures; issue reports with findings and recommendations; members are appointed by NAS president and supported by staff Stand-alone workshops —convening function, summary report Other convening activities —planning meetings, seminars, etc. Principles and Practices —a report authored by CNSTAT as a committee of the whole Agency contracts/grants fund individual projects Over 20 agencies contribute small amounts of core support to let CNSTAT maintain staff and undertake cross-cutting, “good citizen” activities, such as P&P: NSF/MMS manages core grant; AHRQ, ASPE, BEA, BJS, BLS, BTS, Census, CIS/DHS, Defense, EIA, ERS, HUD, NCES, NCHS, NCSES, NEA, NIA, OIS/DHS, SOI/IRS, SSA

7. Sampling of CNSTAT Studies

8. “P&P”—Five Editions to Date • 2001– • 44 pp. 1992– 27 pp. • 2009— • 123 pp. • 2005— • 66 pp. • 2013— • 166 pp.

9. What P&P Is and Is Not It IS a statement of principles that are fundamental and intrinsic to the concept of a federal statistical agency; AND a statement of practices that are ways and means of making the basic principles operational and facilitating an agency’s adherence to them It IS NOT a guide to best practices for data collection, processing, estimation, or dissemination; NOR does it provide a set of data quality standards It IS a set of guidelines and NOT a set of prescriptions; it has endeavored to keep up with new challenges and opportunities It not only applies to federal statistical agencies, but may also be useful for statistical activities elsewhere, as in other federal agencies, state/local governments, and other countries

10. Origins – First Edition, 1992 Congressional staff and cabinet departments inquired about what constitutes an effective statistical agency (per establishing BTS and contemplating a statistical agency in EPA) CNSTAT recognized that statistical agencies are sometimes confronted with inappropriate requests for confidential information or to put a policy spin on press releases of data Statistical agencies in our decentralized system must operate under policies and guidance of many departments of government, and not all officials are knowledgeable about proper practices for federal statistics So, CNSTAT undertook to produce a “white paper,” now known as “the purple book”

11. Origins, cont’d– 1st Edition, 1992 Some controversy at the time about whether this might better be a product of OMB CNSTAT went ahead, involving OMB and statistical agencies in commenting on a draft Issued in 1992 on 20th anniversary of CNSTAT’s founding; edited by Margaret Martin and Miron Straf Over time, the importance of CNSTAT/NRC/NAS as the authoring body became clear, given the NAS reputation for independence and objectivity and that OMB inherently is viewed as part of an administration

12. Structure – First Edition, 1992 Preface Acknowledgments Principles and Practices for a Federal Statistical Agency [became Part I in subsequent editions; included a definition of a statistical agency and reasons for establishing same—8 pp. in 1992; 16 pp. in 2013] Commentary [became Part II in subsequent editions—16 pp. in 1992; 54 pp. in 2013] References [featured CNSTAT reports, 3 pp. in 1992; 13 pp. in 2013, including FCSM working papers, OMB documents, etc.] Appendixes [none in 1992; 5 appendixes in 2013, 72 pp.]

13. Definition/Rationale – First Edition, 1992 Definition of a federal statistical agency: A federal statistical agency is a unit of the federal government whose principal function is the compilation and analysis of data and the dissemination of information for statistical purposes. Reasons to establish a statistical agency: Need for information extending beyond narrow scope of individual operating units Need to establish independence of major series from policy or operating control Need to establish confidentiality protection Need to consolidate collection, compilation, analysis and dissemination of statistics in one unit to encourage high-quality performance, eliminate duplication, and streamline operations

14. Content – First Edition, 1992 Three principles— Relevance to policy issues Credibility among data users Trust among data providers and data subjects Eleven practices— A clearly defined and well-accepted mission A strong measure of independence Fair treatment of data providers Cooperation with data users Openness about the data provided Commitment to quality and professional standards Wide dissemination of data An active research program Professional advancement of staff Caution in conducting nonstatistical activities Coordination with other statistical agencies

15. “Strong Measure of Independence” Arguably, most important but also trickiest concept in P&P Tension: Agencies must operate within framework of congressional, OMB, and departmental oversight, but also maintain credibility as impartial purveyors of information Aspects of independence cited in 1st edition: • Organizational separation from department program activities • Professionally qualified head; appointment for a fixed term; direct access to the secretary • Broad authority over scope, content, and frequency of data • Primary authority for selection and promotion of prof. staff • Authority to release statistical info without prior clearance • Authority to speak on agency’s program before Congress, with congressional staff and before public bodies • Adherence to predetermined schedules for release of key stats • Maintenance of clear distinction between statistical releases and policy interpretations put out by others

16. Origins – Second Edition, 2001 First edition used by federal statistical agencies to inform department officials, advisory committees, and others Used in a U.S. GAO (1995) study of the performance of the major statistical agencies Used by Janet Norwood in her 1995 book, Organizing to Count Informed establishment and later assessment of BTS (NRC, 1997) Cited in confirmation hearings (e.g., NCES commissioner, 1999) CNSTAT decided that it would be useful to release a revised and updated version and, moreover, to do so every 4 years at the beginning of a new administration or second term Second edition editors were Margaret Martin, Miron Straf, and Connie Citro

17. Changes – Second Edition, 2001 Appendixes reproduced Fundamental Principles of Official Statistics of the Statistical Commission of the UN (adopted 1994) and provided list of selected federal statistical web sites Added explicit discussion of the need for federal statistics: Statistics that are publicly available from government agencies are essential for a nation to advance the economic well-being and quality of life of its people . . . . Federal statistical agencies are established to be a credible source of useful, accurate statistics in one or more subject areas that are available to the public and policy makers on a timely basis . . . . Private-sector organizations also provide useful statistical information, . . . but private markets are not likely to provide all of the data that are needed . . . or to make data as widely available as needed . . . . Same 3 principles; practices rearranged somewhat

18. Content – 2nd Edition, 2001 Three principles— Relevance to policy issues Credibility among data users Trust among data providers and data subjects Eleven practices— A clearly defined and well-accepted mission A strong positionmeasure of independence Continual development of more useful data [recognizing Internet era] Openness about the data provided Wide dissemination of data Cooperation with data users Fair treatment of data providers Commitment to quality and professional standards of practice An active research program Professional advancement of staff Caution in conducting nonstatistical activities Coordination and cooperation with other statistical agencies

19. Changes – Third Edition, 2005 Added appendix on Relevant Legislation and Guidelines [for federal statistical agencies] Adopted Since 2001: • CIPSEA (Confidential Information Protection and Statistical Efficiency Act, aka Title V of E-Government Act of 2002) • Privacy impact assessments (PIAs) (Section 208 of E-Govt. Act) • Information Quality Act of 2000 • OMB guidance on peer review • Performance assessment rating tool (PART) Added as a characteristic of statistical agencies that strengthens independence: authority [over] . . . information technology systems [to] maintain the integrity and confidentiality of data and reliably support timely and accurate production of key statistics [sent letter to OMB on this topic in 2003]

20. Content – Third Edition, 2005 Three principles— Relevance to policy issues Credibility among data users Trust among data providers Eleven practices— A clearly defined and well-accepted mission A strong position of independence Continual development of more useful data Openness about sources and limitations of the data provided Wide dissemination of data Cooperation with data users Fair treatment of data providers Commitment to quality and professional standards of practice An active research program Professional advancement of staff Coordination and cooperation with other statistical agencies

21. Changes – Fourth Edition, 2009 Appendixes added on Organization of the Federal Statistical System [provides brief histories of major statistical agencies plus examples of other statistical programs] and Legislation and Regulations That Govern Federal Statistics: • Paperwork Reduction Act (PRA) • 2000 Information Quality Act and guidelines • 1997 OMB order providing for confidentiality of statistical info • 2002 CIPSEA and 2007 guidance • 2002 E-Government Act, Section 208, PIAs • 2004 OMB peer review guidance • 2002–2008 OMB PART • OMB statistical policy directives (metro areas, race/ethnicity, industry/occupation codes, survey standards, etc.)

22. Cabinet chart

23. Layering chart

24. Content – Fourth Edition, 2009 Four principles— Relevance to policy issues Credibility among data users Trust among data providers A strong position of independence [moved from practice per agencies] Eleven practices— A clearly defined and well-accepted mission Continual development of more useful data Openness about sources and limitations of the data provided Wide dissemination of data Cooperation with data users Fair treatment of data providers Commitment to quality and professional standards of practice An active research program Professional advancement of staff A strong internal and external evaluation program Coordination and cooperation with other statistical agencies

25. Uses of Second–Fourth Editions Third Edition cited by GAO in Bureau of Justice Statistics: Quality Guidelines Generally Followed for Police-Public Contact Surveys, but Opportunities Exist to Help Assure Agency Independence, March 30, 2007 [question concerned press releases that accompany data releases] Third Edition cited in OMB Statistical Policy Directive No. 4—Release and Dissemination of Statistical Products Produced by Federal Statistical Agencies, March 7, 2008 Fourth Edition endorsed by ASA Board, April 30, 2009 Fourth Edition cited as key document in Statement of Commitment to Scientific Integrity by Principal Statistical Agencies Series cited by GAO in Agencies Can Make Greater Use of Existing Data, but Continued Progress Is Needed on Access and Quality Issues, February 2012

26. Changes – Fifth Edition, 2013 Included Dedication to Margaret Martin, life-time COPAFS board member, who died at age 100 in May 2013 Added Executive Summary (available separately as Highlights) Added European Statistics Code of Practice for the National and Community Statistical Authorities (adopted 2011) To appendix on legislation/regulations, added: • 2010 Government Performance and Results Modernization Act • 2009–2010 Guidance on Scientific Integrity • 2002 Federal Information Security Management Act (FISMA) Beefed up Part II Commentary on Continual Development of More Useful Data to discuss surveys, admin records, and other sources Beefed up discussion of Independence and nonstatistical activities Separated Privacy from Confidentiality Protection Stressed need for active Collaboration among agencies

27. Content – Fifth Edition, 2013 Fourprinciples— Relevance to policy issues Credibility among data users Trust among data providers Independence from political and other undue external influence* A strong position of independence *Footnote (p. 14) A statistical agency actively works to obtain a broad range of external input to develop its programs: “undue external influences” are those that seek to undermine an agency’s impartiality and professional judgment.

28. Content – Fifth Edition, 2013 Thirteen practices— A clearly defined and well-accepted mission Necessary authority to protect independence Continual development of more useful data Openness about sources and limitations of the data provided Wide dissemination of data Cooperation with data users Respect for the privacy and autonomy of data providers Protection of the confidentiality of data providers’ information Fair treatment of data providers Commitment to quality and professional standards of practice An active research program Professional advancement of staff A strong internal and external evaluation program Coordination and collaboration cooperation with other statistical agencies

29. Questions? Comments? (I am readying a File for the 6th Edition of P&P) Constance Citro Director, Committee on National Statistics ccitro@nas.edu • (202) 334-3009