Quality assurance activities at EUROSTAT

1. Quality assurance activities at EUROSTAT CCSA Conference Helsinki, 6-7 May 2010 Martina Hahn, Eurostat

2. Structure of the presentation Code of Practice Institutional environment Process quality Output quality ESS Quality Architecture 11/8/2014

3. European Statistics Code of Practice

4. Content of the Code of Practice 3 headings – 15 principles – 77 indicators Institutional environmentProfessional independence, Mandate for data collection, Adequacy of Resources, Quality commitment, Statistical confidentiality, Impartiality and objectivity Statistical processesSound methodology, Appropriate statistical procedures, Non-excessive burden on respondents, Cost effectiveness Statistical outputRelevance, Accuracy and reliability, Timeliness and punctuality, Coherence and comparability, Accessibility and clarity

5. Institutional environment ESS-wide peer reviews during 2006-08 to monitor the implementation of the Code of Practice to boost quality, enhance credibility to share ESS best practices and transfer knowledge Establishment of the European Statistical Governance Advisory Board (ESGAB): to provide an independent overview of the implementation of the Code of Practice by Eurostat to advise Eurostat and the ESSC on updating and communicating the Code to users and data providers Establishment of general guidelines (e.g. confidentiality manual)

7. ESS Quality Coaching Two quality managers work together to address specific improvement actions Based on experience, dialogue and information exchange Agreement on commitments from both sides Joint working – not doctor/patient relation 17 coaching pairs took part Coaching exercise completed at the end of 2009 Appropriate where there are clear standards and guidelines

8. Process and output quality: the Eurostat Quality Assurance Framework A system of coordinated methods and tools to ensure a sustainable level of quality of processes and outputs in line with the Code of Practice: product/ output quality requirements are explicitely documented processes are defined and made known to all staff the correct implementation of the processes is monitored users are informed on the quality of the products and possible deficits a procedure to guarantee that the necessary improvement measures are planned, implemented and evaluated

9. The Eurostat Quality Assurance Framework

10. The Eurostat Quality Assurance Framework 10

11. Process Quality Assessments (1) Common tool: The Process Quality Assessment Checklist (based on the International ISPAC) Method: chronologically examining the process Follow-up (10) User needs (3) Data collection (4) Validation Country level (5) International level (6) Confidentiality (7) Documentation (8) Dissemination (9) Conceptual framework (2) IT conditions (11) – Management, planning and legislation (12) – Staff, work conditions and competence (13) • Outputs: • Assessment Diagram • Summary Assessment report with improvement actions • Highlight of a good practice 11

12. Process Quality Assessments (2)Each process is matched with an assessment category Self- Assessment Prod. unit: fills in Checklist, produces outputs B1: provides support on demand • Supported • Self-Assessment • Prod. unit: in control of assessment,participates in all steps • B1: pre-fills the Checklist, produces the outputs Peer review Same asSupported Self-Assessment with one or more peers (int/ext)taking part in all steps Rolling review A comprehensive reviewbased on the checklistimplemented by ext.contractor, involving partner & user surveys and cost-benefit analysis Process 4 Process 5 Process 1 Process 2 Process 3 • Characteristics: • Periodicity • Legal Basis • Output • ESTAT intervention • availability of quality report 12

13. Process Quality Assessments Implementation Plan: Principles Minimize burden for units Test of the approach in advance Support from the Quality unit Flexibility Build on existing information Quality reports Process analysis Integrate QA with other horizontal activities

14. Process Quality Assessment Outputs (1) Summary Assessment Report

15. Process Quality Assessment Outputs (2) Accessibility and clarity Assessment Diagram Overall quality of metadata 5 Comparability across countries Relevance 4 User satisfaction 3 Comparability 2 1 0 Comparability over time Accuracy Overall accuracy Coherence Timeliness General coherence Timeliness of final publication

16. The Eurostat Quality Assurance Framework 16

17. ESS Standard & Handbook for Quality Reports (1) Provide (mainly producer oriented)recommendations / guidelines for the preparation of comprehensive quality reports for statistical processes and their outputs by distinguishing between 6 types of statistical processes: Sample survey Census Statistical process using administrative sources Statistical process involving multiple data sources Price or other economic index process Statistical compilation

18. ESS Standard & Handbook for Quality Reports (2) Promote harmonized quality reporting across statistical processes and Member States => facilitate comparisons Are applicable to National Statistical Offices and Eurostat in their roles as producers of statistics Also include 21 quality and performance indicators Handbook: is more detailed, with lots of examples and explanations, describes the quality concepts, explains how the different quality documents relate to each otherand is aligned with the 15 Principles of the Code of Practice

19. ESS Quality architecture ESS level national / Eurostat level ESS Statistical Law national Statistical Law ESS Quality Declaration (national TQM framework) National QAF: Description of practices and procedures for implementing the CoP QAF: ESS standards for implementing the CoP