Managing the quality brand of statistics: How to deal with revisions and publication errors?

1. Managing the quality brand of statistics: How to deal with revisions and publication errors? The European Conference on Quality in Official Statistics Rome, 9 July 2008 Heli Mikkelä

2. The Quality Brand of Official Statistics in Finland - examples Information Services

3. Reliability of information produced by Statistics Finland Source: Public image survey of Statistics Finland 2007, Taloustutkimus Oy Information Services

4. % of respondents Objectivity of information produced by Statistics Finland Source: Public image survey of Statistics Finland 2007, Taloustutkimus Oy Information Services

5. How to maintain and support the ”quality label” and trust?An example: situations when once published data change Information Services

6. Eventual situations of change • 1. Revisions due to a natural cause in the life cycle of a set of statistics, e.g., • Accrual of incoming data (preliminary results/final results) • Methodological or system change • Change in data collection method (e.g., shifting to register-based production) • 2. Error situations, and other exceptional situations, e.g. • Error is discovered in published data • Error in timing (delay, etc.) Information Services

7. Management of revisions to and publication errors in statistical data at Statistics Finland • Guidelines on communication practices confirmed in 2007 • Include principles, outlines and practical guidelines • Preparation of the guidelines • A working group comprised of experts from the fields of statistics production, dissemination, quality management and communications • International and national recommendations, guidelines and best practices widely reviewed during the preparation • Communicative process within the organisation Information Services

8. Revisions to and publication errors in statistical data, cont. • Cover • Revisions (normal life-cycle of statistics) • Errors (exceptional situations) • Emphasis on • Advance preparation • Immediate reaction • Documentation • Informing and communication, openness • Available on the intranet, updated and revised as necessary Information Services

9. Main principles for all situations • Advance preparation • For forthcoming - planned and expected - revisions • For unexpected and unplanned error situations • Identification of stakeholders of each set of statistics • Familiarisation with guidelines and principles • Good process and quality management emphasised in order to avoid errors and other undesired situations • Acknowledgement of the recommendations and principles for official statistics …cont. Information Services

10. …. main principles cont. • Internal co-operation and co-ordination essential (statisticians, communication professionals, etc.) • Immediate – but nonetheless well managed - reaction crucial • Thorough documentation forms the basis for process management and further development of practices • Communication with data users and other stakeholders in all stages of the process is of utmost importance Information Services

11. 1. Revisions to published data • Continuous revising of data • Aim: to increase general understanding of the evolving nature of statistical information • All releases must contain notification and explanation of • Preliminary nature of figures • Time and reason for updating • Reason for changes from previous release • Quality descriptions and methodological descriptions must be informative, up-to-date, usable and easily available …cont. Information Services

12. …..1. Revisions to published data cont. Normaalit… 2/2 • Major revisions to statistical systems, e.g., system revisions and methodological changes • Aims: • To increase understanding about the causes and effects of the revision • Enable stakeholders to prepare for the changes • Role of co-operation and communication with stakeholders is crucial • Systematic pre- and post communication Information Services

13. 2. Errors and other exceptional situations • Guidelines cover • Errors in contents, timing or interpretation • the Internet, printed publications, databases, etc. • Diverse error and exceptional situations can be identified, e.g. • Content error: published data are not correct • Timing error • ”Technical error”, flaws (no actual error in data) • Other exceptional situations: data updated in wrong place, other parties release information produced by Statistics Finland incorrectly, etc. • Detailed instructions provided for specific cases Information Services

14. An example: content errors • Main principle: erroneous data must be removed immediately • If they cannot be replaced with correct data, a new release date must be given • Correction must be clearly marked on the document; recommendations about actual notification and marking are given separately to various channels and media • Public must be informed about the changes as they were informed about the original data Information Services

15. Continuous monitoring • The irregularities in timing as well as corrections made are monitored regularly and analysed for process management purposes • E.g., errors in contents • 2007 total of 37 corrections made, of these • 32 actual changes in contents • 12 made on the day of publishing • 5 minor technical changes Information Services

16. Benefits and gains • Process, quality and risk management • Practices and enhancement of communication about statistics, metadata and quality of statistics • Promotion of the quality brand of official statistics by • Increasing the openness and transparency of statistical processes • Strengthening of trust in official statistics • General understanding of the nature of statistical production and information • Support for statistical literacy • Strengthening of relationships with stakeholders Information Services

17. Further activities and challenges • Continuous improvement of recommendations and guidelines • Promotion of knowledge about recommendations internally and externally • Utilisation of gained experiences in process and quality development • Improvement and development of metadata components of statistical information and communication Information Services

18. % of respondents Do ordinary people receive enough information about the meaning of statistical information? Information Services

19. Thank you!Heli MikkeläDirector, Information ServicesStatistics Finlandemail: heli.mikkela@stat.fitel. +358 9 1734 3200, gsm +358 50 568 7444www.stat.fi Information Services