1 / 18

Aurora De Santis, Riccardo Carbini Istat, Italy

Quality of quality: an analysis of quality indicators and their quality. Aurora De Santis, Riccardo Carbini Istat, Italy European Conference on Quality in Official Statistics – Q2010. Helsinki, 06.05.2010. European Conference on Quality in Official Statistics – Q2010. Summary.

sook
Download Presentation

Aurora De Santis, Riccardo Carbini Istat, Italy

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Quality of quality: an analysis of quality indicators and their quality Aurora De Santis, Riccardo Carbini Istat, Italy European Conference on Quality in Official Statistics – Q2010 Helsinki, 06.05.2010

  2. European Conference on Quality in Official Statistics – Q2010 Summary • The evolution of the SIDI system • An example of quality analysis • Quality of quality indicators • Final remarks Helsinki, 06.05.2010

  3. European Conference on Quality in Official Statistics – Q2010 Summary • The evolution of the SIDI system • An example of quality analysis • Quality of quality indicators • Final remarks Helsinki, 06.05.2010

  4. European Conference on Quality in Official Statistics – Q2010 The evolution of the SIDI system (1) Main characteristics of SIDI, the Istat Documentation System on Surveys - Process metadata and quality indicators collection for all Istat survey and each survey occasion - Integrated management of process metadata and quality indicators - Oriented to internal and external users Description of the production process and its characteristics (also in English) http://siqual.istat.it • - Documentation of production processes • Standard and ad hoc quality analyses Helsinki, 06.05.2010

  5. European Conference on Quality in Official Statistics – Q2010 The evolution of the SIDI system (2) Helsinki, 06.05.2010

  6. European Conference on Quality in Official Statistics – Q2010 The evolution of the SIDI system (3) During the last 5 years efforts have been addressed to increase the series of quality indicators collected The systematic collection of quality indicators started in 2005 with the collection of the timeliness and punctuality group for all Istat surveys From 2006 a new group for each survey was collected and the previously collected series of quality indicators were updated Helsinki, 06.05.2010

  7. Titolo del convegno anche su più righe anche su più righe European Conference on Quality in Official Statistics – Q2010 The evolution of the SIDI system (4) Number of occasions of quality indicators available for each group at 2005 and at 2010 Helsinki, 06.05.2010

  8. European Conference on Quality in Official Statistics – Q2010 The evolution of the SIDI system (5) At present, a great number of quality indicators has been collected and calculated More requirements of quality of the quality indicators collected More opportunities to exploit the information collected Helsinki, 06.05.2010

  9. European Conference on Quality in Official Statistics – Q2010 Summary • The evolution of the SIDI system • An example of quality analysis • Quality of quality indicators • Final remarks Helsinki, 06.05.2010

  10. European Conference on Quality in Official Statistics – Q2010 An example of quality analysis (1) • The expanded set of data now available can be used to perform more complete analyses than in the past. • Some examples of analyses on quality are reported in the paper. • Mean of timeliness by year and survey periodicity • Mean of frame error rate by year and data collection mode • Mean of response rate by data collection mode Helsinki, 06.05.2010

  11. European Conference on Quality in Official Statistics – Q2010 An example of quality analysis (2) Timeliness=(actual date of first release of final data) - (date of reference for the data) Mean of timeliness by year and survey periodicity (days) Monthly Quarterly Yearly 2004 254 (24) 367 (14) 595 (71) 2005 218 (26) 317 (13) 536 (66) 2006 216 (28) 315 (13) 485 (57) 2007254 (29) 274 (13) 419 (45) 2008 242 (25)215 (13) 294 (26) 2009 120 (16) 155 (10) 300 (4) Overall mean 224 (33) 280 (16) 494 (80) (number of different surveys the mean is based on) Helsinki, 06.05.2010

  12. European Conference on Quality in Official Statistics – Q2010 Summary • The evolution of the SIDI system • An example of quality analysis • Quality of quality indicators • Final remarks Helsinki, 06.05.2010

  13. European Conference on Quality in Official Statistics – Q2010 Quality of quality indicators (1) • Quality assurance of quality indicators • Training of the quality pilots in charge of collecting and calculating quality indicators • Automatic check (domain and coherence rules) implemented in SIDI • Support of the central quality unit to the quality pilots Nevertheless some errorsmayoccur, and they can havean impact Helsinki, 06.05.2010

  14. European Conference on Quality in Official Statistics – Q2010 Quality of quality indicators (2) At present, a large set of quality indicators is calculated every year. The opportunity is the availability of series of quality indicators Control charts can be an useful method to analyse the series of quality indicators. Apreliminaryanalysisofseriesoftimelinessindicatorvaluesfor a set ofsurveysusingShewhartindividualcontrolcharts Helsinki, 06.05.2010

  15. Titolo del convegno anche su più righe anche su più righe European Conference on Quality in Official Statistics – Q2010 Quality of quality indicators (3) Helsinki, 06.05.2010

  16. Titolo del convegno anche su più righe anche su più righe European Conference on Quality in Official Statistics – Q2010 Quality of quality indicators (4) Helsinki, 06.05.2010

  17. Final remarks • The increasing use of the quality indicators collected in Istat for quality evaluation requires trustable and accurate measures. • The great amount of indicators that are currently calculated every year requires further developments of our strategy. • The examples reported in this paper suggest that the control chart methodology could fit this purpose. • In the next years Istat could invest on further analyses and on integrating such methods in the SIDI management subsystem, in order to provide graphical tools for quality pilots as alarm bells on the quality indicator accuracy, when loading the indicators or revising them. Helsinki, 06.05.2010

  18. Titolo del convegno anche su più righe anche su più righe European Conference on Quality in Official Statistics – Q2010 Quality of quality indicators Helsinki, 06.05.2010

More Related