The system aspect of statistical quality

1. Q2014 european conference on quality in official statistics Special session: Consistency of Concepts and Applied Methods in Business Statistics The system aspect of statistical quality Norbert Rainer, Josef Richter Vienna, June 2014 1

2. Q2014 european conference on quality in official statistics Special session: Consistency of Concepts and Applied Methods in Business Statistics Introduction In order to guarantee the quality of results of European statistics according Regulation (EC) No 223/2009 Article 12 seven criteria are to be met. The first five of them refer to the quality of a statistical domain (often a single variable) seen in an isolated way. The last two criteria ‘comparability’ and ‘coherence’ address the relationship between statistical projects; they deal with system wide aspects. In many cases the assessment of quality concentrates on the first five criteria and in particular on the criterion of accuracy. Norbert Rainer, Josef Richter Vienna, June 2014 2

3. Q2014 european conference on quality in official statistics Special session: Consistency of Concepts and Applied Methods in Business Statistics Introduction Comparability and coherence and the system aspect of quality are of special relevance in business statistics. This aspect was taken up by the “ESSnet on Consistency”. Based on the “Consistency study” which identified the inconsistencies laid down in the legal framework of European business statistics, the objective was to prepare an inventory of the implementation of the concepts in the Member States and to derive appropriate proposals how inconsistencies can be reduced. Norbert Rainer, Josef Richter Vienna, June 2014 3

4. Q2014 european conference on quality in official statistics Special session: Consistency of Concepts and Applied Methods in Business Statistics Introduction Inconsistencies are caused by inconsistent concepts different ways of implementing concepts inconsistent methodological approaches used in the data generating process. The harmonisation of processes used in the reporting units and in the NSIs is also of decisive importance for consistency and thus for statistical quality in a system wide context. Norbert Rainer, Josef Richter Vienna, June 2014 4

5. Q2014 european conference on quality in official statistics Special session: Consistency of Concepts and Applied Methods in Business Statistics Special characteristics of business statistics Institutional background of the fundus of information available to respondents. High degree of division of labour in the various steps of the data generating process: Step 1 Basic observations at the unit level Step 2 Classification and aggregation of basic observations at the unit level; design of the information system Step 3 Statistical reporting to the NSI Step 4 Statistical editing in the NSI Step 5 Classification and aggregation of micro data by the NSI Norbert Rainer, Josef Richter Vienna, June 2014 5

6. Q2014 european conference on quality in official statistics Special session: Consistency of Concepts and Applied Methods in Business Statistics Important sources of inconsistencies in the data generating process Inconsistencies on the level of single respondents Concepts - Use of different definitions in different statistical domains Processes - Different codification of identical products by different people Inconsistencies on the micro level of micro data between units Concepts - Use of different statistical units in different statistical domains Processes - Different methods of decomposing’ basic information Inconsistencies on the level of Statistical Offices - edited micro data sets Processes - Use of different methods of editing and imputation across domains Processes - Lack of system wide checks for consistency Norbert Rainer, Josef Richter Vienna, June 2014 6

7. Q2014 european conference on quality in official statistics Special session: Consistency of Concepts and Applied Methods in Business Statistics Important sources of inconsistencies in the data generating process Inconsistencies on the level of Statistical Offices – meso data Concepts - Differences in the target populations across domains Concepts - Differences in the breakdown by activities across domains Concepts - Differences in the breakdown by size classes across domains Implementation of concepts - Differences between Member States Implementation of concepts - Differences between domains Norbert Rainer, Josef Richter Vienna, June 2014 7

8. Q2014 european conference on quality in official statistics Special session: Consistency of Concepts and Applied Methods in Business Statistics Conclusions In an integrated system of business statistics a clear priority has to be put on the criteria ‘relevance’, ‘consistency’ and ‘coherence’ thus stressing the system aspect of statistical quality. For most users and most problem areas business statistics are only ‘relevant’ if the results of different domains of business statistics can be used jointly. Consistency is dependent on consistent concepts, a harmonised implementation of these concepts, but at the same time on statistical methods aiming at a high degree of consistency in the data generating processes. Quality assessment has to take the system aspect into account explicitly. Norbert Rainer, Josef Richter Vienna, June 2014 8

9. Q2014 european conference on quality in official statistics Special session: Consistency of Concepts and Applied Methods in Business Statistics Thank you for your attention Norbert Rainer, Josef Richter Vienna, June 2014 9