1 / 23

Standards and processes for integrating metadata in the European Statistical System Session 5

Workshop on Statistical Metadata (METIS) (Geneva, 5-7 October 2011). Standards and processes for integrating metadata in the European Statistical System Session 5 H. Linden, Unit B6 Eurostat. General situation.

kelda
Download Presentation

Standards and processes for integrating metadata in the European Statistical System Session 5

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Workshop on Statistical Metadata (METIS)(Geneva, 5-7 October 2011) Standards and processes for integrating metadata in the European Statistical System Session 5 H. Linden, Unit B6 Eurostat

  2. General situation Commission Communication (August 2009): Vision for improving the production methods of European statistics and their efficiency within the ESS. ESS Committee (May 2010): joint ESS strategy for implementing the Commission Communication. Need of vertical (between Eurostat and NSIs) and horizontal (between statistical domains) integration of the business processes. GSBPM: «metadata are used as an over-arching bridge all across the statistical processes».

  3. General situation • => The use of harmonised reference metadata within the ESS is essential: • * Use at European level (already in place in Eurostat) • * Use at European level (being implemented) • Compliance with international statistical and technical standards => SDMX compliance • Need of a solid legislative basis for suporting and stimulating the integration => re-thinking of the legislative strategy/policy

  4. 1. The standardisation of reference metadata at European level 1a. The standard structure(s) in place: • Until 2008: - Use of the Special Data Dissemination Standard (SDDS) - About 700 Metadata files disseminated on Eurostat website • Since 2008: - Introduction of the Euro-SDMX Metadata Structure (ESMS) - Conversion of former SDDS files to ESMS format - Reduction to ~300 files disseminated (June 2010) - Development of the ESS Standard for Quality Reports Structure (ESQRS) and its progressive implementation

  5. 1. The standardisation of reference metadata at European level 1a. The standard structure(s) in place: • The ESMS: • is made of 21 main concepts, further detailed in sub-concepts • compliance with SDMX • is part of the Commission recommendation 2009/498/EC • is oriented to USERS of statistics • The ESQRS: • is made of 13 main concepts, further detailed in sub-concepts • focuses on quality-related concepts • some concepts in common with ESMS • is oriented to PRODUCERS of statistics

  6. 1. The standardisation of reference metadata at European level: ESMS

  7. 1. The standardisation of reference metadata at European level: ESQRS

  8. 1. The standardisation of reference metadata at European level ESQRS Accuracy Non-response error Non - sampling Unit non-response rate Formulae unit resp. rate Formulae unit resp. rate error Description: Description: Description: Description: Description: The ratio of the number of units with no information or not usable information to the total number of in-scope (eligible) units. The difference Accuracy: Error in survey Ex. calculation between the closeness of estimates which formluae for un - statistics computed computations or cannot be weighted unit from the collected estimates to the exact or attributed to response rate. data and those that true values that the sampling would be computed statistics were intended fluctuations. if there were no to measure. missing values.

  9. 1. The standardisation of reference metadata at European level 1b. The related IT application: • EMIS: - Is part of the ESS - Metadata Handler • Internal IT application used only by Eurostat staff • Dedicated to the production and dissemination of ESMS files: • Production & Management (content): statistical production Units • Validation & Dissemination (structure): horizontal Unit (B6) • Output: ESMS files disseminated on Eurostat website • Interaction with other IT applications (RAMON/ CODED, Eurostat website, etc.) • Permanent enhancement of the application (additional functionalities, accomodation of the ESQRS, etc…)

  10. 2. The standardisation of reference metadata at national level The context: • Several types of files received in the statistical domains (different formats, different structures, etc.) • Information often not comparable or re-usable The objectives: • Horizontal and vertical integration: • Harmonisation between statistical domains • Harmonisation across NSIs • End-to-end process: from collection to dissemination • Reduce costs and burden at both European and National levels The approach: - Profit from the successful implementation at European level

  11. 2. The standardisation of reference metadata at National level 2a. The standard structure(s) to be used: • Same structures as for European metadata: - ESMS structure for national reference metadata files * Commission Recommendation 2009/498/EC - ESQRS structure for quality reports  Consistency between the creation of files at European and National levels  Supports the further horizontal integration of business processes

  12. 2. The standardisation of reference metadata at National level 2b. The related IT application : • The National Reference Metadata Editor (NRME): - Is also part of the ESS - Metadata Handler • IT application dedicated to users in NSIs • Dedicated to the production, exchange and dissemination of national metadata files • Production & Management: NSIs + Eurostat statistical production Units • Validation & Dissemination: horizontal Unit (B6) • Output: national ESMS and ESQRS files (with possible dissemination on Eurostat website) • Interaction with other IT applications (Euro-SDMX Registry, eDamis, Eurostat website, etc.)

  13. 2. The standardisation of reference metadata at National level

  14. 2. The standardisation of reference metadata at National level

  15. 2. The standardisation of reference metadata at National level 2c. Implementation in the ESS : • The first concrete implementation: - Harmonised Indices of Consumer Prices (HICP) • 32 National ESMS files created by NSIs • Validation of the files by the Eurostat domain managers • Dissemination of the files on the Eurostat website • The ongoing/ upcoming implementations : - Several metadata flows opened in various statistical domains • Covering both ESMS and ESQRS • Use of the NRME for the Census Hub project

  16. 2. The standardisation of reference metadata at National level

  17. 2. The standardisation of reference metadata at National level

  18. 3. Compliance with International Standards Compliance with SDMX: • SDMX as the standard for data/metadata exchange in ESS: - Adopted by the ESS Committee in February 2007 - Supported by Eurostat top management (DM Decision 2009) - A main enabler of the ESS ‘Vision’ • Compliance of the structure used: - The 21 ESMS concepts are part of the SDMX statistical standards * SDMX COG Annex I : Cross-domain concepts - ESQRS also based on SDMX cross-domain concepts • Compliance of the IT applications used: - NRME built according to SDMX Technical Standards 2.0 - Euro-SDMX Registry - Use of the SDMX Vocabulary and Terminology

  19. 4. Towards a new legislative policy • The current situation: - Most of the legal acts are specific to statistical domains « Stove-pipe legislation » => no harmonisation, no consistency - Commission Recommendation 2009/498 is a good step forward but not sufficient • The need : - Cross-cutting legislation • Will stimulate the further harmonisation and integration of statistical business processes • Is supported by ESS Committee in 2011: « a comprehensive legislation policy for the ESS »

  20. 4. Towards a new legislative policy • The proposal: two draft legal acts • Framework EP/Council Regulation : Use of harmonised metadata within the ESS * Structural & Reference Metadata * Accompanied with series of implementing measures (e.g. Recommendations: defining the content) 2. EP/Council Regulation : Harmonisation of processes for data and metadata exchange * Use of the Single Entry Point (eDamis) * Use of SDMX standards => These two cross-cutting legal acts to be complemented with framework legislations on broader statistical domains

  21. Conclusions • The use of harmonised metadata is an essential pre-condition for the further integration of statistical business processes (ESS ‘Vision’). • Standard structures and IT applications are well defined and implemented at European level. • The implementation at National level has started and is promising with the help of a dedicated IT- tool (NRME) for NSIs. • Compliance with the SDMX technical and statistical standards a precondition for metadata exchange in the ESS. • Supported by new legislative policy work: strategy towards horizontal cross-cutting regulations in the ESS.

More Related