ESSnet on SDMX phase II Laura Vignola ISTAT

1. ESSnet on SDMX phase II Laura Vignola ISTAT Rome, 3-4 December 2012

2. ESSnet Workshop SISAI Content of the presentation • Objectives of ESSnet on SDMX phase II • Workpackages in the grant • For every Workpackage • Objectives of the wokpackage • Activities to reach the objectives • Results achieved

3. ESSnet Workshop SISAI Objectives of ESSnet on SDMX phase II Implementation of SDMX at international and National level; The further development of the SDMX technical standards used for data and metadata; The further development of SDMX Content-oriented Guidelines; The evaluation adaptation and further development of SDMX IT architecture and IT tools; The increase of SDMX capacity and knowledge at all levels.

4. ESSnet Workshop Workpackages presented • WP1- Management of the action (IT) • WP2- Pc-Axis SDMX Integration phase II (SW) • WP3- Support of SDMX application for Micro Data handling (SK) • WP4 - Harmonization of existing SDMX tools (IT,BG) • WP5- Capacity building actions (IT)

5. ESSnet Workshop WP2 – PC-Axis SDMX integration phase II Objectives: Integration of the PC-Axis environment with SDMX Activities: Finalize and test the architecture developed in the previous ESSnet Provide capacity building actions internally (to other Swedish organizations) and externally to Sweden (to other NSIs) in order to show and promote the new architecture. Provide new User’s manual of the process enriched with more use cases of statistics more complex in terms of variables.

6. ESSnet Workshop WP2 – PC-Axis SDMX integration phase II Results New functionalities were introduced in the PX-Web application (.Net) and in PX-Edit application related to attribute at cell level Visualization tools (“DataExpress”, “eXplorer”) were integrated in the architecture The architecture has been tested with the Census DSDs. Capacity building action with a workshop that was held in Rome at the end of June in cooperation with Istat.

7. ESSnet Workshop WP3 – SDMX for Micro-data handling Objectives: Support the implementation of SDMX, both from a methodological and technological point of view, in the survey design and data collection phases. Activities: Enancement of the Ecollect-X application trough a automated generation of the DSD starting from the questionnaire. Comparison of SDMX and DDI on how the two standard handle microdata and propose a possible SDMX extension.

8. ESSnet Workshop WP3 – SDMX for Micro-data handling Starting point (ESSnet phase I): Respondent User Questionnaire definition User SDMX Metadata Editor E-Collect-X Questionnaire Metadatabase Questionnaire data

9. ESSnet Workshop WP3 – SDMX for Micro-data handling Questionnaire Metadatabase Questionnaire data SDMX-RI SDMX Data Generator DSD generator (automatic mapping) Data retriever Data Retriever DSD Mapping Store

10. ESSnet Workshop WP3 – SDMX for Micro-data handling • Results: • An upgraded version of E-CollectX with the following features: • The automated generation of the DSD from the questionnaire definition • An integration with SDMX-RI trough an automated creation of the mapping inside the Mapping store • New validation rules • More friendly GUI to create the questionnaire based on the WYSWYG editor • A pilot test based on the Tourism survey (BNSI) and on the first module of Labour Force survey (ISTAT) • A document of analysis on a comparison between SDMX and DDI

11. ESSnet Workshop WP4 – Harmonization of existing tools • Objectives: • Promote and support the reusing of code of SDMX-RI • Upgrade the SDMX-RI with a data loader application • Activities: • Realization of step-by-step tutorials based on two examples that reuse some modules of the SDMX-RI. • Loader development with the following requirements: • Creation of a database from scratch (Istat SDMX framework 2.0) • interaction with Mapping Store, • rewriting of the code in C#,

12. ESSnet Workshop WP4 – Step by step tutorial – first example Improvement of the Web Service of the SDMX-RI to load SDMX artefacts in Mapping Store database Submit Structure MAPPING STORE DB SDMX Structure file SDMX-RI Web Service SDMX-RI classes

13. ESSnet Workshop WP4 – Step by step tutorial – second example Improvement of an excel plug-in that will allows user to require (using a constrained search between category schema, dataflows, concepts) data by a web service exposed. GetStructure/GetData MAPPING STORE DB Structural Metadata/ SDMX data EXCEL plug-in SDMX-RI Web Service SDMX-RI classes

14. SDMX, flr, gesmesTS,csv Structure files WP4 – Loader architecture Loader Loader database Web Service Mapping assistant Mapping Store database Structural tables Data tables Mapping set tables

15. ESSnet Workshop WP4 – Harmonization of existing tools Results: Two tutorials based on two practical examples that reuse some modules of SDMX-RI. These example were realized both using .Net environment (ISTAT) and Java environment (BNSI). Analysis of the source code of modules of SDMX-RI and the related documentation, providing where they can be improved. Release of the “Loader” application and related documentation.

16. ESSnet Workshop WP5 – Capacity Building • Objectives: • Provide capacity building actions on SDMX • Activities: • Producing a Handbook on how to model data in SDMX oriented to statisticians • Workshop on SDMX based on the model of UNESWA (United Nation's Economic and Social Commission for Western Asia's) for IT staff and statisticians

17. ESSnet Workshop WP5 – Capacity Building • Results: • Handbook on how to model data in SDMX. It include: • a comparison between statistical data concepts and SDMX artefacts • the examples in the handbook are based on the Labour Force Istat survey • The modeling has be done in two different moments of the statistical business process • Two workshops were held at the beginning and at the end of June: • strong interaction between Statisticians and IT staff • interested in sharing knowledge