Alessio Cardacino

1. The Single Exit Point system Alessio Cardacino DCIQ - Directorate for the development of integrated information systems and products, the management of information assets and the quality evaluation

2. Summary • The Single Exit Point system (SEP) • The Eurostat’s SDMX Reference Infrastructure (SDMX-RI) • The Single Exit Point project in ISTAT • SDMX-RI add-ons developed by ISTAT • The current dissemination metadata flow in ISTAT and the hypothesis of an SDMX Driven dissemination metadata flow

3. The Single Exit Point system (SEP) • Generalized system for the dissemination of I.stat data in SDMX format via web-service (machine-to-machine logic) using a single end point. • Is based on the SDMX Reference Infrastructure (SDMX-RI) produced by Eurostat • The main objective is the machine to machine dissemination of statistical data to enterprises, institutions and other organizations

4. The Eurostat’s SDMX Reference Infrastructure • Software infrastructure developed by Eurostat in order to allow the NSIs to disseminate statistical data, coming from their dissemination databases, in SDMX format via web service • Is based on the generalized mapping of local datasets on the SDMX constructs. • The following software modules are part of the infrastructure : • Mapping Assistant • NSI Web Service provider • Test Client • NSI Web Client • A set of APIs

5. The Eurostat’s SDMX Reference Infrastructure • The SDMX queries are translated in SQL queries for the local databases • This translation is performed through an interpreter using specific parameters to map the SDMX data structure definition on the metadata describing the database’s table or view containing the data which are to be queried • The result of the SQL query is converted into the selected SDMX data format (generic, compact or cross sectional) before to be returned by the web service • Through the infrastructure is also possible to make queries on SDMX metadata (codelists, concept schemes, key-families, ecc..)

6. The Eurostat’s SDMX Reference Infrastructure • The infrastructure is based on a registry database (usually called Mapping Store) containing: • SDMX metadata related to the available datasets • Parameters for the connection to the local databases and also for the mapping of local dataset • The Mapping Store’s populating is performed by the Mapping Assistant tool, through which the operators can upload the SDMX metadata and making mappings of the local datasets on the SDMX artefacts and constructs

7. Advantages of the SDMX-RI • Data and metadata in the «local» databases are neither modified nor duplicated but only mapped on the SDMX DSDs • Data of different statistical domains and coming from different databases can be disseminated through an only one end point in a transparent way for the end users • The infrastructure allows a metadata harmonization, as well as an harmonization of the workflow (some functions are performed by statisticians, others by IT staff)

8. The Single Exit Point project in ISTAT • The project began in February 2011 • By June 30,2011 have been implemented the test and production system and application environments • Between September and December 2011 has began a phase of controlled dissemination of data via the SDMX web service, in which are participating: • National Bank of Italy, JP Morgan • OECD for STS data collection • Some Regional bodies (initial contacts for the dissemination of Census data) • The data currently available through the SDMX web service are related to the following themes: Industry, Constructions,Services,Prices,Employment

9. The Single Exit Point «planet» Software development for final users (Windows client, Excel plug-in,…) User Group SDMX-RI (Eurostat) Development of new features for the Registry (Registry web GUI,Submit Structure method,..) OECD SIS-CC Single Exit Point Hardware and software infrastructure Census Hub SDMX Registry STS - OECD Structural metadata of the aggregated data (Registry) Mapping Store MtoM dissemination of I.Stat data MtoM dissemination of Census data

10. Hardware and software architecture SDMX Data Collectors Load-Balancing Server SDMX Reference Infrastructure (modules developed by ESTAT) SDMX Web service 3 SDMX Web service 2 SDMX Web service 1 Mapping Assistant ISTAT Mapping Store DB Dissemination RDBMS

11. SEP and the ISTAT data dissemination Institutional website. Classifications Server Classif.serv. GUI I.Stat data warehouse I.Stat web GUI Registry Registry web GUI category scheme mapping data WebService dataflows, constrains.. metadata concepts classif./ codelist DSD SDMX Data and Metadata SEP

12. SDMX-RI add-ons developed by ISTAT • GetSDMXDataWinApp: Windows application to browse the structural metadata and extract data from the SDMX web service (SEP) • GetSDMXDataExcelApp: Microsoft Excel plugin to browse the structural metadata and extract data from the SDMX web service (SEP) • SubmitQuery method for the SDMXweb service: a new web service method to insert structural metadata in the SDMX registry (mapping store) according to a machine to machine logic • Registry Web GUI: web application to browse the structured metadata stored into an SDMX Registry (mapping store)

13. SDMX ISTAT Strategy Data providing to national and/or international organizations Machine-to-machine dissemination Support to end users Harmonization of contents

14. Future directions: from MS to SDMX registry • Improve the authentication/authorization mechanism for accessing to both methods of extraction of data and metadata on the web service provider • Implement an authentication/authorization mechanism for the "SubmitStructure" interface to the Registry (developed by ISTAT) • Improve the web registry GUI (developed by ISTAT) in order to extend its features also for creating, archiving and updating SDMX artefacts • Improve the Mapping Store database in order to store other SDMX artefacts as “provisional agreements” (calendars), "Agency" etc • implement a "wrapper" application on the SDMX registry in order to search and display information in a more friendly way