160 likes | 309 Views
Building a Data Portal with SDMX. Gabriele Becker, Massimo Bruschi Bank for International Settlements. The BIS SDMX Sandbox exercise. METIS, 7 May 2013. 1. The SDMX vision. Users need … good quality data, up-to-date numbers, documentation
E N D
Building a Data Portal with SDMX Gabriele Becker, Massimo Bruschi Bank for International Settlements The BIS SDMX Sandbox exercise METIS, 7 May 2013 1
The SDMX vision • Users need … • good quality data, up-to-date numbers, documentation • Single access point for data from different sources • User interfaces: GUI and also a Web service (for automation) • The SDMX vision … • Data providers (originators) offer their data “in SDMX” • Dissemination = reporting = data sharing … from a single storage • SDMX registries help users to find data via a GUI and an SDMX Webservice • The BIS SDMX Sandbox exercise • How “real” is this SDMX vision? • How does it work with difference scenarios?
The Sandbox setup “in the cloud” • 14 participating central banks: 14 Sandboxes with • SDMX registry with user interface and SDMX web service • SDMX data base with data browser and SDMX web service • Space to host SDMX data files • User interface for loading files, performing registrations etc. • Detailed tutorials • 1 Sandbox for the BIS to experiment and participate “as central bank” • The Data Portal (Unified Data Catalogue or “UDC”) • “central” SDMX registry operated by the BIS • Data browser • Synchronises with the central and CB registries • Accesses registered data sources • SDMX Sandbox was developed by Metadata Technology
Scenarios • Data reporting against a “global” SDMX DSD, eg BOP • Making internal data available via a “data portal” • Bringing external (public) data to your users via a data portal • Common aspects • data is available in SDMX files in a known DSD or … • SDMX web service offers data in a known DSD • We just need to know where the data is and what DSD is follows • An SDMX Registry is the place for this information • A simple data browser can offer data selection based on the known DSDs
Scenario 3: “making public data available in a data portal” • ECB publishes a lot of data as SDMX files on its website • IMF and BIS offer the Joint Ext. Debt Hub (JEDH) contributions as SDMX files • Users may want to see this data via a common data portal • Organisations may wish to access this data via a SDMX 2.1web service (that ECB, IMF and BIS currently do NOT offer) demo • Load SDMX DSD into registry demo • In future: SDMX registries provide this as a service … • Define Dataflow, Provision agreement and Categorisation • Register the data files as sources for the UDC demo
Conclusions I • Building blocks are working in the SDMX Sandbox • SDMX registry and SDMX 2.1 webservice • SDMX data base (with webservice) • Data browser • Unified Data Catalogue (data portal) • Connecting different data sources … • SDMX files from public websites • SDMX files from other (accessible) locations • Different SDMX versions (1.0, 2.0, 2.1) • SDMX databases (internal and external) • SDMX Web service • First implementation of a federated SDMX registry
Conclusions II • Harmonisation of data access techniques was achieved… • User sees all data through one “data portal” (the UDC) • SDMX 2.1 web service for all data, also for those offered as SDMX 1.0 files • Web service supports automated data retrieval • Works best with harmonised data structures! • BOP Sandbox example • Key deliverables for SDMX: DSD for global use for BOP and National Accounts … under way • BOP Sandbox created by IMF based on this exercise • The SDMX Vision is real!
Background slides • Scenario 1 and 2
Scenario I: “BOP” (Balance of payments) • National agencies have to “report” their data based on a commonly agreed data structure • This data needs to be available at international organisations and for the public • This is a real life scenario for BOP, National Accounts and other data domains. demo • SDMX BOP data structure “for global use” is a prerequisite • We invented a simplified one for the purpose of this exercise …
Scenario 2: “dissemination database for BIS” • BIS wishes to disseminate data to internal users via a common data portal (example: BIS Banking Statistics) • BIS has/defines SDMX DSD for this data demo • BIS uses the SDMX database in the Sandbox as “dissemination database” and loads the data via SDMX files demo • Internal users use the data browser (similar to UDC) to access the “BIS SDMX database”. demo • For external dissemination the BIS registers its Sandbox database as a data source to the UDC demo • This also enables the UDC search service for this data demo