SDMX at United Nations Statistics Division

1. SDMX at United Nations Statistics Division Experience, Challenges, Strategies

2. Background • United Nations Statistics Division is an SDMX Sponsor agency • Has used SDMX in its business processes since 2005 (UN Comtrade) • Participates in the development of global Data Structure Definitions (DSDs) • Uses SDMX in dissemination

3. Development of Global Data Structures • Under the auspices of Interagency Expert Group on Millennium Indicators, led the development of DSD for MDG indicators • Participates in the development of DSD for National Accounts • Leads the development of DSD for International Merchandise Trade Statistics • Development of DSD for Demography is in the initial stages

4. SDMX Exchange • Exchange of IMTS data between UNSD and OECD • Pilot project underway to exchange National Accounts data with IMF, Eurostat, OECD; full-scale exchange to be established after the pilot • As part of the UNSD-DfIDCountryData project, exchange of development indicator data was established with 11 countries • Several non-project countries have also joined the exchange • The system has been in production since 2011 and receives new data on a regular basis • SDMX-related modules (mapping, registry) were developed for the DevInfo software

5. SDMX Dissemination • Commodity Trade web site has disseminated international merchandise trade data since 2005 • CountryData disseminates development indicator data supplied by approximately 100 countries • Implementation of SDMX is underway at UNdata, UNSD’s flagship dissemination platform • 6 UNdata datasets are currently available, including World Population Prospects and Millennium Development Goals • Eurostat’s SDMX Reference Infrastructure is used as the engine

6. SDMX Issues: Technology • Extensibility • Countries very often wish to extend a standard DSD to customize it for their own needs, while some classifications explicitly provide for user-defined codes. • Metadata • In terms of tools available, limited support for metadata exchange in versions prior to SDMX 2.1 • Metadata is often stored and maintained outside of a database, which complicates the automation of metadata exchange.

7. SDMX Issues: National Statistical Office • Lack of IT expertise • Many of the developing countries are unfamiliar with SDMX or underlying technologies, as well as with the principles of data modeling. • Lack of established aggregated data processing and dissemination • Many countries do not use databases to store aggregated data, which complicates both reporting and dissemination. • Standards • Not all countries adhere to the latest versions of international classifications. • Lack of coordination between IT and Statisticians • SDMX is often treated as a technology project, and statisticians are not involved to the extent required – or at all.

8. SDMX Exchange Issues: International Agency • Complexity of data structures • Global Data Structure Definitions can be very comprehensive but also difficult to have the countries adopt and use them. • Mapping errors • Can lead to issues such as incorrectly coded or duplicate series, etc. This is a problem even for moderately complex DSDs. • Uncertainty • Scepticism about SDMX applicability at the global level given its perceived complexity

9. SDMX Exchange Solutions:Technology • Further refinement of the standard • Improve support for modern and emerging technological standards. • Better documentation • Better support for metadata • New tools oriented at beginners • Implementing SDMX support for existing tools

10. SDMX Exchange Solutions: National Statistical Office • Capacity building with assistance from international agencies • Improving data processing and dissemination • Formulating and implementing aggregated data dissemination strategy • Using standard tools and platforms • As a way to lower the barrier to entry

11. SDMX Exchange Solutions:International Agency • Address the complexity of comprehensive data structures • IMTS Working Group (UNSD, Eurostat, OECD, UNCTAD and ITC) broke data exchange into manageable data-flows, which results in less complexity than a single overarching DSD would create. • Support capacity building at national level • Support further development of the standard • Support further development of tools

12. SDMX Exchange: Recent Developments • Many capacity building projects underway (Eurostat, IMF, UNSD, ADB, …) • New tools greatly simplify dissemination and help lower the costs of implementation • SDMX Reference Infrastructure, Fusion Registry, DevInfo • Eurostat’s SDMX InfoSpace is an excellent resource to help master the technology • Global DSDs gradually become available • Greater awareness of SDMX and willingness to participate in exchange on the part of both countries and international agencies • Rapidly growing dissemination • Global SDMX Registry

13. SDMX Exchange: The Future • It is expected that SDMX Exchange will be established for Post-2015 development indicators, both with international agencies and countries • Building on MDG and CountryData experience • Opportunities for capacity building and increasing SDMX adoption rates at country level • Brings to the fore sub-national geography issues • Gradual implementation of exchange with Member States • Reducing reporting burden • Eurotrace software (used by 80 developing countries) planned to be adapted for SDMX Exchange in International Merchandise Trade Statistics

14. SDMX Exchange: The Future (2) • Expanding dissemination • Not only popularizes SDMX but enables and encourages use of advanced technologies such as API/Web services • Expanding exchange using the DevInfo software, available for over 100 countries • SDMX tools developed for the UNSD-DfID project are now part of the standard DevInfo package • Explore the possibilities for the use of SDMX in the context of Data Revolution

15. Thank you!