80 likes | 242 Views
OECD Expert Group on Statistical Data and Metadata Exchange. Geneva 10-11 May 2007 OECD.Stat SDMX Web Service (II) Jens Dossé, OECD. OECD SDMX web service. Fully conforms to SDMX standard v2
E N D
OECD Expert Group on Statistical Data and Metadata Exchange Geneva 10-11 May 2007 OECD.Stat SDMX Web Service (II) Jens Dossé, OECD
OECD SDMX web service • Fully conforms to SDMX standard v2 • Currently two methods: extract data in GenericData message extract Data Structure Definitions (Key Families), concepts and code lists • Method planned for the future: extract Metadata • Secured access: integrates authorization to OECD.Stat • Still needs a few additional conventions bridging SDMX-ML limits
Additional conventions I • For time series data, independently of the presence of a FREQUENCY dimension, the requested time frequency must be specified using a TIME_FORMAT attribute:<query:And><query:Attribute name="TIME_FORMAT">P1M</query:Attribute><query:Time><query:StartTime>1960-01</query:StartTime><query:EndTime>2005-12</query:EndTime></query:Time></query:And>Format values: P1Y=annual, P6M=semi-annual, P3M=quarterly and P1M=monthly data Time formats: yyyy, yyyy-mm or yyyy-mm-dd with digits y=year, m=month and d=day, e.g. '2005-09-01'. This attribute is then also present in the resulting generic data message at the Series level of time-series data. • Non-time-series data: omit all time specifications in the query message The response messages will use 9999 as (obligatory) time periods • NEW: If the <query:StartTime> or <query:EndTime> nodes are not present then all the available values are returned
Additional conventions II • OECD.Stat data warehouse:Each dataset (multidimensional cube) has its own Data Structure Definition. Datasets can share common dimensions but which can have additional non-common codes • Several datasets within one message Query message can request data from several datasets NEW: Different datasets can now be returned within one SDMX generic message using “MessageGroup”s
Additional conventions III • For simplicity during the start-up of this generic web service, the returned message headers only contain required information • Maximum size of generic data message: 5 MB message is truncated if necessary Requests need to be split if this limit is exceeded
(Still) open challenges for SDMX web services • !!!: Big message sizes for any data format: need standard for compression • !!!: Security: need standard for authentication/encryption • !!!: Need for another message format: generic cross-sectional format remove the special status of the Time dimension (as not all statistical data are time series) • Registering SDMX web services in the SDMX registry • Multilaterally connecting organisations’ SDMX web services to build international networks
OECD SDMX web service client example • Simple web application • Provides example query messages • Source code freely available • Secured access to all data in data warehouse (integrates authentication and authorization to OECD.Stat)
Contacts/Links • jens.dosse@oecd.orgdouglas.paterson@oecd.org • http://stats.oecd.org/OECDStatWS_SDMX/SDMX.asmx • http://stats.oecd.org/OECDStatWS_SDMX/Test_SDMX.aspx