1 / 36

Meeting today’s dissemination challenges:

Meeting today’s dissemination challenges:. Implementing international standards in .Stat. Prepared by Jonathan Challener , OECD For MSIS , April 2014 - Dublin, Ireland. Doesn’t non-standard power supplies make things difficult?. What happens when standards are not applied well?.

dexter
Download Presentation

Meeting today’s dissemination challenges:

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Meeting today’s dissemination challenges: Implementing international standards in .Stat Prepared by Jonathan Challener, OECD For MSIS, April 2014 - Dublin, Ireland

  2. Doesn’t non-standard power supplies make things difficult?

  3. What happens when standards are not applied well?

  4. Confusion entails Picture: ‘The day Sweden changed from left-hand drive to right’

  5. This all adds up…

  6. …high costs…

  7. and inefficiencies!

  8. and inefficiencies! “A little like the grade 8 student who doesn’t pay attention in class all year”.

  9. What is .Stat?

  10. What is .Stat? .Stat is the central repository ("warehouse") of validated statistics and related metadata .Stat is the central hub connecting data production, sharing & dissemination processes It is the corporate source of data for data sharing and disseminationpurposes

  11. What is .Stat? “.Stat is now being used and shared with 10 organisations including the OECD, as part of the Statistical Information System Collaboration Community (SIS-CC)”. .Stat is the central repository ("warehouse") of validated statistics and related metadata .Stat is the central hub connecting data production, sharing & dissemination processes It is the corporate source of data for data sharing and disseminationpurposes

  12. .Stat Positioning in Statistical Information System INTERNAL DATA SHARING DATA DISSEMINATION DATA DELIVERY .STAT DATA PRODUCTION

  13. “The diagram illustrates the .Stat contribution to the SIS processes. .Stat’s core value-added lies in “Data Delivery”, a set of functions that enable dissemination and data sharing, and “Data Upload”, a set of functions interfacing data production processes into a single upload mechanism to feed dissemination channels”. .Stat Positioning in Statistical Information System INTERNAL DATA SHARING DATA DISSEMINATION DATA DELIVERY .STAT DATA PRODUCTION

  14. .Stat Functional Representation DATA DISSEMINATION DATA SHARING C P SDMX OUTPUT .STAT BROWSER DATA ANALYSIS TOOLS SEARCH ENGINES WEBSITES, APPS PUBLICATIONS .Stat Component X SDMX GLOBAL REGISTRY X DATA EXTRACTION SERVICES TABLE & CHART EXTRACTIONSERVICES .STAT BROWSER CONFIGURATION Process PUBLISHING BACK OFFICE API or Webservice .STAT DATA DELIVERY ENGINE RELEASE MGT SERVICES E Other SDMX hubs Human user Data Producer Data Editor Data Consumer DATA PRODUCTION .STAT DATA UPLOAD ENGINE BATCH UPLOAD FILE UPLOAD DATA MAPPING SDMX IMPORT DATA PRODUCTION TOOLS SDMX INPUT P

  15. “The grey shaded boxes in the figure below show a visual representation of how .Stat fits within a broader Data Dissemination Information System of organisations; the boxes with dotted lines represent other components of the Data Dissemination Information System that are not supported by .Stat but are enabled by it”. .Stat Functional Representation DATA DISSEMINATION DATA SHARING C P SDMX OUTPUT .STAT BROWSER DATA ANALYSIS TOOLS SEARCH ENGINES WEBSITES, APPS PUBLICATIONS .Stat Component X SDMX GLOBAL REGISTRY X DATA EXTRACTION SERVICES TABLE & CHART EXTRACTIONSERVICES .STAT BROWSER CONFIGURATION Process PUBLISHING BACK OFFICE API or Webservice .STAT DATA DELIVERY ENGINE RELEASE MGT SERVICES E Other SDMX hubs Human user Data Producer Data Editor Data Consumer DATA PRODUCTION .STAT DATA UPLOAD ENGINE BATCH UPLOAD FILE UPLOAD DATA MAPPING SDMX IMPORT DATA PRODUCTION TOOLS SDMX INPUT P

  16. .Stat Functional Representation In particular, .Stat provides the following 3 key functional areas…

  17. .Stat Data Upload Engine .Stat Functional Representation

  18. .Stat Data Delivery Engine .Stat Functional Representation

  19. .Stat Data Browser .Stat Functional Representation

  20. .Stat Positioning in GSBPM Reference Model

  21. “.Statcan be mapped today to the Generic Statistical Business Process Model (GSBPM) under “Disseminate” and “Build”. In the future it will also incorporate archive functions as part of the over-arching process for data and metadata management”. .Stat Positioning in GSBPM Reference Model

  22. Multipurpose SDMX within .Stat…

  23. For dissemination and data eXchange • SDMXWS and RESTful API • SDMX 2.0 compliant • SOAP + REST • Pull • SDMX-ML • SDMX Structural metadata created on the fly

  24. For ‘Open Data’ dissemination SDMX-JSON (beta) SDMX-TWG agreed in mid 2013 on proposal for data and their structural metadata (inc. flat & sliced layouts) and referential metadata (dataset, series, obs) as annotations. Further enhancements to come: Complete data structures and referential metadata

  25. Mapping Assistant SDMX-RI Web Service For data reporting • SDMX-Reference Infrastructure (RI)* • SDMX 2.0 and 2.1 compliant • SOAP + REST • SDMX Common APIs (SdmxSource.NET) • Pull + Push • SDMX-ML, GESMES , CSV • Structural metadata stored in mini registry • One web service - several mapped database instances XXX.Stat Data warehouse Mapping Store DB Dissemination SDMX-RI * The integration of SDMX-RI in .Stat is based on collaboration with Eurostat, provider of the SDMX-RI component with ISTAT taking the lead on behalf of the OECD’s Statistical Information System Collaboration Community.

  26. For internal data sharing Excel-add-in • DirectAccess(Rest SDMX) • Native Excel pivot table • Wizard to select data DirectAccess Restful SDMX query Flat data, flags, units Referential metadata

  27. For a decentralised publishing environment • DataHub* • One interface to the publishing tools • Centralised reporting and auditing • SDMX based structural metadata, and referential metadata management • Flexible load tool that promotes ‘self publish’ for data custodians • In-built checks and safeguards to minimise errors • Manages security and access rights • Can be extended to manage other outputs and not limited to .Stat * DataHubhas been developed and integrated with .Stat by Statistics NZ, with an additional connection to the Fusion Registry for managing structural metadata through the definition of DSDs.

  28. Future outlook…

  29. Further SDMX artifact support

  30. SDMX ingest (Import)

  31. SDMX global registry API

  32. SDMX-RDF data cube vocabulary pilot

  33. SDMX-RDF data cube vocabulary pilot “Explore further semantic web/linked data opportunities (SDMX-RDF data cube vocabulary). To be taken forward by ISTAT and ABS under the SIS-CC umbrella”.

  34. We all know the… • Reduced focus on infrastructure • Ability to create composite interfaces that are tailored to the needs of specific task • Improved application portability • Enable faster time to market because it is easier to use off the shelf components and applications that can integrate and provide features for the solution Lower technology adoption costs Increased development consistency, simplicity and predictability Improved code reuse Reduced cost, time and effort to transition between different solutions

  35. References Operationalising .Stat in a decentralised publishing environment (DataHub) by Tony Breen SNZ : https://community.oecd.org/docs/DOC-68362 Building a scalable architecture (.Stat) by Jens DosseOECD: https://community.oecd.org/docs/DOC-68363 SDMX-RI and .Stat integration by Francesco Rizzo Istat: https://community.oecd.org/docs/DOC-68696 SDMX-JSON API: http://stats.oecd.org/opendataapi/Index.htm

  36. Thank you Jonathan Challener, OECD jonathan.challener@oecd.org @Challener MSIS - Dublin, 14-16 April2014 Meeting today’s dissemination challenges: Implementing international standards in .Stat

More Related