1 / 18

SDMX Open Source Software

23 April 2011 Eurostat Luxembourg. SDMX Open Source Software. Erik van Ingen FAO/D4Science. D4Science-II project |. www.d4science.eu. Content. OpenSDMX is used in FAO/TechCDR FAO/Fisheries ( www.fao.org/figis/sdmx )

nonnie
Download Presentation

SDMX Open Source Software

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 23 April 2011 Eurostat Luxembourg SDMX Open Source Software Erik van Ingen FAO/D4Science D4Science-II project | www.d4science.eu

  2. SDMX open source software Content • OpenSDMX is used in • FAO/TechCDR • FAO/Fisheries (www.fao.org/figis/sdmx) • EU/D4Science (http://opensdmxdevelopers.wikispaces.com/PisaCNR2011January) • Looking at ECB & FED • Why OpenSDMX? • OpenSDMX pillars • OpenSDMX Development Environment & Technical Architecture • OpenSDMX Communication channels • Synergy analyses MT, Eurostat & OS • JAXB and XMLBeans SDMX Open Source Software 23 April 2011

  3. SDMX open source software TechCDR :Corporate Data Repository • TechCDR is a platform of services • Provide external access to FAO digital assets • Search and access statistics, maps, feature layers, documents, imagery, etc. • External APIs • Provide internal services to Analytical Field Systems • Workspace management, ETL uploads, security, publishing, notification and update alerts

  4. SDMX open source software ETL Jobs (Talend OS) GIS Services (Geoservers) TechCDR Platform front-end layer Portlet container Portlets, Javascript, Flex components Working System Metadata Manager Ontologies Manager Custom Applications service layer Working System Services Business Services Integration Services JBoss ESB Content Management Services (Alfresco) Metadata Services Infrastructural Services (Security, log, cache, etc) Data Analysis Services (Pentaho) SDMX Services Statistical Data Services data layer Staging Area Working System RDBMS Multidimensional Data Model (DWH) Metadata Repository Statistical Data Repository Content Repository Structured Data Repository

  5. SDMX open source software TechCDR use of OpenSDMX • TechCDR publishes multidimensional data and artifacts in SDMX • OpenSDMX features: • OpenSDMX repository services obtain datasets from OLAP database • OpenSDMX services dynamically map data in OLAP cubes to SDMX datasets • Codelists, Codes, DSDs are published

  6. SDMX open source software TechCDR SDMX Services • OpenSDMX services exposed via • Restful API following SDMX 2.1 guidelines • TechCDR Custom • Interfaces to OpenSDMX services

  7. SDMX open source software TechCDR SDMX Services REST Controller front-end layer SDMX Services service layer Olap4j Relational Data Services XMLA Data Analysis Services (Pentaho) Multidimensional Data Model (DWH) Metadata Repository Statistical Data Repository data layer

  8. SDMX open source software OpenSDMX and D4Science Codelist uploader

  9. SDMX open source software Looking at the the ECB SDMX visualisation framework: • Federal Reserve Bank of New York and European Central Bank

  10. SDMX open source software Why OpenSDMX? • A platform was needed in order to serve three different projects • The platform needed to be open source in a full fledged manner. • FAO and D4Science needed to have a good governance position in the open source project. SDMX Open Source Software 23 April 2011

  11. SDMX open source software OpenSDMX pillars • Implementation of version 2.1 REST webservice specification • Supporting (only) version 2.1 (still migrating yet from version 2.0) • Focus on core SDMX Artefacts (Codelist, DSD, Generic and Compact Dataset) • Integratable in every context through adapters • Not necessarily having its own persistence layer, maybe added in the future SDMX Open Source Software 23 April 2011

  12. SDMX open source software OpenSDMX Development Environment • Building with Maven • Continuous Integration with Continuum • Code Metrics with Sonar & FindBugs • Artifact server with Artifactory • SCM with Subversion on SourceForge • Documentation on Wikispaces (http://opensdmxdevelopers.wikispaces.com/) • Eclipse IDE mostly used amongst developers • Automatic deployment on Tomcat • Integration testing • Unit Testing

  13. SDMX open source software OpenSDMX Technical Architecture • Spring for dependency injection, REST and caching • JAXB as the Java binding for XML • EHCache for caching • Java 6

  14. SDMX open source software OpenSDMX Communication channels • https://sourceforge.net/p/opensdmx/home/ • http://opensdmx.wikispaces.com/ • http://opensdmxdevelopers.wikispaces.com/

  15. SDMX open source software Synergy analyses MT, Eurostat & OS

  16. SDMX open source software JAXB and XMLBeans I • Metadata Technology uses XMLBeans, OpenSDMX uses JAXB. Eurostat uses a proprietary XML binding. • XMLBeans • Email traffic from 18/02/2011-18/03/2012 is 15 (10 developermail, 10 usermail) • Last release 2.5.0 was on 14-12-2009 • JAXB • Email traffic from 18/02/2011-18/03/2012 is 70 usermail • Last release 2.2.3.1 was on 16/01/2011

  17. SDMX open source software JAXB and XMLBeans II • Observations: • Spring supports Castor, Jaxb, Jibx and XmlBeans • XmlBeans is less active than JAXB • JAXB is natively part of Java

  18. END End of Presentation Erik van Ingen FAO/D4Science SDMX Open Source Software 23 April 2011

More Related