1 / 21

WP Item 6

WP Item 6. Expressions and Calculations . Background. The SDMX 2.0 and 2.1 versions already envisaged the introduction of a standard language for validations and calculations

raiden
Download Presentation

WP Item 6

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. WP Item 6 Expressions and Calculations SDMX Technical Working Group Luxembourg

  2. Background The SDMX 2.0 and 2.1 versions already envisaged the introduction of a standard language for validations and calculations The SDMX 2.1 package n. 13 (Transformations and Expressions) is a generic model aimed to track the validation and the calculation of data, derived from the CWM (Common Warehouse Metamodel), a OMG standard (Object Management Group) However this model is not operational in-itself, because it requires a language to specify the validation and calculation expressions SDMX Technical Working Group Luxembourg

  3. Some outcomes of the TWG WP5 A language for validations and calculations is needed to support the statistical processes Other standards already have a calculation language (e.g. XBRL Formula) Both SDMX and other standards like DDI and GSIM, which don’t have it, are interested in adopting a calculation language Some institutions already adopted a “reference” language, (e.g. Eurostat, Unesco, Banca d’Italia …), in some cases for internal purposes and sometimes also to communicate with their reporting entities Other institutions are willing to adopt a standard language (e.g. FAO …) SDMX Technical Working Group Luxembourg

  4. The assignment At the end of 2012 the SDMX Secretariat launched the WP Item 6 (Expressions and Calculations), aimed to introduce in the standard a language for validations and calculations, so making the SDMX 2.1 package n. 13 (Transformations and Expressions) fully operational The Secretariat asked to the TWG of elaborating a note, to be shared with the SWG, for describing the work and defining an implementation plan The Secretariat also asked to raise thepriority of this item as possible SDMX Technical Working Group Luxembourg

  5. Paris Meeting Dec. 2012 The TWG discussed this topic in Dec. 2012: The basic goals of the WP Item were identified The aim and the general model of the SDMX Transformations and Expressions wereassessed Three existing business cases were presented Bank of Poland use of XBRL Formula Eurostat VIP on Validation project Banca d’Italia EXL Some requisites of the language were discussed The interested TWG members were identified (the relevant documentation is in the EBIS site, folders: TWG Documents/Deliverables/Expressions and Calculations TWG Documents/TWG Meetings/2012-12)

  6. Some general requisites User orientation Integrated Approach IT implementation independence Active Role for processing Extensibility and customizability Language Effectiveness

  7. The Note for the Secretatiat A first version of the Note was draft according to the outcome of the Paris meeting and was circulated in the TWG and in the SWG It emerged the need ofclear business cases, which were discussed in a joint TWG-SWG teleconference; It was decided to: Add the description of the business cases that the language should support,which is mainly the definition and the exchange of validation rules; Clarify that an “agnostic” solution should be pursued, to be potentially employed in different standards (e.g. SDMX, DDI, GSIM); Form a joint TWG-SWG subgoup Prepare a realistic plan for the implementation of the language, according to the SWG and TWG resources available to dedicate a significant amount of time to work concretely on this topic

  8. The business cases Most of the institutions need a standard language for internal validation Many institutions need a standard language also to share validation rules with reporters Some organizations also share with reporters common data production rules, to guarantee data quality and consistency and lower the reporting burden Some organization uses a reference language also for internal compilation, for example: For achieving an automated compilation (industrial process) For enforcing an harmonized compilation made by different people (e.g. banks’ supervision analysis) For preserving the know-how of the compilation agorithms (if employees change, if tools change)

  9. The priority For most organizations the Validation is the priority: • In the first step, aim at theValidation, to achieve faster the main goal • Support validations also with the needed algorithm for calculations • Adopt a language open to improvements • Face the compilation in following steps (facing also more complex algorithm)

  10. The effort available The interested SWG members were identified to form a joint TWG-SWG subgroup It was asked to the interested members the FTE available to WP6in the next year and a half (from June 2013 to Dec. 2014) At the moment, IMF, Eurostat and Banca d’Italia gave their concrete availability At the moment, the effort available is: 1,1 FTE of person-power Consultancy in 2014 for about 0.5 FTE for 6 months

  11. Goals of the work Make it possible to: define the algorithms for data validation and calculation exchange such algorithms between different organizations use such algorithms in the collection and production processes

  12. May a language be agnostic? • Agnostic: independent of the particular standard and applicable to all of them (SDMX, DDI, GSIM) • Property of closure: a language must operate the model artefacts to produce other model artefacts • A language built for SDMX wouldn’t fit DDI & GSIM and vice-versa, because the artefacts are different • How can a shareable languagebe built ?

  13. The first idea • If it existed a formal and unequivocal mapping, the same language could be applied to the matching artefacts • Last february a task force has been formed within the DDI-SDMX dialogue to map GSIM with SDMX and DDI • Some SDMX TWG and SWG members has been involved and a mapping has been done • At the moment, the mapping isn’t formal and unequivocal enough • Moreover, alternative mappings are possible as a consequence of alternative profiles of application of the standards

  14. The idea of the “agnostic” model • Refer the language to an agnostic information model, in principle independent of SDMX, DDI, GSIM • The different standards may adopt the language by mapping their artefacts to the agnostic model • The model should be as much minimal and simple as possible, just to support validations and calculations

  15. Possile Workstreams Language for defining expressions (expression language) – that is the list of the operators available and their formal grammar SDMX IT formats and messages for exchanging the definitions of the expressions SDMX Web Services to interact with the registry (dictionary), for storing the expressions and retrieving them SDMX Technical Working Group Luxembourg

  16. Possible working steps for the language Identify the use cases • Data to be supported • Algorithms to be supported Design the reference model Define the language general behaviour • Different dimensionality • Missing / unknown data • Aggregation on hierarchies • Historicity of expressions • Multi-measure and attributes behaviour • … Identify the operators and their syntax

  17. Proposed Plan Use cases for Validation: • From eachinstitution: end of August 2013 • Merged in a common list: end of October 2013 • Finalization of the cases: next SDMX TWG meeting Reference Model Design: Jan – March 2014 Language general behaviour: Jan – March 2014 Operators and theirsyntax: • Basic set: April – June 2014 • Integration and finalization – July-Oct 2014 • Documentation – Oct-Dic 2014

  18. SDMX activities SDMX IT formats and messages: • Design: Jan – June 2014 ??? • Formalization: July – Oct 2014 ??? SDMX Web Services: • Design: Jan – June 2014 ??? • Formalization: July – Oct 2014 ???

  19. Requisites for the Governance • Standard language under a common governance • Need of customized parts under the private governance of single institutions, which integrate the language for their own purposes • Coordination rules between the standard part and the customized parts

  20. Wednesday 9.30 – 13.00 - special session • Presentation of the work made in Bank of Italy • Presentation of the work made in Eurostat • Presentation of the work made in Unesco • Discussion • Organization of the future workand agreement on the work plan

  21. WP Item 6 Expressions and Calculations SDMX Technical Working Group Paris

More Related