1 / 54

ESCWA SDMX Workshop

ESCWA SDMX Workshop. Session: SDMX and Data. Session Objectives. At the end of this session you will: Know the SDMX model of a data structure definition Understand the techniques to identify the structure of data Identify the concepts in a simple data set

irisa
Download Presentation

ESCWA SDMX Workshop

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ESCWA SDMX Workshop Session: SDMX and Data

  2. Session Objectives • At the end of this session you will: • Know the SDMX model of a data structure definition • Understand the techniques to identify the structure of data • Identify the concepts in a simple data set • Be able to develop simple data structure definitions using SDMX tools

  3. Data Set

  4. Data Set: Structure

  5. Data Set Structure • Computers need to know the structure of data in terms of: • Concepts • Code Lists • Dimensionality • Additional metadata

  6. First: Identify the Concepts • A concept is a unit of knowledge created by a unique combination of characteristics (SDMX Information Model)

  7. Stock/Flow Country Unit Multiplier Unit Time/Frequency Topic Data Set Structure: Concepts

  8. TOPIC COUNTRY STOCK/FLOW A Brady Bonds B Bank Loans C Debt Securities AR Argentina MX Mexico ZA South Africa 1 Stock 2 Flow CONCEPTS Topic Country Flow Data Set Structure: Code Lists Concepts Code Lists

  9. 16457 Data Makes Sense Q,ZA,B,1,1999-06-30=16457

  10. Data Set Structure: Defining Multi-dimensional Structures • Comprises • Concepts that identify the observation value • Concepts that add additional metadata about the observation value • Concept that is the observation value • Any of these may be • coded • text • date/time • number • etc. Dimensions Attributes Measure Representation

  11. Stock/Flow Country Unit Multiplier Unit Time/Frequency Topic Observation Data Set Structure: Concept Usage (Dimension) (Dimension) (Attribute) (Attribute) (Dimension) (Dimension) (Dimension) (Measure)

  12. CONCEPTS Topic Country Flow Data Structure Definition concepts that identify groups of keys concepts that identify the observation Key Group Key concepts that are observed phenomenon concepts that add metadata Attributes Measures Dimensions has format takes semantic from has format takes semantic from takes semantic from Representation Non-coded Concept Coded has code list has format TOPIC A Brady Bonds B Bank Loans C Debt Securities Code List

  13. 16457 Data Makes Sense Frequency,Country,Topic,Stock/Flow,Time=Observation Q,ZA,B,1,1999-06-30=16457 Quarterly, South Africa, Bank Loans, Stocks, 2nd quarter 1999

  14. Identifying Concepts • Identifying Concepts - Sources • Existing data set tables • From website • From applications • Data Collection Instruments • Questionnaires • Excel spreadsheets • Regulations, Handbooks, User Guides • Labour Statistics Convention, 1985 (No. 160), Recommendation, 1985 (No. 170) • Council Regulation No: 311/76/EEC of 09/021976; OJ: L039 of 14/02/1976; Compilation of statistics on foreign workers • Database Tables • Existing Data Structure Definitions • From other organisations

  15. Identify Concepts – from website Measurement = 1,000 Kg Source: FAO proof of concept project

  16. Concepts Measure Type Observation Value Frequency and Time Commodity Reference Region Measurement = 1,000 Kg Unit and Unit Multiplier

  17. Exercise: Identify Concept Role

  18. Concept Role: Reminder • Dimensions • Are the concepts that identify the observation value • Attributes • Are the concepts that add additional metadata about the observation value • Measure • Is the concept that is the observation value

  19. Concepts Measure Type Observation Value Frequency and Time Commodity Reference Region Measurement = 1,000 Kg Unit and Unit Multiplier

  20. Exercise:Concept Role Measure Type Observation Value Frequency and Time (Dimension) (Measure) (Dimensions) Commodity (Dimension) Reference Region (Dimension) Measurement = 1,000 Kg Unit and Unit Multiplier (Attributes)

  21. Data Set and Structure

  22. Identify/Define Code Lists • Purpose of a Code List • Constrains the value domain of concepts when used in a structure like a data structure definition • Defines a shortened language independent representation of the values • Gives semantic meaning to the values, possibly in multiple languages • Agreeing on harmonised code lists is the most difficult aspect of defining a data structure definition

  23. Code Lists Required Measure Type Frequency Commodity Reference Region Source: FAO proof of concept project Measurement = 1,000 Kg Unit and Unit Multiplier

  24. Code Lists

  25. Code Lists

  26. Code Lists (CL_) For Time Series the SDMX Cross Domain Concepts recommend all observations have a status code (Concept = OBS_STATUS) and a confidentiality code (Concept = OBS_CONF)

  27. Data Structure Definition

  28. Data Structure Definition - Reminder Data Structure Definition concepts that identify groups of keys concepts that identify the observation Key Group Key concepts that are observed phenomenon concepts that add metadata Attributes Measures Dimensions has format takes semantic from has format Representation takes semantic from takes semantic from Non-coded Coded Concept has code list has format Code List

  29. Data Structure Definition - Agriculture Data Structure Definition AGRICULTURE_COMMODITY Key Group Key FREQREF_AREA_REGCOMMODITYMEASURE_TYPETIME Attributes Measures Dimensions OBS_STATUSOBS_CONFUNITUNIT_MULT CL_FREQCL_AREA_CTYCL_COMMODITYCL_MEASURE_ELEMENT OBS_VALUE Representation Concept Non-coded Coded CL_OBS_STATUSCL_OBS_CONFCL_UNITCL_UNIT_MULT Code List

  30. SDMX and Data Formats Exercise: Identify Concepts

  31. Identifying Concepts • Identifying Concepts - Sources • Existing data set tables • From website • From applications • Data Collection Instruments • Questionnaires • Excel spreadsheets • Regulations, Handbooks, User Guides • Labour Statistics Convention, 1985 (No. 160), Recommendation, 1985 (No. 170) • Council Regulation No: 311/76/EEC of 09/021976; OJ: L039 of 14/02/1976; Compilation of statistics on foreign workers • Database Tables • Existing Data Structure Definitions • From other organisations

  32. Identifying Concepts • Identifying Concepts - Sources • Existing data set tables • From website • From applications • Data Collection Instruments • Questionnaires • Excel spreadsheets • Regulations, Handbooks, User Guides • Labour Statistics Convention, 1985 (No. 160), Recommendation, 1985 (No. 170) • Council Regulation No: 311/76/EEC of 09/021976; OJ: L039 of 14/02/1976; Compilation of statistics on foreign workers • Database Tables • Existing Data Structure Definitions • From other organisations

  33. Exercise: Identify Concepts – from collection instrument Source: UNESCO Institute for Statistics

  34. Data Entry - Table 2.1 Source: UNESCO Institute for Statistics

  35. Data Entry - Table 2.2 Source: UNESCO Institute for Statistics

  36. Identifying Concepts • Identifying Concepts - Sources • Existing data set tables • From website • From applications • Data Collection Instruments • Questionnaires • Excel spreadsheets • Regulations, Handbooks, User Guides • Labour Statistics Convention, 1985 (No. 160), Recommendation, 1985 (No. 170) • Council Regulation No: 311/76/EEC of 09/021976; OJ: L039 of 14/02/1976; Compilation of statistics on foreign workers • Database Tables • Existing Data Structure Definitions • From other organisations

  37. Exercise: Identify Dimension Concepts – from website Source: International Labor Organisation

  38. Identify Concepts: Table 2A Source: International Labor Organisation

  39. Identify Concepts: Table 2B Source: International Labor Organisation

  40. Identify Concepts: Table 2C Source: International Labor Organisation

  41. Identify Concepts: Table 2D Source: International Labor Organisation

  42. Identify Concepts: Table 2E Source: International Labor Organisation

  43. Identify Concepts: Table 2A Measure Type Reference Area Time Period Frequency Sex

  44. Identify Concepts: Table 2B Measure Type Economic Activity

  45. Identify Concepts: Table 2C Measure Type OCCUPATION

  46. Identify Concepts: Table 2D Measure Type Status in Employment

  47. Identify Concepts: Table 2E Measure Type

  48. Exercise: Identify Concepts – from collection instrument Reference Area Time Source: UNESCO Institute for Statistics

  49. Dimension Concepts - Tables 2.1/2.2 Education Level Institution Type Measure Type Sex Programme Orientation Work Mode Source: UNESCO Institute for Statistics

More Related