110 likes | 117 Views
Explore how SDMX revolutionized data management from manual entry to automatic updates, saving time, reducing errors, and improving standardization. Learn about the challenges faced and the solutions implemented. Discover the benefits and considerations for future applications.
E N D
7th SDMX Global Conference 16-19 September Budapest - Hungary SDMX: From Labour Force Department to the Statistical Database Houssam El Ouinkhir Kamel Gaanoun h.elouinkhir@hcp.ma k.gaanoun@hcp.ma
Statistical Database • Created early 80s. • Contains 900 indicators. • Covers various themes: social, demographic and economic. • Published under open data recommendations via this link: bds.hcp.ma (214 indicators published)
SDB updating mode LFS Data • More than 170 LFS indicators, including quarterly and annual results both for urban and rural areas • Generated from SPSS in an Excel format by the LFS department • Sent regularly by email to the SDB unit • All indicators are manually entered in the SDB by the SDB unit technicians • if any update is needed, the same process is then repeated
The challenge Human error Time consuming Manual data entry Non standard input Excel format
The SDMX Solution Enable automatic SDB Update Mapping DSD SPSS Construct DSD Generate SDMX file Mapping DSD SDB 1 2 3 4 5
The SDMX Solution DSD No common LFS DSD was available at the time of the implementation Various sources were used : With respect of SDMX Guidelines (COG,DSD Design, Cross domains CL) SDMX Global Registry IMF SDMX Central EURO SDMX Registry HCP’s Specific code_lists Use of the Eurostat Data Structure Wizard (DSW) for DSD generation.
The SDMX Solution From SPSS to SDMX • First solution : • Generate Excel files from SPSS • Convert to SDMX using Eurostat SDMX Converter • Second solution : • Generate XML file from SPSS • Extract Dimensions from the XML • Map to the DSD
The SDMX Solution SDB automatic update The SDB automatic update is enabled towards the following two steps: • Mapping between SDB and Labor force DSD. • Using mapping file and SDMX file to update the SDB. The SDB update is done by a python batch.
Conclusion SDMX allowed us to : • Reduce human-error rate • Save data-entry time • Standardise the update • Get ready to futur SDMX exchanges Even if : • Learning/training consumes time • DSD construction (especially for LF) isn’t straightforward • Generalisation of the process to other operations should be studied (training, concepts complexity, DSD’s Availability…)
THANKS Houssam El Ouinkhir Kamel Gaanoun h.elouinkhir@hcp.ma k.gaanoun@hcp.ma