210 likes | 485 Views
Case Study. Integrated Metadata Driven Statistical Data Management System (IMD SDMS) CSB of Latvia Julija.Drozdova@csb.gov.lv METIS 2010. Outline. The main steps for IMD SDMS creation IMD SDMS fundamental elements Costs & benefits IMD SDMS implementation strategy
E N D
Case Study Integrated Metadata Driven Statistical Data Management System (IMD SDMS) CSB of Latvia Julija.Drozdova@csb.gov.lv METIS 2010
Outline • The main steps for IMD SDMS creation • IMD SDMS fundamental elements • Costs & benefits • IMD SDMS implementation strategy • GSBPM versus SBPM of CSB • Current situation and further developments • The main lessons learned • Proposal for GSBPM
The main steps for IMD SDMS creation (1) • Data and metadata collection (1999) • Thoughtful analysis of data and metadata flows (1999) • To set the requirements to the system (1997-1999)
The main steps for IMD SDMS creation (2) the main requirements to IMD SDMS were: • covers full cycle of statistical data processing; • uses process oriented approach; • IMD SDMS must be: - standardized; -integrated; - meta data-driven; - allows automatedgeneration of user application forms (incl. web); -centralized; - has a modular structure; - transparent;
IMD SDMS fundamental elements (1) • Core Meta data base module handles all processes of IMD SDMS • Structure of Micro data [Bo Sundgren model] Objects characteristics: Co = O(t).V(t) where: O - is an object type; V - is a variable; t - is a time parameter. Every results of observations is a value of variable (data element) – Co • Two types of tables • Structure of Macro data
IMD SDMS fundamental elements (2) • Structure of Micro data (an example)
IMD SDMS fundamental elements (3) • Two types of tables: - fixed table (data matrix); - open table (data matrix with various number of rows or columns); Questionnaire consists of chapters and chapters consist of tables.
IMD SDMS fundamental elements (4) • Structure of Macro data The estimations are made on the basis of a set of Micro data. Statistical characteristics: Cs = O(t).V(t).f where: O and V - is an object characteristics; t - is a time parameter, f – is an aggregation function (sum, count, average, etc) summarizing the true values of V(t) for the objects in O(t).
Costs & benefits • Standardization of statistical data production processes • The basis for the CSB regional restructuring (2003-2004): 5 Data Collection and processing centres replaced previously existing 26 Statistical Regional offices and city Riga office; • Decreasing of statisticians from 180 to 115
IMD SDMS implementation strategy (1) • Step-wise approach • 1997 – 1999 CSB and PricewaterhouseCoopers experts were prepared General Technical Requirements for the project “Modernisation of CSB – Data Management System”
IMD SDMS implementation strategy (2) • The main requirement: Meta data should be used as the key element in statistical data processing • Additional requirements: - Increase efficiency of the production ofstatistical information; - Avoid hard code programming via standardisation of procedures and use of Meta data within the statistical data processing; - Increase the quality of the information produced; - Improve processes of statistical data analysis; - Modernise and increase the quality of data dissemination;
GSBPM versus SBPM of CSB (draft version) GSBPM versus SBPM of CSB ~51 %
Further developments • Since 2009 a project has been launched for the IMD SDMS to cover Social statistics domain. Starting from: - Population Census; - Agricultural Census; - Labour Force Survey; - EU-SILC …
The main lessons learned (1) • Design of the new information system should be based on the results of deep analysis of statistical surveys: - statistical questionnaires and variables; - statistical processes and data flows; • Statistical data processes and “Variables and questionnaires system” must be harmonized and standardized before creation of the new system;
The main lessons learned (2) • The system should provide a full cycle of statistical data processing; • The systemshould be: - standardized; -integrated; - meta data-driven; - allows automatedgeneration of user application forms (incl. web); -centralized; - has a modular structure; - transparent;
The main lessons learned (3) • Motivation of the statisticians to move (from stove-pipe to process oriented) to the new data processing environment is essential; • To establish Metadata group; • Data electronic archiving reduces human resources, expenses of CSB for deposition in the State Archives, time of archiving and physical amount of archiving information (In 2000, Population Census - 21 m3 = 4 DVD)
Proposal for GSBPM (1) • Extension of phase 4 – Collect,between sub-processes 4.1 and 4.2 • Extension, between sub-processes 4.3 and 4.4 Why ?: - statistician’s work with respondents and with the list of respondents is a very difficult,heavy process and time consuming process (…; sending of letters to respondents; conduction of the respondents lists;creation of the sample Matrix;clarifications; response control; reminding process; …); - sometimes statistician’s work is pressed for time (…Business tendencies survey…)
Proposal for GSBPM (2)Survey’s integration From analytic’s view List of indicators • From statistician’s view: • amount of work • respondents burden • statisticians burden • response control • etc. Sample Matrix … From mathematician’s view