120 likes | 129 Views
Learn about the main subsystems and databases in HCSO, including data processing, dissemination, data entry and validation, metadata maintenance, and more.
E N D
Standard data entry & validation system in HCSO(ADEL) Erzsébet Kómár 3. May 2011. erzsebet.komar@ksh.hu IT Department
The main subsystems and concerning database in HCSO Dissemination (STATINFO) Data Processing (EAR) Dissemination Database M E T A D A T A Data entry and validation (ADEL,BLUMEN) Data Warehouse Metadata-maintenance (META) Production Database Data Collection (KSHXML) Data for Survey Control Registers Survey Control (GESA, LAKOS) Register- maintenance (business reg. address reg, etc.)
Data entry & validation – past, present In regional offices, in-house activity • 1982 – pdp/1140 computes, different validation programs • 1993 – PC – nothing changed • 1999 – reorganized the process – Oracle DB, tools unified user interface, same functions-same solutions, survey&term independent system, meta driven modules, central DB -> in-house developed application framework (ADEL) • 2000 – introduction 1 survey • 2001 – 28 surveys … contiguous development, new functions • 2010 – 100 surveys
Users and main functions • Statisticians in the centre (subject matter department) • design the questionnaire, select enterprises, define validation rules (in predefined format) • IT experts in the centre • provide IT background, define META information, design DB structure, grant/revoke roles, develop validation rules and data entry application • Statisticians in the regional offices • keep contact with enterprises, collect questionnaires, update response information in survey control system, data entry, validate (paper and e-questionnaire) and correct data with ADEL framework • Statisticians in the centre (subject matter department) • allowed to modify data on specific consideration (apply the validation rules) ->Three groups of users – standard roles select – view all information update – enter&modify data (statisticians) admin – modify META information (IT experts)
Features of application framework • Use META information • documentation – measures, nomenclatures • code validation – nomenclature code, name • meta driven function – access control: department/survey, user/survey • Standard menu structure with the functions • flow control, data entry and validation, online and offline validation, reporting, management information, documents • Standard user interface • font type & size, colours (input/display item) • navigation between tables/rows/items, using of mouse
Validation rules • Inside questionnaire a+b<=c, 01+02=03, 1T/05a+1T/05b=2T/01a • Between questionnaires same survey (previous periods), other surveys • Information in META system - error message, error level • Statisticians (in the centre) define the categories • Categories (levels) • 1: warning– information • 2: caution – correct or user can accept with explanation • 3: error – must be corrected or accepted by superuser with explanation • 4: critical – must correct immediately
Reusable building blocks Same functions – same solutions – reusable modules • Code validation nomenclature code, enterprise identity code • Standard first page contact information, open text note from the enterprise • Update response information in survey control system • Exception handling error level: 2, 3 -> explanations are stored in DB • Transform report format reports in text or Excel (format transform module) • Validation modules are generated predefined format (Excel) -> program code generator
Management information • Survey status • data entry or receiving e-questionnaires are allowed/permitted • Processes • audit trail, who & when started the process • Questionnaires • who & when entered/loaded, modified, deleted, controlled • Errors • levels, status (accepted or not), who & when accepted, explanation text
Development rules • Conventions, standards on object level • compulsory tables and columns in the DB • folder structure in the file system • strong name convention in the DB, in the file system
Control and quality • Workflow control in the development • controlled development phases, checkpoints, feedback • Workflow control in production • more hundred users – harmonized and controlled activity • audit trail, compulsory steps, alert messages • Product control • evaluation reports, explanations and messages from the enterprise
Advantages of standardisation ADEL - one common system for • different questionnaires • different years Solution • unified data structure, conventions • common program elements, unified menu-structure Result • unified user interface, unified workflow for statisticians – easy using • saving development capacity • unified quality and monitoring reports for all surveys from the common data tables
IT environment • Developed with Oracle tools • Development (on Windows XP) • Designer 10.1.2 • Developer Suite 10g (Form, Report-Writer, Sqlplus, PL/Sql) • Running: WEB technology • Client side: Windows XP, Sun Java 1.6 • Application side: Red Hat Enterprise Linux, Oracle Application Server 10g • Database side: HP UNIX 11.23, RDBMS 10.2