The Impact of a Changing Business Architecture on Editing

1. The Impact of a ChangingBusiness Architecture on Editing UN/ECE Work Session on Statistical Data Editing Topic (v): Changing organizational cultures Claude Poirier Ljubljana, Slovenia, 9 – 11 May 2011

2. Outline • The changing culture • Impacts on editing processes • Current tool kit • Reducing the tool kit • Improving the general functionality • Consolidating existing functions • Impacts on other dimensions Statistics Canada • Statistique Canada

3. Changing culture • Statistics Canada’s priorities include • Relevance, Quality, Accessibility, Efficiency • Corporate Business Architecture principle • To maintain and promote reusable tools • E&I functions evolved through two systems • BANFF and CANCEIS Statistics Canada • Statistique Canada

4. Changing culture - Governance • Processes are being examined • Emphasis on global rather than local efficiency • Risk management strategy • Corporate solutions include generalized systems • Generalized systems’ check list for managers Statistics Canada • Statistique Canada

5. Changing culture – Recent actions • Generalized System Planning Committee • Focus on sampling, edit & imputation, estimation • Investigation of development needs and priorities • Major accomplishments • Documentation of current tools • Solutions resulting from research initiatives • Identification of programs’ needs • Analysis of opportunities from a collaboration network Statistics Canada • Statistique Canada

6. Impact – Current tool kit The tool kit: • BANFF – Recommended for business surveys • CANCEIS – Census system (suitable for few social surveys) • IMPUDON – SAS macros for donor imputation • AG2000 – Edit and imputation customized for agriculture • StEPS – USCB Standard Economic Processing System • Other – A collection of customized systems Statistics Canada • Statistique Canada

7. Impact – Reducing the tool kit • Too many systems offer similar functions • To reduce the number of software components, but with no disruption to any statistical programs • Recommendation: BANFF and CANCEIS • Users will develop pre-inter-post processors if needed • Exceptions to the one “set” fits all rule • Business cases must be well documented • System improvements should make less exemptions in the future Statistics Canada • Statistique Canada

8. Impact – Improving the functionality • The set of robust methods can be improved • Enhancements are guided by the granted exceptions • Enhancements are expected from any live system • IMPUDON and other SAS applications • Some applications survived through the years • Their requirements are being analyzed Statistics Canada • Statistique Canada

9. Impact – Improving the functionality • Priorities for BANFF • Outlier detection • Percentiles; Sigma-gap; Top-k report; • Error localization • To consider pre-imputed values as potential errors; • Donor imputation • In addition to L∞ norm, consider the L1 and L2 norms • Diagnostic reports • Reports to help the user in understanding the processes Statistics Canada • Statistique Canada

10. Impact – Improving the functionality • Future BANFF items • Selective editing • To prioritize editing – SELEKT (Sweden) will be considered • Connectivity • Interaction with a central database will be considered • Metadata • Metadata should be integral to the process • Editing of e-collection • Batch functions should be portable to e-collection Statistics Canada • Statistique Canada

11. Impact – Consolidating functions • BANFF and CANCEIS deserve their places in the suite of generalized systems • Several systems are based on SAS components • SAS is a standard for statistical data processing • Consolidating the two systems in SAS? • Strengths, weaknesses, opportunities and threats should be studied before updating the corporate 10‑year investment plan Statistics Canada • Statistique Canada

12. Impact on other dimensions • Research initiatives • Statistical Research and Innovation Division is responsible for managing research activities • Initiatives are related to editing & imputation, estimation, data analysis, confidentiality, time series • Corporate desire to improve data collection has positive impacts on editing research • - Non-response follow-up - Quality indicators • - Mode effect - Selective editing Statistics Canada • Statistique Canada

13. Impact on other dimensions • Efficiency with respect to human resources • The changing culture encourages creativity • Making good use of standardized systems • Understanding the data and the editing results • Learning from the data will improve abilities • Using common systems will: • - Improve employees’ learning curve • - Ensure portability of staff Statistics Canada • Statistique Canada

14. Collaboration network • Industrialization of editing • Culture is still changing, but at a higher level • The network should contribute to remove barriers between NSI’s • Impacts are yet to come Statistics Canada • Statistique Canada

15. Conclusion The changing culture hasseveral impacts on editing Learning from editing willalso change the culture Statistics Canada • Statistique Canada

16. For more information, Pour plus d’information,please contact: veuillez contacter : Claude.Poirier@statcan.gc.ca Statistics Canada • Statistique Canada