Draft Methodology for impact analysis of ESS.VIP Projects

1. Draft Methodology for impact analysis of ESS.VIP Projects Pieter Everaers, Director Eurostat Directorate A, Cooperation in the ESS, International cooperation, Resources

2. A methodology for impact assessment is needed • ESSneton Standardisation :establish the business case of European standardisation initiatives • November 2012: new ESS.VIP projects have to be evaluated on the implementation costs and future benefits =>task force • May 2014: ESS Vision 2020 agreed by ESSC. Projects will be selected on criteria including cost/benefit

3. Two initiatives assessed the possible approaches • Sponsorship on Standardisation (end 2012- September 2013) proposed: • Common assessment approach • Link to the life cycle of a new initiative • Structured qualitative approach • Assessment by all stakeholders in the ESS • Scoring mechanism • Basis for the European business case

4. Two initiatives assessed the possible approaches • Task Force "Impact analysis of ESS.VIP projects" • (April-December 2013) proposed: • Draftmethodology and pilot project • Quantitative and qualitative assessments • Information collected at NSI level but impact for the ESS • Statistical and financial experts + senior management • In the quantitative phase: “benefits” = potential cost savings • SWOT based on proposal from ESSnet + flexible items

5. Reconcile the 2 methodologies • ESSC asked RDG and ITDG-DIME for a single methodology • Proposal • Combination of qualitative and quantitative analysis • 2 phases of the methodology linked to the life cycle of the initiative

6. Phase 1Business case including SWOT and Vision general criteria Phase 2 including detailed quantitative analysis

7. Phase 1 • Business case includes criteria from ESS 2020 Vision • SWOT analysis based on 3 fixed items and 3 flexible items • Rating of the SWOT items • Leads to decision to launch the project or to go to phase 2

9. Phase 1: Vision general criteria • A. Does the proposal have a positive business case at both ESS and national levels, in the sense that benefits outweigh costs? • B. Does the proposal contribute to the needs and drivers identified in the ESS Vision 2020 such as efficiency, quality and promoting public trust? • C. Can the proposal be clearly positioned according to our ambitions for collaboration set out in the ESS Vision 2020? • D. Does the scope in that sense look manageable? • E. Is the proposal consistent with the full set of other proposals? • F. Does the proposal take into account generic European principles like proportionality and subsidiarity? • G. Is the proposal in line with the European Statistics Code of Practice? • H. Does the proposal observe existing legislation, both at national and EU level? • I. Is the proposal in line with the ESS reference architectural decision and design principles? • J. Does the proposal sustain a simultaneous production of European and national statistics?

10. Phase 1: List of SWOT items • 1. STRENGTHS • a. Saving of resources • b. Improved process and systems quality • c. Enhanced flexibility • d. Increased harmonisation • e. Increased level of detail • f. Better availability of metadata • 2. WEAKNESSES • a. Costs (of development, transition, support and maintenance) • b. Lack of agility/flexibility of the new system • c. Heterogeneity of the starting point between countries and partners • d. Complexity • e. Impact of changed process (risk of errors, discontinuity of the data) • f. Lack of appropriate skills in the NSIs

11. Phase 1: List of SWOT items • 3. OPPORTUNITIES • a. Reduced burden on respondents • b. Enhance standardisation • c. Better communication with users and stakeholders • d. Opportunity to change to cope with resource constraints • e. Shared technical developments • f. Possibility to provide tailored feedback to data providers • 4. THREATS • a. Lack of consistency with national policies (including legal) • b. Lack of support from stakeholders/financers • c. Lack of suitable skill profiles • d. Threat to data security • e. Competition from other providers • f. Dependency from external actors/providers

12. Phase 1: SWOT rating

13. Phase 2 • Quantitative analysis • Focus only on business processes affected by change • Estimation of current costs, development costs and future costs of production • Cost categories: Staff , IT solutions, Overhead, Other costs • In case potential savings cannot be quantified: build on concrete experiences from countries which have modernized their production system, develop scenarios • + Adaptation of the SWOT analysis

15. To be kept in mindLessons from the pilot on quantitative approach • Analysis of current costs is possible for most countries • Data not always available for different steps of the GSBPM • Difficult for some countries to estimate their costs • =>group countries • Future costs: impact analysis of the modernization process at Member State level not systematic

16. Way forward • Proposaldiscussedtoday and in ITDG-DIME steering group 18 June • Feed the work of the high level Task Force working on the ESS Vision 2020 Implementation strategy (for DGINS) • Test on projectsimplementing Vision 2020 (2014-2015) • Evaluate and adapt the methodology