1 / 18

Comparative review of SDC and SA standards Jean Marc MUSEUX – Eurostat

This review provides an overview of statistical disclosure control (SDC) and seasonal adjustment (SA) standards, including the process leading to standardization, learnings, and conclusions. It discusses the importance of SDC and SA, their impact on EU data utility, and the need for harmonization. The review also highlights the barriers and difficulties in implementing SA and SDC standards and proposes strategies for fostering implementation.

pfournier
Download Presentation

Comparative review of SDC and SA standards Jean Marc MUSEUX – Eurostat

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Comparative review of SDC and SA standardsJean Marc MUSEUX – Eurostat Workshop on Standardisation Brussels, 14-15 October 2010

  2. Seasonal Adjustment and Statistical Disclosure Control • Why SA and SDC ? • State of the art of standards • Process leading to standard • Learnings and conclusions

  3. Statistical Disclosure Control X-sectional tabular output and micro data files Seasonal Adjustment Time-series and Infra-annual statistics Why SA and SDC ? “Historical” domains where Eurostat central methodological unit developed some expertise in the 90’s Important steps for Eurostat business process 5. Process - 6. Analyse Specific expertise independent of statistical domains

  4. Statistical Disclosure Control Committee on Statistical Confidentiality since 1997 Working group on Statistical Confidentiality since 2009 Seasonal Adjustment Informal Working Group on Seasonal Adjustment 1999-2002 SA Steering Group since 2007 Why SA and SDC ? Dedicated working group for ESS coordination

  5. Statistical Disclosure Control Coherence required because secondary confidentiality in tables Disclosure risk increases if uncoordinated release at EU and MS level Strong impact on EU data utility Sensitivity of breach of confidentiality in the ESS Seasonal Adjustment Seasonally adjusted data: reference key indicators for analysis and forecasting exercises Reliability and comparability EU aggregate derived from MS series – need for coherence Non linear process with propagation of error Why SA and SDC Striking need for harmonisation

  6. ESS Guidelines on Seasonal Adjustment Endorsed by CMFB and SPC in 2008 SA process decomposed in substeps (pre-treatment, signal extraction, revision, release, metadata, …) For each elementary steps, the guidelines lists three alternatives A, the best approach to be aimed at; B, acceptable and viable if A proved to be too costly C, practice to be avoided. Provide a open framework to design SA process (guidance), Improve SA process (clear preference) benchmark several processes SA standards – state of the art (1/3)

  7. ESS Guidelines Implementation Seasonal Adjustment Steering Group (SASG) in charge of overseeing the implementation of the guidelines SASG is high level group bringing together Eurostat, ECB and SA experts in MS and CB Main barriers for implementation 1) Lack of institutional recognition 2) Organisational issues 3) Cost of option A (human resources) 4) Methodological issues 5) Knowledge, skills 6) IT infrastructure SA standards – state of the art (2/3)

  8. ESS Guidelines Implementation Three main strands for fostering implementation decided by SASG Information in sectoral WG, Scientific conferences Cooperative (re)development of a software tool (Demetra+) in line with the guidelines (A and B options can be implemented) Training, workshops (with experts) for spreading knowledge and exchange of experience Further difficulties No global review and impact assessment Need to continuously refine, going beyond guidelines (crisis, …) SA standards – state of the art (3/3)

  9. resources Knowledge Generation (R&D/innovation) Knowledge Formalisation Competence Building Good-practiceGeneration (ESSnets) Operational Governance output input Methodology Implementation Production Quality & Tools Strategy products Framework for analysing process leading to standard

  10. Tracking back SA standardisation process Tracking back Seasonal Adjustment works 2: Eurostat development of Demetra1999-2005 1:Eurostat initial studies on SA: 1995-1999 input Knowledge Generation (R&D/innovation) Knowledge Formalisation Competence & Capacity Building 3 3 1 2 2 Good-practiceGeneration (ESSnets) Methodology 3 3 Operational gouvernance 2 3 RESOURCES 1 2 PRODUCTS 4 2 Production Strategy Implementation 4 1 Quality & Tools 3: SASG and guidelines2006-2009 output 4: Demetra + cooperative development2009-2010

  11. Early expertise development First Demetra tool Did not achieve harmonisation Distanciation from expertise SASG breakthrough Technical expertise Governance Methodology Demetra+ After methodology Cooperative and open source SA standards process – key elements

  12. Tabular data Standard tool for protecting tables Tau-Argus developed by Statistics Netherlands Financial support from Commission since 2001 Difficulty to integrate in standard production processes Handbook on SDC (last edition 2010) – glossary and review of options No standard (SDC depends on national perception and rules) EU table protection at the border of feasibility Lack of standardisation hampers release of EU figures (suboptimal solution) SDC standards – state of the art (1/4)

  13. Tabular data First step towards standardisation Confidentiality Charter (SBS, Prodcom, Animal production) Objective rules for protection of cells Practical rules at domain level ensuring consistency between Eurostat and MS processing Flexibility in primary confidentiality documented (flags) Methodology for SDC of EU aggregate SDC standards – state of the art (2/4)

  14. Micro data protection Baseline methodology and corresponding software (Mu-Argus – Statistics Netherlands) – suitable for one off application Domain in constant development (computer science, Web, ..) Anonymisation of EU micro data Output harmonisation through input harmonisation Same global recoding for all MS datasets Micro aggregation for all records Little flexibility Least common denominator effect Low information content (almost public use files) SDC standards – state of the art (3/4)

  15. Micro data protection – way forward ISTAT model Q2008 – Community Innovation Survey PSD2010 – Harmonisation of SDC Bounded flexibility Core common disclosure scenario and risk assesment methodology Flexibility among a common set of methodologies for protecting records adapting to country specificities transparent parametrisation good balance between global methods (recoding, top coding) and local methods (perturbation) Data utility target (threshold) using common measures and constraint on comparability (benchmarking using key research statistics) SDC standards – state of the art (4/4)

  16. Tracking back SDC developments Tracking back Seasonal Adjustment works 2: CENEX project 2004-2006 1:CASC FP5 research project 1997-2004 input Knowledge Generation (R&D/innovation) Knowledge Formalisation(LDF) Competence & Capacity Building 2 3 3 1 2 2 Good-practiceGeneration (ESSnets) Methodology Operational gouvernance 3 4 4 2 3 RESOURCES 1 2 PRODUCTS 4 Production Strategy (CVD) Implementation 3 4 2 1 Quality & Tools (IT) 3: ESSnet SDC II2007-2009 output 4 Next steps

  17. Importance of research – early steps towards tool development – centric development – difficulty of integration Importance of ESSnet for sharing good practices Lack of technical governance To identify best practices To set up priorities Next steps Technical governance (TF) Open source development More flexibility – stronger metadata SDC standards – key elements of the process

  18. Review of two cases studies with difference outcomes Main differences political context technical governance type of guidelines Main communalities Need for harmonisation Need for expertise – research Process steps relatively independent of the statistical domain Need for flexibility Need for appropriate – sustainable software tools Living standard – need for maintenance Conclusions * Not yet active * Not yet active

More Related