Paper by Tim Hawkes, presented by Emma Bentley

1. Improving efficiency by introducing macro editing in Statistics New Zealand business performance surveys Paper by Tim Hawkes, presented by Emma Bentley May 2011

2. Background • Suite of business performance (BP) surveys, many started around 2005 • Quality review recommendations: • Reduce manual micro edits, cut processing costs • More emphasis needed on macro editing • ‘Big picture’ perspective needed for analysts • New approach to editing developed for 2010 survey round

3. Previous editing approach • Micro editing – time consuming (2 FTE!) • Validity edits • Consistency edits • Statistical edits • Very few automatic edits, lots of manual intervention • Macro editing – limited • Pressure of publication deadlines • Not standard across the suite of surveys

4. Recommended new approach • BP analysts reviewed macro practices in-house and editing strategies internationally • Decided to improve: • Automatic micro editing • Consistency edits • Validity edits • Macro editing • Create more time for this • Wider range of strategies available to BP team

5. Automatic micro editing • Sum of components not equal to given totals • Routing and validity edits

6. Macro editing • Calculating estimates during the data collection process • Produce initial estimates when 50-60% of target response rate achieved • Early detection of influential observations for validation • Problems with processing or estimation system can be identified and resolved early in production process • Issue if insufficient responses for estimation purposes • Analysts gain better understanding of estimation process

7. Macro editing • Top-down editing • E.g. drill down to industry and stratum level estimates • Identify unusual components, drill down more • May provide evidence for estimate, or identify error • Use of sampling errors to identify suspicious estimates • Compare with expectations and previous results • Unusual responses that have effect on sample errors, may also effect estimates

8. Macro editing • Top contributor method • Ranked lists of top contributors e.g. for level estimates or movements • Compare lists to previous years • Graphical analysis • Not widely used by BP surveys • Processing checks • Monitoring template developed containing these checks

9. Implementation • Interactive training provided by Statistical Methods division • Analysts asked to identify potential questions suitable for automatic edits • Up to the lead analyst to implement the new approach and determine their macro editing techniques • BP team able to take more ownership of editing strategy now that the surveys are well established and topic knowledge developed

10. Review: Quality indicators Quality indicators for business performance surveys No decline in quality detected

11. Review: Qualitative feedback • Generally positive • Reduction in amount of manual micro editing – popular with analysts! • Timing for incorporating changes a challenge • Set up of automatic editing, will be reusable in future • Saved time from micro editing taken up by implementation of other methodology changes

12. Review: Qualitative feedback • More time allocated to macro editing • Better macro strategies helped analysts better understand the processes and data, this in turn helps subsequent analysis stage • Conservative approach for first cycle of new editing strategy, room for more efficiencies • Improved ability to calculate quality indicators would help assess efficiencies in future • Reviews are useful!