Use and Editing of Administrative Data in the Business Indicators Unit, Statistics New Zealand

1. Use and Editing of Administrative Data in the Business Indicators Unit, Statistics New Zealand Blair Cardno & Vera Costa Statistics New Zealand

2. Overview • Introduction • Summary of the Business Indicators Unit at Statistics New Zealand • Collections, uses and comparisons • Recent developments in editing • Future directions in editing • Conclusions

3. Summary of the Business Indicators Unit at Statistics New Zealand • Focus of the Business Indicators Unit (BIU) • Summary of the BIU administrative data collections • Customs Service trade data • Goods and Services Tax (GST) data • Local government building consents data • Electronic Transaction Data (Electronic Point of Sale data)

4. Collections, uses and comparisons • Customs Service Data • Used for Overseas Merchandise Trade statistics • Current collection and editing methods • Electronic files sent to Statistics New Zealand in agreed format • Files are loaded into a custom capture / edit program • Auto edits • Manual edits • Macro checks • Issues • Introduction of threshold editing

5. Collections, uses and comparisons cont.. • Goods and Services Tax (GST) data • Used for the Business Activity Indicator Series (BAI), supplementing surveys • Current collection, editing & imputation methods for BAI • Inland Revenue Dept extracts records once per month • The GST units matched by their IRD number to the Business Frame • Data imputation • Data editing • Issues • Uses of BAI data to supplement surveys • Used in the Manufacturing, Wholesale and Retail Trade Surveys to remove the need to survey small to medium sized businesses • Annual GST sales used as an indicator of size in some surveys

6. Collections, uses and comparisons cont.. • Comparisons of BIU administrative data • Providers • Customs and GST files contain a similar number of records per month (approx. 500,000). • Building consents data are provided by 74 providers (6,000 – 7,000 consents per month). • Administrative data design at source & processing for use • None of the administrative sources are specifically designed with statistical use in mind. • Customs data vs GST data vs Building Consents

7. Collections, uses and comparisons cont.. • Comparisons of BIU administrative data • Variable types & editing • GST data are numerical • Customs data and Building consents are a mix of numeric & categorical • Data editing approaches • “perfect data” vs “do what we can”

8. Recent Developments – Overseas Trade • Current situation • Editing system targets “perfect data” • Too much manual editing • Short term objectives • Methodology that: • Can be implemented easily • Has minimal impact on current editing procedures • Reduces the quality of key outputs by an amount that is acceptable • To reduce the editing effort by about 25%

9. Recent Developments – Overseas Trade • Proposal: entries with non-missing $ values below a specified threshold are never brought up for editor action • Analysis: impact on estimates at different country and commodity levels • Data used • Before and after editing • 2 months (Apr04, May04) • Exports and imports

10. Recent Developments – Overseas Trade • Impact of the amount of data manually edited

11. Recent Developments – Overseas Trade • No change in the current system needed • Preliminary results • Labour • Expected reduction – 2 people per year • Obtained reduction – 3 people per year • Amount of data manually edited • Before the threshold introduction – 12% • After the threshold introduction – 7%

12. Recent Developments – Business Activity Indicators • Methodology takes into account the nature of the data • All data to be edited is numeric • Data is available at the unit record level • Long time series for most units • No weights involved • Outliers detection • Automatic • Based on Statistical Process Control • Outliers editing • Manual / automatic • Based on selective editing • Project implementation

13. Future Directions - Overall editing strategy • Generalised Systems • Modular approach • Editing for numeric / categorical variables, etc • Overall strategy of Statistics NZ • Automation • Labour / cost • Consistency • Bias • Timeliness • More time for analysis • Selective / significance editing • Quality Assessment • Collections improvement

14. Future Directions - Challenges • What is “fitness for use”? • Different uses of the data

15. Conclusions • Collections at different stages of development • Emphasis on use of administrative data • Administrative data: • is crucial in producing statistical outputs • is continually investigated in Statistics NZ • Data available at different government agencies: • potential for the production of a new range of statistical information • Editing Strategy • Fundamental for the production of good quality data