DATA QUALITY AND META DATA PRESENTED AT THE TRAINING COURSE ON ‘INTERNATIONAL TRADE STATISTICS’

1. DATA QUALITY AND META DATA PRESENTED AT THE TRAINING COURSE ON ‘INTERNATIONAL TRADE STATISTICS’ BRUNEI DARUSSALAM 20-22 SEPTEMBER 2016

2. OUTLINE

3. IMTS 2010 RECOMMENDATIONS

4. 1. IMTS 2010 RECOMMENDATIONS

5. DATA QUALITY

6. WHY Data Quality is Important 2. DATA QUALITY • Credibility • To ensure reliable and accurate data for input into BOP and GDP • And also for policy making

7. Validation Errors 2. DATA QUALITY • Data is certainly incorrect • May cause processing Problems • Cannot be included in statistical analysis • Must be removed/corrected • Examples • text characters in value field • invalid commodity or country code

8. Credibility Errors 2. DATA QUALITY – cont. • Data is valid but possibly incorrect • Can be processed • Examples: • inflated value (currency) or quantity (*000) • valid but wrong commodity code

9. Invalid Validity Check Valid Auto-correction Validation Steps 2. DATA QUALITY – cont. Electronic Entry Credibility Check Server JP Credible Server SMK Interim data of External Trade Statistics

10. Quality Controls 2. DATA QUALITY – cont. 10

11. (a) Coverage Check 2. DATA QUALITY – cont. Electronic Statements from Free Zone Authority, Power Producer Company, Petroleum Company, MAS, Air Asia, etc) Hardcopy 11

12. 2. DATA QUALITY – cont.

13. EXAMPLES OF VALIDATION CHECK

14. BXXXXW COMPANY ABC JALAN ABCXYZ High Value Transactions COMPANY XYZ ADDRESS OVERSEAS AZ9999 SYARIKAT XXX TAN AH BAH

15. BXXXXW COMPANY ABC JALAN ABCXYZ XXXX XXXX YYYYYYYYYYYYY NEGERI AA Country of Origin COMPANY XXXMMMM JALAN ABCXYZ XXXX XXXX YYYYYYYYYYYYY AZ9999 COMPANY XXXMMMM TAN AH BAH

16. BXXXXW COMPANYABC JALAN ABCXYZ XXXX XXXX YYYYYYYYYYYYY NEGERI AA Unit of Quantity COMPANY XXXMMMM JALAN ABCXYZ XXXX XXXX YYYYYYYYYYYYY NEGERI CINA AZ9999 COMPANY XXXMMMM TAN AH BAH

17. BXXXXW COMPANY ABC JALAN ABCXYZ XXXX Country of destination COMPA JALAN ABCXYZ XXXX XXXX YYYYYYYYYYYYY NEGERI AA AZ9999 COMPANY XX TAN AH BAH

18. BXXXXW Channel COMPANY ABC JALAN ABCXYZ XXXX XXXX YYYYYYYYYYYYY NEGERI AA COMPANY XXXMMMM JALAN ABCXYZ XXXX AZ9999 COMPANY XXXM TAN AH BAH

19. Code BXXXXW COMPANY ABC JALAN ABCXYZ XXXX XXXY COMPANY XXX AZ9999 COMPANY XXXMMMM TAN AH BAH

20. Quality Check (METSS) BXXXXW COMPANYABC JALAN ABCXYZ XXXXXXXXXXXX XXXX YYYYYYYYYYYYYXXX COMPANY XXXMMMMMMM AUP CHECK JALAN ABCXYZ XXXXXXXXXXX XXXX YYYYYYYYYYYYYXXXXX AZ9999 COMPANY XXXMMMMMMMMM TAN AH BAH

21. Quality Check (METSS) cont. BXXXXW COMPANY ABC JALAN ABCXYZ XXXX XXXX YYYYYYYYYYYYY COMPANY X JALAN ABCXYZ XXXXMMMMM XXXX YYYYYYYYYYYY AZ9999 Error in Tariff Code Change HS  AHTN COMPANY XXXMMMMMMMMM TAN AH BAH ASEAN country

22. (c) Back-end Check 2. DATA QUALITY – cont.

23. QUALITY REPORT - MALAYSIA

24. 3. QUALITY REPORT • Malaysia External Trade Statistics Quality Report was sent to ASEAN Secretariat in August 2013 – in line with the recommendation of IMTS 2010. • Benchmark to UK’s Quality Report

25. Quality Dimensions 3. QUALITY REPORT 1. Timeliness 2. Accuracy 3. Accessibility & Clarity 4. Comparability 5. Coherence

26. Quality Indicators 3. QUALITY REPORT – cont. • Timeliness - Advanced Release Calendar (ARC)

27. Quality Indicators – cont. 3. QUALITY REPORT – cont. 2) Accuracy 2.1 Accuracy Characteristics 2.2 Controls using primary information/ sources 2.3 Controls using secondary information/source 2.4 Confidentiality 2.5 Actions to improve accuracy

28. Quality Indicators – cont. 3. QUALITY REPORT – cont. 2) Accuracy – cont. 2.1 Accuracy Characteristics i) Statistical Value Exports – free on board (f.o.b.) Imports – cost, freight and insurance(c.i.f.) ii) Level of thresholds From January 2013, single transactions with a value ‘below RM1,000’ for Peninsular Malaysia are excluded except for major commodities and selected goods.

29. Quality Indicators – cont. 3. QUALITY REPORT – cont. 2) Accuracy – cont. 2.1 Accuracy Characteristics – cont. iii) Exclusions Transit and goods for repair iv) Revisions - The monthly data are provisional when 1st released and subject to revision in the following month. - For annual, it will be released 9 months after the reference year.

30. Quality Indicators – cont. 3. QUALITY REPORT – cont. 2) Accuracy – cont. 2.2 Controls using primary information/sources i) General information ii) Information on controls and errors

31. Quality Indicators – cont. 3. QUALITY REPORT – cont. 2) Accuracy – cont. 2.2 Controls using primary information/sources i) General information a) Data providers b) Number of transactions

32. Quality Indicators – cont. 3. QUALITY REPORT – cont. 2) Accuracy – cont. 2.2 Controls using primary information/sources i) General information – cont. c) Electronic declarations

33. Quality Indicators – cont. 3. QUALITY REPORT – cont. 2) Accuracy – cont. 2.2 Controls using primary information/sources i) General information – cont. d) Information on controls and errors - Validity checks - Credibility checks - Methodology for credibility checks

34. Quality Indicators – cont. 3. QUALITY REPORT – cont. 2) Accuracy – cont. 2.2 Controls using primary information/sources i) General information – cont. d) Information on controls and errors - Validity checks - Credibility checks - Methodology for credibility checks

35. Quality Indicators – cont. 3. QUALITY REPORT – cont. 2) Accuracy – cont. 2.3 Controls using secondary information/source i) Statement on the imports/exports of oil and gas ii) Statement on the imports of tobacco iii) Report of total values and total number of transactions declared in Free Commercial Zone

36. Quality Indicators – cont. 3. QUALITY REPORT – cont. 2) Accuracy – cont. 2.4 Confidentiality - Statistics Act 1965 (Revised – 1989) 2.5 Actions to improve accuracy

37. Quality Indicators – cont. 3. QUALITY REPORT – cont. 3) Accessibility and clarity 3.1 Accessibility - Publications - METS Online 3.2 Clarity - Documentation available to users in all publications

38. Quality Indicators – cont. 3. QUALITY REPORT – cont. 4) Comparability and Issues 4.1 Comparability over space - Methodological discrepancies in reporting - Triangular trade - Exchange rate - Reporting time lags - Classification of goods - Confidentiality 4.2 Comparability over time

39. 3. QUALITY REPORT – cont. Quality Indicators – cont. 5) Coherence - Inclusion of ‘ Specific Movements’ data into METS - Mirror statistics analysis

40. 4. METADATA The term metadata defines all information used to describe other data. It is recommended that countries at least cover the following categories of metadata for their international merchandise trade statistics: (i) description of all underlying concepts and definitions, including the trade system used, and deviations from international standards, if any; (ii) legal framework, institutional arrangements and description of data sources; (iii) description of data collection and data processing procedures; (iv) description of estimation methods; (v) data dissemination policy including release and revision schedules; (vi) description of the all data fields/ variables (reference period, trade flow, commodity classification used, valuation, currency, quantity (net weight), weight unit used, supplementary quantity, supplementary quantity unit used, partner country (origin, last known destination, consignment) (this applies to all forms of data dissemination); (vii) explanations and footnotes concerning the data as required, i.e. informing about revisions, break in series and application of confidentiality; (viii) quality reporting

41. 5. DISSEMINATION The dissemination of data and metadata, therefore, should be treated as an integral part of the national programme of trade statistics and carried out with great care and attention to the needs of users, while at the same time ensuring adequate confidentiality of data providers. It is recommended that passive confidentiality be applied as much as possible unless the use of active confidentiality is already the established, desired and accepted practice. Reference period. It is recommended that countries make their data available on a calendar period basis, according to the Gregorian calendar and consistent with the present IMTS 2010 recommendations. Countries are encouraged to issue their first releases of data as follows: (a) Monthly totals of exports and imports within 45 days after the end of the month, at least by major trading partners and basic commodity breakdown; (b) Quarterly data within 60 days after the end of the quarter; (c) Annual data within 90 days after the end of the year Revision policy. Countries are encouraged to develop a revision policy that is well designed, carefully managed, transparent and well coordinated with other areas of statistics, and hence allows users to cope with revisions in a systematic manner. It is recommended that countries choose the dissemination format that best suits their user needs. It is further recommended that international merchandise trade statistics be accessed through the electronic dissemination databases maintained by the responsible agency.