UNITED NATIONS STATISTICS DIVISION Trade Statistics Branch Distributive Trade Statistics Section

1. Enhancing Data Quality of Distributive Trade StatisticsWorkshop for African countries on the Implementation of International Recommendations for Distributive Trade Statistics27-30 May 2008, Addis Ababa, Ethiopia UNITED NATIONS STATISTICS DIVISION Trade Statistics Branch Distributive Trade Statistics Section

2. Outline of the presentation • Quality measurement of Distributive Trade Statistics (DTS) • Quality indicators versus direct quality measures • Metadata on DTS • Recommendations

3. Quality measurement ofDTS • Goal of quality measurement • To provide the user with sufficient information to judge whether or not the data are of adequate quality for their intended use • “Fitness for use” of the data • The users must be able to: • Verify that the conceptual framework and definitions that would satisfy their particular data needs are the same as, or sufficiently close to those employed in collecting and processing the data • Asses the degree to which the accuracy of the data is consistent with their intended use or interpretation • Quality management - all the measures that NSO takes to assure quality of statistical information

4. Data quality assessment frameworks • QAFs – integrate various dimensions (aspects) of quality, their definitions and quality measurement • Overall aim of QAFs • Standardize and systematize statistical quality measurement and reporting across countries • Allow an assessment of national practices to be made against internationally accepted statistical approaches for quality measurement • Use of QAFs • Guide countries ’ efforts for strengthening their statistical systems by providing a self-assessment tool and for identifying areas of improvement • Technical assistance purposes • Reviews of particular statistical domains performed by international organization • Assessment by other groups of data users

6. Dimensions of quality (1) • Prerequisites of quality • All institutional and organizational conditions that have an impact on the quality of DTS data • Elements – legal basis; adequacy of data sharing and coordination; assurance of confidentiality; adequacy of human, financial, and technical resources; quality awareness • Relevance • Degree to which DTS data meet the real needs of users • Measuring relevance requires identification of user groups and their needs • Credibility • Confidence that users place in the data based on the image of the statistical agency that produces the data • Trust in objectivity of the data • Data are perceived to be produced professionally in accordance with appropriate statistical standards • Policies and practices are transparent

7. Dimensions of quality (2) • Accuracy • Degree to which the data correctly estimate or describe the characteristics they are designed to measure • Defined in terms of errors in statistical estimates • Systematic errors • Random errors • Timeliness • Delay between the end of the reference period to which the data pertain and the date on which the data are released • Closely related to the existence of a publication schedule • Involved in a trade-off against accuracy • Accessibility • Ease with which data can be obtained from the statistical office • Suitability of the form or the media of dissemination through which the information can be accessed

8. Dimensions of quality (3) • Methodological soundness • Application of international standards, guidelines and good practices in production of DTS • Elements - adequacy of the definitions and concepts, target population of units, variables and terminology underlying the data; information describing the limitations of the data • Closely related to the interpretability of data • Interpretability reflects the ease with which the user may understand and properly use/analyze the data • Coherence • Degree to which the data are logically connected and mutually consistent • Coherence within datasets • Coherence across datasets • Coherence over time • Coherence across countries

9. Quality indicators versus direct quality measures • Quality measures • Items that measure directly a particular aspect of quality - time lag from the reference date to the release date • Most of them are difficult or costly to calculate in practice • Quality indicators • Summarize quantitative information to provide evidence about the quality or standard of data • Do not measure quality directly but provide enough information for the assessment of quality - response rate is a proxy quality indicator for measurement of non-response bias • Quality measures and quality indicators can either supplement or act as substitutes for the desired quality measurement

10. Quality Indicators • Criteria for defining quality indicators • Cover part or all of the dimensions of quality • Methodology for their compilation is well established • Indicators are easy to interpret • Types of quality indicators • Key indicators – coefficient of variations (accuracy), time lag (timeliness) • Supportive indicators – average size of revisions (accuracy) • Indicators for further analysis – user satisfaction survey (relevance)

11. Key quality indicators for DTS

12. Content of statistical data • Microdata - data on the characteristics of units of the population • Macrodata - derived from the microdata by grouping or aggregation • Metadata - “data about data”, describes the microdata, macrodata or other metadata

13. Statistical metadata • Fundamental purposes of metadata • Describe or document statistical data • Facilitate sharing, querying, and understanding of statistical data over the lifetime of the data • Help users understand, interpret and analyze the data • Help the producers of statistics to enhance the production and the dissemination of the data • A bi-directional relationship between metadata and quality • Metadata describe the quality of statistics • Metadata are a quality component • Provide a mechanism for comparing national practices in the compilation of DTS

14. Metadata on DTS • Levels of metadata • Structural metadata – integral part of DTS data tables • Reference metadata - provide details on the content and quality of data, may accompany the tables or may be presented separately • Components of DTS metadata • Data coverage, periodicity, and timeliness • Access by the public • Integrity of disseminated data • Data quality • Summary methodology • Dissemination formats

15. Recommendations (1) • Quality dimensions are overlapping and interrelated and form a complex relationship. NSOs can decide to: • Implement directly one of the existing QAFs • Develop national QAFs that fit best their countries practices and circumstances • Not all quality dimensions should be addressed for all data • Countries are encouraged to select those quality measures/indicators that together provide an assessment of the overall strengths, limitations and appropriate uses of a given dataset • Quality review of DTS should be undertaken every 4 to 5 years or more frequently if significant methodological changes or changes in the data sources occur

16. Recommendations (2) • Countries are encouraged to: • Accord a high priority to development of metadata • Consider their dissemination an integral part of dissemination of DTS • Adopt a coherent system and a structured approach to metadata across all areas of economic statistics, focusing on improving their quantity and coverage • Identify user needs of metadata and arrange users into groups so a layered approach to metadata presentation can be applied • Issue regularly, quality reports as part of their metadata

17. Thank You