HIGH OUTPUT QUALITY MONITORING VIA THE ECB QUALITY REPORTS ON STATISTICS

2. HIGH OUTPUT QUALITY MONITORING VIA THE ECB QUALITY REPORTS ON STATISTICS • Introduction. Main elements of the Statistics Quality Framework: towards high-quality output • Quality monitoring and reporting on financial accounts • Quantitative quality indicators • Output enhancement from the ECB quality monitoring reports: the experience of Banco de España • Conclusions

3. 1. MAIN ELEMENTS OF THE STATISTICAL QUALITY FRAMEWORK: TOWARDS HIGH-QUALITY OUTPUT • The ECB statistical system rests on three pillars: • the regulations setting obligations on economic agents • the guidelines given to NCBs detailing the datasets to be transmitted and other statistical requirements, and • the quality framework • The ESCB Public Commitment on European Statistics set the principles that govern the collection, compilation and dissemination of statistics by the ECB. • The ECB Statistics Quality Framework combines these principles and others of good governance with the quality assurance procedure developed by the ECB. • Quality reports are at the core of the system as they review main quality elements and draw some conclusions/recommendations in order to enhance statistics.

4. 1. MAIN ELEMENTS OF THE STATISTICAL QUALITY FRAMEWORK: TOWARDS HIGH-QUALITY OUTPUT • Quality is defined in relation to how well statistics are fit for purpose, and, therefore, meet users´ and other stakeholders´ expectations. This is a multidimensional concept.

5. 1. MAIN ELEMENTS OF THE STATISTICAL QUALITY FRAMEWORK: TOWARDS HIGH-QUALITY OUTPUT

6. 2. QUALITY MONITORING AND REPORTING ON FINANCIAL ACCOUNTS – QUALITY REPORTS • The main two procedures implemented by the SQF to report on quality to the ECB senior management are: • The audit reports prepared by the Internal Auditors Committee (IAC) • Quality reports prepared by the Statistics Committee (STC) • Quality reports prepared by the STC review main quality elements with the help of some usually pre-defined indicators and draw some conclusions and/or recommendations in order to enhance the quality of statistics • Several ECB statistical guidelines require the preparation of quality reports, mainly focused on the compilation of euro aggregates: • Balance of Payments and International Investment Position Statistics (biannual), • Government Finance Statistics (internal), • Monetary and Financial Statistics, • Centralised Securities Database (internal) • Financial Accounts. • All these reports are based on a common framework, although they are tailored to each specific statistical domain and so their content is not fully homogenous. • The preparation of these reports by the ECB with the assistance of the expert groups can be seen as the final step of the quality monitoring process, which also includes other internal quality reports.

7. 2. QUALITY MONITORING AND REPORTING ON FINANCIAL ACCOUNTS • ECB compiles euro area financial accounts on the basis of: • national financial accounts transmitted under Guideline ECB/2013/24, and • datasets already available in the ECB data warehouse from other ECB guidelines. The most important are: banking statistics, securities statistics, government finance, and external statistics. • ECB Guideline requires the transmission of: • stocks, transactions, other flows and counterpart information for selected items • information on major events and revisions • the preparation of an annual quality report including at least: coverage, compliance with relevant definitions and revisions • The quality monitoring process carried out by the ECB with the assistance of the expert group on financial accounts also includes several internal quality reports.

8. 2. QUALITY MONITORING AND REPORTING ON FINANCIAL ACCOUNTS

9. 2. QUALITY MONITORING AND REPORTING ON FINANCIAL ACCOUNTS EURO AREA QUARTERLY FINANCIAL ACCOUNTS REPORT • It is inspired by the basic principles of the SQF • Public (since 2013) • Coverage: • Institutional and legal framework • Quality assurance procedures and user involvement • Coverage • Revision analysis • Consistency and comparability • Timeliness of national contributions • Data accessibility • Recent developments • NCB-specific assessment, recommendations and non-compliance cases (only internal version) • Comprises a set of quantitative indicators at EA level

10. 2.1 QUANTITATIVE QUALITY INDICATORS • Revision analysis • Errors or revisions are defined as the difference between the current values of a variable (as transactions) and the values when they were released for the first time. The quality report presents four indicators to assess accuracy and reliability. • Positive revisions (R) as % of positive revisions in relation to total observations • Directional reliability (Q) as % of first release of data with the same sign as the last release • MACE [Mean Absolute Comparable Errors] which is the mean of absolute errors on transactions in relation to stocks, and • RMSRE [Root Mean Square Relative Errors] which is a relative measure of errors in relation to the variability series • Consistency analysis • Statistical discrepancies arise in the compilation process due to the integration of different data sources. The quality report assesses discrepancies in financial instruments (horizontal inconsistencies), and between the capital and the financial accounts (vertical inconsistencies) by means of the RMSE indicator. • RMSE [Root Mean Square Errors]

11. 2.1 QUANTITATIVE QUALITY INDICATORS Euro area quality reliability indicators for households

12. 3. OUTPUT ENHANCEMENT FROM THE ECB QUALITY REPORTS: THE EXPERIENCE OF THE BANCO DE ESPAÑA • The BdE contribution is fully compliant with the ECB data requirements • The BdE takes note of the conclusions and recommendations to enhance, as far as possible, the Spanish financial accounts and therefore the national contributions to the ECB • In recent years, changes have been introduced in several areas following quality monitoring reports: • to reduce inconsistencies with securities issues statistics due to the treatment of own securities used as collateral at the ECB • to reduce inconsistencies with money and banking statistics due to the treatment of factoring operations related to general government transactions • to harmonise the treatment of repos transactions across euro area countries • other discrepancies between data sources are regularly monitored • Discrepancies between net lending/borrowing and net financial transactions of non-financial corporations and households are monitored by the BdE and the INE

13. 3. OUTPUT ENHANCEMENT FROM THE ECB QUALITY REPORTS: THE EXPERIENCE OF THE BANCO DE ESPAÑA Spain. Discrepancies between net lending/borrowing and net financial transactions

14. 4. CONCLUSIONS • Monitoring and reporting on quality is an essential element of the Statistics Quality Framework. • Several ECB Guidelines require the preparation of annual quality reports, which to some extent are the final step of the quality monitoring process carried out by the ECB with the assistance of the expert groups. • By means of ECB quality reports the Statistics Committee can draw the attention of the highest decision-making bodies of the ECB to relevant quality problems on euro area statistics, and related national contributions, so that the ECB and national central banks can resolve underlying problems. • The quality report on financial accounts is an example of this strategy. This and other internal quality monitoring reports cover the compliance by countries with the ECB regulations and other quality aspects, in particular revisions and consistency with other statistics, which are examined on the basis of a set of quantitative indicators. • Finally, the paper provides some information about the changes introduced in the Spanish financial accounts following the conclusions and recommendations of the ECB quality reports.