European Court of Auditors

1. How Does the ECA Assess Member States’ Internal Control Systems?Alan Findlay(CEAD Directorate B - Audit and Audit Supervision) Workshop on Audit/Evaluation of Public Internal Financial Control Systems (PIFC)Ankara, 8-9 July 2008 European Court of Auditors

2. Agenda • The Court’s “Assurance Model” • The DAS audit process • Understanding the environment • Planning audit procedures • Performing the audit & evaluating results • Forming audit opinion • Questions / Discussion European Court of Auditors

3. The Assurance Model (1) European Court of Auditors

4. The Assurance Model (2) Assurance Model as a tool to manage audit risk: • Preliminary assessment of inherent and control risk (combined risk assessment) to determine: • Reasonable expectations concerning level of confidence (controls assurance) that can be drawn from supervisory and control systems • Resulting extent and scope of substantive testing to be carried out, aiming to reach required overall level of confidence (95%) • Revision and / or final confirmation of preliminary assessment on the basis of audit results achieved European Court of Auditors

5. Agenda • The Court’s “Assurance Model” • The DAS audit process • Understanding the environment • Planning audit procedures • Performing the audit & evaluating results • Forming audit opinion • Questions / Discussion European Court of Auditors

6. DAS audit process PERFORM AUDIT P LAN AUDIT AND EVALUATE PROCEDURES RESULTS Audit Audit Environment Evidence UNDERSTAND F ORM AUDIT THE AUDIT International CONCLUSIONS AND Audit ENVIRONMENT Legal Basis AUDIT OPINION Standards THE DAS • International Art. 248 of the Treaty Standards of • Reliability Auditing (ISA) • Legality & published by IFAC regularity • INTOSAI auditing (Art. 129(4) of Financial standards Regulation) • COSO Framework • published with financial statements European Court of Auditors

7. Agenda • The Court’s “Assurance Model” • The DAS audit process • Understanding the environment • Planning audit procedures • Performing the audit & evaluating results • Forming audit opinion • Questions / Discussion European Court of Auditors

8. Audit environment • Collect /update and analyse information about the audit environment: • Legal framework • Design and functioning of the supervisory and control systems • Nature of payments • Audit standards to be applied, etc. Gain understanding of the audit environment European Court of Auditors

9. Agenda • The Court’s “Assurance Model” • The DAS audit process • Understanding the environment • Planning audit procedures • Performing the audit & evaluating results • Forming audit opinion • Questions / Discussion European Court of Auditors

10. Determine materiality • Preliminary assessment of: • Inherent risk • Quality of supervisory and control systems Audit procedures Plan audit procedures Plan audit procedures allowing to obtain reasonable assurance European Court of Auditors

11. Audit procedures (2) Determine materiality • Errors are material if, individually or aggregated with other errors, they would reasonably affect decisions of addressees of opinion • General practice: between 0,5% and 2% • Possibility to set different level(s) taking into account addressees requirements (“tolerable or residual risk”) • Issues material by context/nature to be disclosed • Materiality rules not (always) directly applicable to systems weaknesses (often only risk) • Qualitative aspects (seriousness of deficiency or frequency) • Quantitative aspects (possible financial impact) European Court of Auditors

12. Audit procedures (3) • Preliminary assessment of inherent risk • Economic and technical environment • Type and operating of activities, programmes, etc. • Characteristics of beneficiaries • Complexity of rules • Management policies and practices • Other information to be taken into account: • Previous work and knowledge / experience • Annual activity reports and declarations of Commission’s Directors-general and their Synthesis • Other obligatory reports of different services of Commission and Member States • Relevant other information (National declarations, national certificates, ...) • Reports of other auditors European Court of Auditors

13. Audit procedures (4) Preliminary assessment of quality of supervisory and control systems • Central question: Are supervisory and control systems adequate to manage the risk insofar as compliance with relevant laws, regulations, contracts, etc. is concerned? • Strong link between inherent risk and control risk • Different implementation levels to be considered European Court of Auditors

14. Agenda • The Court’s “Assurance Model” • The DAS audit process • Understanding the environment • Planning audit procedures • Performing the audit & evaluating results • Forming audit opinion • Questions / Discussion European Court of Auditors

15. Performing the audit • Perform audit procedures: • Examination of supervisory and control systems • Substantive testing of payments • Analysis of management representation (National Declarations Summaries, Annual Activity Reports, etc.) • Possibly work on other sources of audit evidence (e.g. other auditors) Perform audit procedures and evaluate results • Evaluate audit results: • (Re-)examine / (Re-)assess preliminary expectations • Modify initial planning on the basis of preliminary audit results, ifneeded • Perform supplementary audit work, if necessary European Court of Auditors

16. Audit objectives for supervisory and control systems Assessment of their capacity to • Supply management with information needed to ensure that legal, regulatory and contractual provisions have been complied with and, where applicable, corrective action is taken • Give reasonable assurance concerning compliance of revenue and expenditure with laws, regulations, contracts, etc. European Court of Auditors

17. Main steps of evaluatingsupervisory and control Systems • Auditors should first identify the management departments, national/regional/local authorities or other third parties having a role of supervision or control • Preparation of a programme of tests to check systems’ compliance with three key control objectives • Design according to regulation or other provisions • Transposition by parties concerned • Continuous and effective functioning European Court of Auditors

18. The role of substantive testing in evaluating systems • Substantive testing samples may also be used for tests of control to limit number of on-the-spot checks • Examination of payments affected by errors • Identification of systems which should have prevented or identified and corrected them • Analysis of control deficiencies at their origin • Overall analysis of the results • Assessment of level of assurance obtained European Court of Auditors

19. Agenda • The Court’s “Assurance Model” • The DAS audit process • Understanding the environment • Planning audit procedures • Performing the audit & evaluating results • Forming audit opinion • Questions / Discussion European Court of Auditors

20. Form audit conclusions at the level of specific assessments for different areas Forming audit conclusions & opinions Form audit conclusions and audit opinion Form audit opinion European Court of Auditors

21. Forming audit conclusions & opinions • Straightforward forming of audit conclusions requires that • Actual audit results confirm reasonable expectations formed on the basis of preliminary combined risk assessment • Findings for different sources are compatible with each other European Court of Auditors

22. Form audit opinion Examination of work of other auditors Analysis of annual activity reports and declarations Substantive testing Evaluationof supervisory and control systems • Professional judgement and materiality • Qualitative evaluation of results on work on systems • Quantitative evaluation of results of substantive testing • Analysis of coherence of audit results Audit conclusions Audit conclusions Audit conclusions Audit opinion – the DAS European Court of Auditors

23. Agenda • The Court’s “Assurance Model” • The DAS audit process • Understanding the environment • Planning audit procedures • Performing the audit & evaluating results • Forming audit opinion • Questions / Discussion European Court of Auditors