Evaluating QA Agencies INQAAHE BIENNIAL CONFERENCE, ABU DHABI 2009

1. Evaluating QA Agencies INQAAHE BIENNIAL CONFERENCE, ABU DHABI 2009 Pre-conference workshop 30 March 2009 1.30 to 4.30pm Dorte Kristoffersen Deputy Executive Director HKCAAVQ APQN Vice-President dkristoffersen@hkcaavq.edu.hk

2. Welcome Introduction of participants Overview of the workshop Introduction to the workshop on Evaluating QA Agencies

3. After the workshop the participants will be able to: Formulate possible purposes and values of an external evaluation of a external QA Agency (EQAA) Make decisions on the framework for an external evaluation Make decisions on the organisation of the external evaluation Understand the options for having the external evaluation recognised Workshop outcomes

4. Accountability locally and internationally ‘Taking your own medicine’ Effectiveness of performance Improvement Recognition Other? Purpose and value of an external evaluation

5. Frameworks for evaluation of EQAAs • INQAAHE Guidelines for Good Practice • The European Standards and Guidelines • The Chiba Principles • OECD/Unesco Guidelines for Cross-border higher education • Network membership criteria • European Excellence model • Other?

6. Purpose and content of GGP The main purpose of the Guidelines of Good Practice in Quality assurance (GGP) is to increase professional standards among INQAAHE member agencies Section I. The EQAA: Accountability, Transparency, and Resources 1. The Governance of the EQAA 2. Resources 3. Quality Assurance of the EQAA 4. Reporting Public Information

7. Section II. Institutions of Higher Education and the EQAA: Relationship, Standards, and Internal Reviews 5. The Relationship Between the EQAA and Higher Education Institutions 6. The EQAA's Requirements for Institutional/Program Performance 7. The EQAA’s Requirements Institutional Self-Evaluation and Reporting to the EQAA Content of GGP con’t

8. Section III. EQAA Review of Institutions: Evaluation, Decision, and Appeals 8. The EQAA's Evaluation of the Institution and/or Program 9. Decisions 10. Appeals Content of GGP con’t

9. Purpose and content of ESG The main purpose of the European Standards and Guidelines is to provide a source of assistance and guidance to EQAAs and HEI in developing their QA processes. The ESG are divided into three parts: For internal QA within HEI For external QA of HE For external EQAAs

10. ESG for EQAAs 1. Use of external quality assurance procedures for higher education 2. Official Status 3. Activities 4. Resources 5. Mission Statement 6. Independence 7. External quality assurance criteria and processes used by the agencies 8. Accountability procedures

11. Discussion session #1 You constitute the Board of the Accreditation Agency of Olympia (AAS). Is it the right decision to initiate an evaluation? How would you formulate the Terms of Reference? Would the Guidelines of Good Practice or the ESG be a valuable framework for the process – please explain why and why not.

12. Independence, ie clear division of labour between EQAA and panel Self evaluation report Peer Review and site visit Public report Evaluation approach

13. Administrative support, ie recruitment of panel members, organisation of travel, site visit program The agency under review >< independent secretariat Clear division of labour between the agency and the panel Considerations for appropriate administrative arrangements

14. Size Profile Stakeholder representation Nationalities Language Considerations forpanel composition

15. Preparation of self evaluation report Existing documentation, ie handbooks, policies, plans, regular surveys among client and stakeholders, and staff consultations New evidence, ie analysis through the self evaluation process, tailor-made surveys, call for Considerations forthe self evaluationprocess

16. Duration Balance between discussion/preparation, study of evidence and interviews Interview groups , eg agency board and staff, government, reviewers and other main stakeholders, ie educational institutions and students Considerations forsite visit programme

17. Discussion session #1 You constitute the Board of the Accreditation Agency of Olympia (AAS). Is it the right decision to initiate an evaluation? How would you formulate the Terms of Reference? Would the Guidelines of Good Practice or the ESG be a valuable framework for the process – please explain why and why not.

18. Target group Public >< confidential Balance between description, analysis and conclusion Balance between commendations, identification of areas for improvement and recommendations to address areas for improvement Report requirements

19. Discussion session #2 Taking the starting point in your own EQAA or in the situation in your home country if you are from a HEI, what issues would be relevant to take into consideration when organising the following stages of an external review: Organisational support Self evaluation process, including involvement of all relevant stakeholder Panel formation Site visit Report writing and follow-up

20. INQAAHE offers a service to its members to record a member agency that has been found to adhere comprehensively to the GGP INQAAHE can organise the external review process against a fee www.inqaahe.org Public recognition for external review

21. The European Quality Assurance Register for Higher Education (EQAR) is a partnership of European organisations in higher education EQAR is an information tool offering clear and reliable information on quality assurance provision by listing agencies who have proven compliance with the ESG through an external review (self evaluation, site visit by external experts (a total of 4 including at least one academic staff member, a student, one international member) The European Register Committee’s (ERC) 11 members are identified by the EQAR Board in their capacity as independent experts in HE and QA Application to be assessed by ERC Approval or non approval. Before announcement of non-approval the application can make additional representation Approval will lead to inclusion on the Register www.eqar.eu Public recognition for external review

22. Thank you