ENQA procedures for external review – the Bulgarian experience

1. ENQA procedures for external review – the Bulgarian experience Todor.Shopov@yahoo.com Vilnius,30 May 2009, 11.15-11.45 Hrs.

2. Objectives • The Agency (3 – 16) • The External Review (17 – 30) • The External Review Report (31 – 35) • The EQAR Application (36) • The Current State of Affairs (37 – 40)

3. The Agency (est. 1996)

4. Official Status a statutory body for evaluation, accreditation and monitoring of the quality in higher education institutions and scientific organizations, aiming at the enhancement of their teaching and research as well as of their development as scientific and cultural organizations

5. Activities • reviews the ability of institutions to provide good quality of education and scientific research • performs project evaluation, institutional and programme accreditation, post-accreditation monitoring of quality of higher education

6. Mission to encourage higher education institutions in assuring and enhancing the quality of education they offer by sustaining high academic standards and good education traditions in Bulgaria

7. Structure Decision-making bodies • Accreditation Council • Standing Committees on areas of higher education Administration Expert teams (ad hoc)

8. Standing Committees • formed by the Accreditation Council which appoints their members through selection by applications • comprised of 3 to 7 members, one of whom is Chairman of the Committee • the Chairman of the Accreditation Council appoints the members of the Committees for a three-year term

9. Standing Committees (in alphabetical order) • AGRARIAN SCIENCES AND VETERINARY MEDICINE • ECONOMIC SCIENCES AND MANAGEMENT • EDUCATIONAL SCIENCES AND MUSICAL AND DANCE ART • HEALTHCARE AND SPORTS • HUMANITIES AND ARTS • NATURAL SCIENCES, MATHEMATICS AND COMPUTING • SOCIAL SCIENCE, LAW AND NATIONAL SECURITY STUDIES • TECHNICHAL AND MILITARY SCIENCES

10. Standing Committee on POST-ACCREDITATION MONITORING AND CONTROL supervise HEI’s implementation of the recommendations received in the evaluation and accreditation process

11. Internal Quality Assurance of NEAA • IQA system • Corrector-NEAA

12. IQA system Four subsystems: • legal base • standards, policies, procedures and documents • management subsystem • material resources and financing

13. Corrector-NEAA Mechanisms for collecting feedback and recommendations on improving the activity: • Internal relations for feedback from administration, Standing Committees and Accreditation Council • External relations from individuals (academic experts, students, professionals) and higher-education institutions

14. External Quality Management (Control, Assurance, Improvement) quality terms: Quality Improvement can be distinguished from Quality Control in that Quality Improvement is the purposeful change of a process to improve the reliability of achieving an outcome. Quality Control is the ongoing effort to maintain the integrity of a process to maintain the reliability of achieving an outcome. Quality Assurance is the planned or systematic action necessary to provide enough confidence that a product or service will satisfy the given requirements for quality.

15. The national system of quality management in higher education PRINCIPLES • quality is not in the hands of TPTB ("the powers that be") • the "chorus of stakeholders" rules

16. The national system of quality management in higher education

17. The External Review • the application: 7 November 2007 • the language: "it's not an English exam“ • the patterns: to design the preparation process effectively

18. The Patterns • The patterns (A and B) help us plan and execute the external evaluation procedure. • That provides the Agency with positive experiences.

19. Pattern A, identified in the preparation for the external review • Pattern A is a complicated scheme. It is known as an ill-formed system. • We could not use it effectively.

21. Pattern B, identified in the preparation for the external review • We needed a well-organised, lean structure. • So, in June 2007, we developed Pattern B, based on classic model of quality management.

22. Pattern B, identified in the preparation for the external review INPUT → PREPARATION PROCESS → OUTPUT → SITE VISIT → EFFORTS TO IMPLEMENT RECOMMENDATIONS

23. INPUT the state of affairs in June 2007

24. PREPARATION PROCESS • translation of “Guidelines ”; “Marinov” task force appointed, issues identified: leadership, transparency of processes, accountability of participants, timelines, etc. • other issues (we are working on them even now): Participation in NEAA’s expert teams (ESG 2.4, 3.7); Reporting (ESG 2.5); Independence of recruiting experts (ESG 3.6); Composition of decision-making bodies (ESG 3.6, 3.8)

25. OUTPUT self-evaluation report (May 2008)

26. The site visit of the Review Panel • On June 17-18, 2008 an external expert group under the aegis of ENQA, coordinated by the Spanish Quality Assurance Agency ANECA, visited the NEAA to perform an external review of its activities. The on-site visit is part of the procedures for ENQA membership established in the ESG. • Within the two-day visit the expert group had working meetings with representatives of the NEAA leadership and its staff as well as with representatives of the State Administration, Local Communities, Trade-Unions, Rector’s Council, Higher Education Institutions and Students.

27. The TOR The TOR were approved by the NEAA (14 Feb 2008) and ENQA (3 March 2008).

28. describe the purpose and the structure of the external review, i. e. what has to be achieved, who will take part in it, how it will be achieved, when it will be achieved. The TOR are enclosed in the evaluation report. They contain 8 sections explaining the review process, the review panel (6 members), the self-evaluation report, the site visit, the evaluation report, the final decision by the board of ENQA, the publication of the evaluation report, and the budget of the review.

29. Official correspondence with ANECA

30. The self-evaluation report The task force, appointed by the Chairman of NEAA (June 2007), met regularly in the period June 2007 – June 2008.

31. THE EXTERNAL REVIEW REPORT It was produced, as planned, in July 2008. Its conclusions are presented in Chapter 5, entitled "NEAA's compliance with ESG". The first section, "Summary of strengths and areas of improvement" describes the positive and negative aspects of the activities of NEAA (strengths and weaknesses).

32. The second section of Chapter 5, "Final statement", includes the following positive evaluation "In the light of the documentation and oral evidence provided during the site visit, the Review Panel is satisfied that in the performance of its functions NEAA is in substantial compliance with the ENQA Membership Criteria and with the Standards and Guidelines for Quality Assurance in the EHEA. The Panel therefore recommends to the Board of ENQA that NEAA should be awarded Full Membership for a period of five years.”

33. July 2008 comments of NEAA on the factual accuracy of the external review report

35. ISSUES Problems solved at present: • Participation in NEAA’s expert teams (ESG 2.4, 3.7) • Reporting (ESG 2.5) • Independence of recruiting experts (ESG 3.6) • Composition of decision-making bodies (ESG 3.6, 3.8)

36. THE EQAR APPLICATION The application was submitted in February 2009. Two letters with additional explanations were sent to the EQAR Chair in March and May 2009.

37. THE CURRENT STATE OF AFFAIRS NEAA has a new goal: to make the process of evaluation and accreditation leaner, more user-friendly and more effective. For example, the two procedures of institutional and programme accreditation may be merged into one appraisal procedure at institutional level.

38. THE CURRENT STATE OF AFFAIRS NEAA has formed a working group to examine all issues related to its work and to propose fundamental changes in the accreditation of higher education institutions in line with current developments in quality assurance in education.

39. THE CURRENT STATE OF AFFAIRS In September 2009, the term of office of all Standing Committees will come to an end. New members will be elected and in this process students and non-academic professionals will be included as members of the committees. This decision was taken by the Accreditation Council in March 2009.

40. THE CURRENT STATE OF AFFAIRS In 2010, the NEAA will submit an interim report on the progress in implementing the changes recommended by ENQA.

41. THANK YOU!