Dr. Patricia Georgieva, National Evaluation and Accreditation Agency, Bulgaria

1. 1st Task Force Meeting 13 and 14 July 2006 SarajevoStandards and Guidelines for Quality Assurance in the European Higher Education Area:Definition of Key Terms Dr. Patricia Georgieva, National Evaluation and Accreditation Agency, Bulgaria e-mail: pag@neaa.government.bg

2. Who gave ENQA mandate for the development of ESG? In the Berlin communique of 19 September 2003 the Ministers of the Bologna Process signatory statesinvited the European Network for Quality Assurance in Higher Education (ENQA) “through its members,in cooperation with the EUA, EURASHE, and ESIB”, to develop “an agreed set of standards, proceduresand guidelines on quality assurance” and to “explore ways of ensuring an adequate peer review system forquality assurance and/or accreditation agencies or bodies, and to report back through the Bologna Follow-Up Group to Ministers in 2005”.

3. The purpose of the ESG • To improve consistency of QA across EHEA by the use of agreed standards and guidelines • To enable HEIs and QA Agencies to use common reference points for QA • To strengthen the procedures for recognition of qualifications • To enhance the credibility of Agencies’ work

4. The structure of the ESG • The first set of 7 standards reflects the institutions’ own activities related to the design and application of quality processes and structures. • The standards and guidelines in Part 2 provide (though not in detail) ways about what should be examined or how quality assurance activities should be conducted. • Part 3 of the standards and guidelines concerns external quality assurance agencies, their establishment, organization and recognition.

5. The relationship between the three parts of the standards • External QA procedures should take into account the effectiveness of the internal QA processes described in Part 1 of the standards and guidelines. • The external quality assurance of agencies should take into account the presence and effectiveness of the external quality assurance processes described in Part 2 of the European Standards and Guidelines.

6. The Work of ENQA two working groups: 2004-2005 • The focus on generic quality standards, which could be shared with by all higher education institutions and national QA systems across Europe. • The wording of the standards is broad and open to different systems, approaches, cultures and traditions; • Standards and guidelines represent shared expectations about quality and good practice in relation to quality assurance that can be generally accepted as valid within higher education institutions across the whole of EHEA.

7. What is standard? Standards are defined as general reference statements describing policies and practices that are intended to ensure that pre-determined and explicit levels of quality are achieved in the design, delivery and evaluation of higher education courses and awards. The standard states what the outcome should be, rather than how it can be achieved.

8. Quality in higher education is a description of the effectiveness of everything that is done to ensure that diligent students can derive maximum benefit from the educational opportunities available to them, and also fulfil the requirements for the award for which they are working.

9. What is quality assurance? • a process of establishing stakeholder confidence that provision (input, process and outcomes) fulfils expectations or measures up to threshold minimum requirements • All quality activities inside and outside the organisation

10. What is internal QA? all the actions and activities undertaken by institutions of higher education to ensure that the quality of their programmes and awards meets their own specifications and those of any other bodies legitimately empowered to make specifications

11. What is external QA? a process by which an external person or team check that procedures are in place across an institution to assure quality, integrity or standards of provision and outcomes

12. ESG for HE Institutions • Part 1 of the standards focuses on the activities that are expected to be undertaken by an institution of higher education itself to ensure good quality of education for its students. • It includes 7 standards and each one describes what the institution should have or is expected to demonstrate in order to achieve the standard.

13. Standard 1.1: Policy, procedures, enhancement strategy The policy statement should include a description of the quality strategy, the organisation of the quality assurance processes and procedures, including the distribution of responsibilities, the ways in which the policy is implemented, monitored and revised. The place for students should be guaranteed as well. All HEIs should aspire to improve (to step forward for better quality) and enhance (to reach the point of transformation) the education they offer their students.

14. Standard 1.2: Formal mechanisms for the approval, periodic review and monitoring of their programmes and awards

15. What is approval? • Approval is an overarching term to cover various forms of academic recognition of a programme or institution • successful formal acceptance of a programme, by either or both an internal or external validating or accrediting panel(UK)

16. What is review? • The process of evaluating the provision, work process or output of an individual or collective.

17. What is monitoring? Monitoring has two meanings: • the specific process of keeping quality activities under review; • a generic term covering all forms of internal and external quality assurance and improvement processes including audit, assessment, accreditation and external examination.

18. What is award? The final result, or outcome of successful completion of a higher education programme, e.g., the first or second higher education degree (Bachelor, Master, etc.)

19. Standard 1.3: Students should be assessed using published criteria, regulations and procedures which are applied consistently.

20. Standard 1.3 Assessment of students: This standard requires from institutions to make explicit their expectations toward student progression and achievements within programmes of study on offer, to set clear examination regulations, to provide training opportunities for the new staff in relation to assessment methods and to ensure at all times that assessments are conducted in accordance with the stated procedures.

21. Assessment Criteria: Descriptions of what the learner will have to demonstrate in order that learning outcomes specific to a module have been achieved. The purpose of assessment criteria is to establish clear and unambiguous standards of achievement in respect of each learning outcome. Level descriptors are used as a guide during this process.

22. Standard 1.4: Qualified and competent teaching staff • If you insist that your teaching staff is competent and qualified for the job, you have to find ways to prove that.

23. Standard 1.5: Adequate and appropriate learning resources Standard: • Institutions should ensure that the resources available for the support of student learning are adequate andappropriate for each programme offered.

24. Standard 1.6: Information systems relevant for the effective management of study programmes • The quality-related information systems of individual institutions are expected to cover: student progression and success rates; employability of graduates; students’ satisfaction with their programmes and the effectiveness of their teachers, etc.

25. What is employability of graduates? Employability is the acquisition of attributes (knowledge, skills, and abilities) that make graduates more likely to be successful in their chosen occupations (whether paid employment or not).

26. Standard 1.7: Institutions should publish accurate, impartial and objective information about the programmes and awards they are offering

27. ESG for the external quality assurance of higher education • The second set of 8 standards reflect basic good practice across Europe in external quality assurance, be it in the form of institutional, subject or programme evaluations, or a combination of these.

28. Standard 2.1: External QA procedures should take into account the effectiveness of the internal QA processes described in Part 1. • It is important that the institutions’ own internal policies and procedures arecarefully evaluated in the course of external procedures, to determine the extent to which the standardsare being met.

29. What is effectiveness of quality assurance? • Effectiveness is the extent to which an activity fulfils its intended purpose or function. Fraser (1994) in the context of quality, Effectiveness is: a measure of the match between stated goals and their achievement. It is always possible to achieve ‘easy’, low-standard goals. In other words, quality in higher education cannot only be a question of achievements ‘outputs’ but must also involve judgements about the goals (part of ‘inputs’).

30. Standard 2.2: This standard requires that: • Development of external QA processes starts with determining aims and objectives • Purpose and procedure should be negotiated with stakeholders; • Finally agreed procedures should be published • A preliminary impact assessment should be undertaken to avoid overburden

31. How quality assurance activities should be conducted? • Standard 2.3: Any formal decisions should be based on explicit published criteria that are applied consistently • Standard 2.4: All External QA processes should be designed to ensure their fitness to achieve the aims and objectives set for them • Standard 2.5:Reports should be published and should be written in a style which is clear and readily accessible to itsintended readership. Any decisions, commendations or recommendations contained in reports should beeasy for a reader to find.

32. What should be monitored? Standard 2.6: Quality assurance processes which contain recommendations for action or which require a subsequentaction plan, should have a predetermined follow-up procedure which is implemented consistently. The objective is to ensure that areas identified for improvementare dealt with speedily and that further enhancement is encouraged.

33. What is Follow up? • Follow up is shorthand for procedures to ensure that outcomes of review processes have been, or are being, addressed. • Most review processes make recommendations for improvements or specify requirements for compliance. Follow-up procedures monitor whether these have been addressed. • Follow-up may be internal or external to an institution.

34. When quality assurance activities should be conducted? Standard 2.7: External quality assurance of institutions and/or programmes should be undertaken on a cyclical basis. The length of the cycle and the review procedures to be used should be clearly defined and published inadvance. Subsequent external reviews should take into account progress that hasbeen made since the previous event.

35. Standard 2.8: Quality assurance agencies should produce from time to time summary reports describing and analysingthe general findings of their reviews, evaluations, assessments etc.

36. ESG for external quality assurance agencies What is Agency? • Agency is, in the context of quality in higher education, shorthand for any organisation that undertakes any kind of monitoring, evaluation or review of the quality of higher education.

37. Standard 3.1: Use of external quality assurance procedures for higher education This standard requires from the agencies to integrate standards 2.1 to 2.8 into the processes of review, audit, accreditation, etc. applied towards the higher education institutions. It means that the way the Agency develops its aims and objectives, it reaches audit or accreditation decisions, prepares its reports, organizes its follow-up procedures, etc., should comply with ENQA requirements originating from Part 2 of the standards and guidelines.

38. Standard 3.2: Official status Agencies should be formally recognised by competent public authorities in the EHEAand should have an established legal basis. • The emphasis on the public recognition and legal bases of the agencies suggests that external quality assurance in the future could be performed by organisations which are not, by definition, part of the national quality assurance system of higher education.

39. Standard 3.3: ActivitiesAgencies should undertake external quality assurance activities (at institutional or programme level) on a regular basis. • The standard specifies the range of activities that may be included in the Agency portfolio and requires that these are taken on a regular basis, e.g., every five, or six years. • It also stresses the need for an agency to establish its external quality assurance into its core function

40. Standard 3.4: Resources • Agencies should have adequate and proportional resources, both human and financial, to enable them to organise and run their external quality assurance process(es) in an effective and efficient manner, with appropriate provision for the development of their processes and procedures.

41. This standard suggests that: 1) the agency financial resources allow it to carry out its mandate effectively in terms of its current portfolio of quality assurance processes and in the timeframe suggested by its predefined length of cycle; 2) the agency has sufficient resources to run each individual quality assurance process effectively, e.g., to gather, analyse, and assess documentation in a way that imparts credibility to any conclusions and recommendations in its reports; 3) the agency has appropriate provision for the development of its processes and procedures- that is, the capability to conduct internal quality assurance and use the results for its own enhancement and development; 4) the agency has adequate and proportionate human resources for a solid organization capable of managing and administrating quality assurance processes.

42. Standard 3.5: Mission statementAgencies should have clear and explicit goals and objectives for their work, contained in a publicly available statement. Standard 3.5 and the attached guidelines can be divided into three requirements, focusing on: • Clear and explicit goals and objectives • Division of labour with other stakeholders, existing context and activities • Systematic approach, clear policy and management plan

43. Standard 3.6: Independence An agency will need to demonstrate its independence through measures, such as: • Its operational independence from higher education institutions and governments is guaranteed in official documentation (e.g. instruments of governance or legislative acts). • The definition and operation of its procedures and methods, the nomination and appointment of external experts and the determination of the outcomes of its quality assurance processes are undertaken autonomously and independently from governments, higher education institutions, and organs of political influence. • While relevant stakeholders in higher education, particularly students/learners, are consulted in the course of quality assurance processes, the final outcomes of the quality assurance processes remain the responsibility of the agency.

44. Standard 3.7: The processes, criteria and procedures used by agencies should be pre-defined and publicly available The following elements should be covered: • Pre-defined and publicly available processes, criteria and procedures • A 4-steps procedure, including self-assessment, external assessment by a group of experts, site visits ending with a public report, and a follow-up procedure • Professional management • Consistency of conclusions and decisions • The existence of an appeals procedure

45. Standard 3.8: Agencies should have in place procedures for their own accountability Elements that are expected to be part of the accountability processes and procedures of the agencies: • Published policy for internal quality assurance; • Evaluation of fulfillment of mission and goals, e.g., through regular surveys • No-conflict-of –interest of external experts • Feedback mechanisms • Internal reflection mechanism- means to react to feedbacks and recommendations for improvement • External review

46. What is Accountability? Accountability is the requirement, when undertaking an activity, to expressly address the concerns, requirements or perspectives of others.