I. Institutional background 1. AEQES 2. CTI

1. Combining evaluation and accreditation culturesCaty Duykaerts (AEQES), Teresa Sanchez, Jean-Claude Arditti, Pierre Fleischmann, Bernard Remaud (CTI)A collaboration for the joint evaluation and accreditation of civil and bio engineering programmes in the French Community of Belgium 1

2. I. Institutional background 1. AEQES 2. CTI II. Origin ans scope of the collaboration III. Phases of the collaboration 1. Preparatory phase 2. Design of the collaboration 3. Execution phase IV. Development and main outomes of the preparatory phase 1. Assessing the feasability of the collaboration 2. Collaboration agreement between AEQES and CTI V. Conclusions and questions for the audience 2

3. Institutional background • Origin and scope of the collaboration • Phases of the collaboration • Development and main outomes of the preparatory phase • Conclusions and questions for the audience AEQES brief history & legal framework From 1998 to 2002 First evaluations of University programmes conducted by the CRef AEQES 1 : Act of November 2002 All components of higher education Evaluation of study programmes Self-assessment reports, peer reviews and confidential reports AEQES 2 : Act of February 2008 A more independent agency, evaluation of programmes based on a 10-year planning, self-assessment reports, external reviews, publication of results and follow-up procedures Agency review every 5 years (ESG compliance) ENQA full member since september 2011 3

4. AEQES methodology • Institutional background • Origin and scope of the collaboration • Phases of the collaboration • Development and main outomes of the preparatory phase • Conclusions and questions for the audience Key features • formative QA evaluation process (no accreditation) • programme-based quality assurance • scope : 1st and 2nd cycle degrees (Bachelor and Master) • all similar programmes are evaluated simultaneously, system-wide analysis • Self evaluation reports based on a reference list of indicators • No formal effects in terms of institution fundings or authorisation • no rankings, no scores 4

5. Institutional background • Origin and scope of the collaboration • Phases of the collaboration • Development and main outomes of the preparatory phase • Conclusions and questions for the audience Presentation of CTI 5

6. Institutional background • Origin and scope of the collaboration • Phases of the collaboration • Development and main outomes of the preparatory phase • Conclusions and questions for the audience Presentation of CTI Full member since 2005 Founding Member 2005. Mutual Recognition Agreements (OAQ, NVAO) Founding member 2000 (one of the 7 institutions that can deliver the EUR-ACE label) • Since 2010 (European Registry of quality agencies) 6

7. Institutional background • Origin and scope of the collaboration • Phases of the collaboration • Development and main outomes of the preparatory phase • Conclusions and questions for the audience Origin In the FCB: all HEIs must be evaluated / engineering programmes make no exception October 2009: official demand of the four universities concerned for a joint mission AEQES/CTI Twofold objective behind: • Evaluation of the programmes according to AEQES ten-year plan (compliance with AEQES decree requirements) • Accreditation of the programmes according CTI’s criteria (providing access to the « admission » by the French governement and to EUR-ACE label) 7

8. Institutional background Origin and scope of the collaboration Phases of the collaboration Development and main outomes of the preparatory phase Conclusions and questions for the audience Scope Four universities concerned • Catholic Universitiy of Louvain (UCL) • Free University of Brussels (ULB) • University of Liège (Ulg) (including Faculty of Agricultural Sciences of Gembloux – FUSAGx) • University of Mons (UMons) Four (4) engineering programmes (BA + MA) in the field of agronomic sciences and biological engineering Fifteen (15 BA + MA) in the field of engineering sciences 8

9. Institutional background Origin and scope of the collaboration Phases of the collaboration Development and main outomes of the preparatory phase Conclusions and questions for the audience Scope 9

10. Institutional background Origin and scope of the collaboration Phases of the collaboration Development and main outomes of the preparatory phase Conclusions and questions for the audience Phases of the collaboration • Preparatory phase / December 2010 Two purposes To assess the feasibility of the collaboration To agree on a set of common principles Signature of a formal collaboration agreement/January 2011 Design of the collaboration / from January to December 2011 Execution phase / from January 2012 to September 2013 10

11. Institutional background Origin and scope of the collaboration Phases of the collaboration Development and main outomes of the preparatory phase Conclusions and questions for the audience Assessing the feasibility Three main issues 1) Compatibility of two national QA frameworks 2) Compatibility of evaluation / accreditation methods and procedures 3) Applicability of CTI’s accreditation framework to engineering programmes in the FCB 11

12. Institutional background Origin and scope of the collaboration Phases of the collaboration Development and main outomes of the preparatory phase Conclusions and questions for the audience Assessing the feasibility 1) Compatibility of the two national frameworks The two national legal frameworks posed no major barriers to the collaboration However, one difficulty detected: difference between the two periodic calendars 12

13. Institutional background Origin and scope of the collaboration Phases of the collaboration Development and main outomes of the preparatory phase Conclusions and questions for the audience Assessing the feasibility 2) Compatibility of evaluation/ accreditation methods and procedures put in place by the two agencies Methodology: • Use of the comparison tools and procedures developed by ECA in TEAM project (as a basis for reflection) • Organisation of several work meetings to perform a comparative analysis of standards and procedures • Inclusion of observers of the two agencies in a CTI and an AEQES mission (visit) 13

14. Institutional background Origin and scope of the collaboration Phases of the collaboration Development and main outomes of the preparatory phase Conclusions and questions for the audience Assessing the feasibility 2) Compatibility of evaluation/ accreditation methods Conclusions: Global compatibility with regard to principles and conception. However detection of a number of implementation differences regarding: - the dynamics and objectives of the HEI’s self-evaluation phase - the balance between quantitative and qualitative criteria - the role of the panel members during the site-visits - the attitude of the HEI’s with regard to the accreditation / evaluation process 14

15. Institutional background Origin and scope of the collaboration Phases of the collaboration Development and main outomes of the preparatory phase Conclusions and questions for the audience Assessing the feasibility 3) Applicability of CTI’s accredidation criteria to engineering programmes in the FCB. A number of criteria identified as « mandatory ». • A strong and broad basis in fundamental sciences • A guarantee of efficiency and short term adaptation to a professional activity • Business culture and economic, social, human, environmental ethcis awareness • Communication skills and international awareness 15

16. Institutional background Origin and scope of the collaboration Phases of the collaboration Development and main outomes of the preparatory phase Conclusions and questions for the audience The agreement The main outcome of the preparatory phase has been the establishment of a formal agreement signed by the presidents of the two agencies and covering: - The objectives and the scope of the collaboration - The general organisation of the project - The organisation of the site visits - The composition of the panel - The main outcomes of the mission (productions of several reports) 16

17. Institutional background Origin and scope of the collaboration Phases of the collaboration Development and main outomes of the preparatory phase Conclusions and questions for the audience Conclusion up to now Benefits of the collaboration from now on: • Agreement on a certain set of basic principles • Setting of a basic work framework Illustrative and interesting exercice for both parties. Opportunity to: • confront procedures and methods • bring to the surface the underlying principles behind two apparently similar procedures This step by step collaboration has built a space of trust and confidence 17

18. Institutional background Origin and scope of the collaboration Phases of the collaboration Development and main outomes of the preparatory phase Conclusions and questions for the audience Conclusion up to now Evaluation vs accreditation (1) - Accountability and quality enhancement in a single exercice ? - Accreditation interviews are more « punchy » - Audit report ? Final « product » for the evaluation Work document to the accreditation committee - Oral restitution at the end of the audit ? No problem for the evaluation Ambiguous for the accreditation 18

19. Institutional background Origin and scope of the collaboration Phases of the collaboration Development and main outomes of the preparatory phase Conclusions and questions for the audience Conclusion up to now Evaluation vs accreditation (2) Differences in the processes - The common process ends with the publication of the report with the recommendations - For the accreditation, the report is presented and discussed by the (CTI) committee, which may : Put different weights to the recommendations Add new ones (?) Decide accreditation (Y/N, duration) for each program Decide EUR-ACE labels and/or « titre d'ingénieur » 19

20. Institutional background Origin and scope of the collaboration Phases of the collaboration Development and main outomes of the preparatory phase Conclusions and questions for the audience Conclusion up to now Evaluation vs accreditation (3) Differences in the processes - The follow-up of recommendations may have not the same objectives 20

21. Institutional background Origin and scope of the collaboration Phases of the collaboration Development and main outomes of the preparatory phase Conclusions and questions for the audience Conclusion up to now First audit vs routine audit This joint audit was the first for the belgian engineering faculties. In France, HEI's have about 15 years experience of periodic accreditation Then HEI's are less prepared and suspicious The agency (AEQES) has committed very substantial ressources : prepatory meetings, number of days on site, size of the teams. 21

22. Institutional background Origin and scope of the collaboration Phases of the collaboration Development and main outomes of the preparatory phase Conclusions and questions for the audience Conclusion up to now Cultural differences The French-speaking Belgians are soft speaking and they dislike disputes The French like controversy and are more direct Some deans felt uncomfortable during interviews 22

23. Institutional background Origin and scope of the collaboration Phases of the collaboration Development and main outomes of the preparatory phase Conclusions and questions for the audience Conclusion up to now CTI is familiar to international accreditation and to distinguish the core of the standards from the elements linked to the local context But Decisions for HEI's a few kilometers apart on both sides of the border must be consistent. 23

24. Institutional background Origin and scope of the collaboration Phases of the collaboration Development and main outomes of the preparatory phase Conclusions and questions for the audience Open questions • What are the fundamental differences between an evaluation and an accreditation approach? • To what extent can an agency conciliate the sometimes confronted objectives of the accountability and quality enhancement in a single exercice? • When accreditation becomes a routine : what changes in the procedures , should we shift more from program to institution assessment ? 24

25. . Thank you www.aeqes.be/ www.cti-commission.fr 25