Developing a quality culture: The basic framework Mag. Oliver Vettori

1. Developing a quality culture: The basic framework Mag. Oliver Vettori March 2008

2. Introduction • ‘The quality of higher education has proven to be at the heart of a European Higher Education area’. • (Berlin Communiqué) Evaluierung & Qualitätsmanagement in der Lehre

3. Introduction • ‘The concept of quality is not new: it has always been part of the academic tradition. It is the outside world that now emphasises the need for attention to quality…It is the relationship between higher education and society which has changed’. • (Vroeijenstijn 1995) Evaluierung & Qualitätsmanagement in der Lehre

4. Introduction • Bologna process - European Higher Education Area: degree structure, student centred learning, student mobility • Accountability demands: evaluations, controlling, performance based funding schemes, employability rates • Institutional profiles and international competition: rankings, ratings, benchmarking • Changing role of the university management entrepreneurial universities, increased autonomy Rising importance of quality assurance Evaluierung & Qualitätsmanagement in der Lehre

5. Quality notions Graduates Students Government Value for money Stakeholder satisfaction … Quality Corporations, employers Fitness for purpose Transformation Teachers excellence Management Administrators Society at large Evaluierung & Qualitätsmanagement in der Lehre

6. Differing views on Quality Cultures control oriented quality as an objective that can be created and managed clear understanding of quality required top down implementation of quality standards & strategies quality improvement can be measured – with unintended consequences participative quality as an emerging result that changes in a dynamic way permanent negotiations of quality notions required participative development & implementation of quality st. quality development can be influenced – with uncertain results Evaluierung & Qualitätsmanagement in der Lehre

7. Setting the frame… Quality as a dynamic, multidimensional and perspective-boundconcept Culture as a complex, interaction-based, and continuously changing construct Quality Cultures as stakeholder-dependent, historically grown and learning-oriented social phenomena The way(s) quality and culture can be related to each other have important consequences for QM and QA strategies Evaluierung & Qualitätsmanagement in der Lehre

8. Principles of a quality culture Empowering the stakeholders to develop their own quality goals, initiatives and measures Guaranteeingtransparency and common standards without succumbing to a purely formal quality approach Showingtrust without disregarding the risks involved Strengthening reciprocal communication processes Balancing the delegation and acceptance of responsibility Evaluierung & Qualitätsmanagement in der Lehre

9. Fitness for purpose approach Graduate surveys, round tables, feedback box, awards, student evaluation of teaching questionnaires, program monitoring, focus groups, advisory board, accreditation, process analysis, student support program, discussion groups, staff development, premiums, fast feedback, midterm review, external evaluation, tutoring, COMMUNICATION, peer review, implementation of standards, rules and procedures, online evaluation, reporting, accompanying research projects, drop out study, steering committee, interviews, data ware house, meta-evaluation, counselling etc. Evaluierung & Qualitätsmanagement in der Lehre

10. QA roles: the formal structure Vice-Rector f. Academic Programs a. Student Affairs lays foundation Reports to Evaluation Board of the Senate Teaching Evaluation & QM Office concepts organisation operation conciliates Evaluierung & Qualitätsmanagement in der Lehre

11. measurement improvement Quality Culture dialogue QA roles in a quality culture perspective Teachers ……. Students Management Administrators Evaluierung & Qualitätsmanagement in der Lehre

12. QA roles: A few examples at WU Wien • Encouraging quality intiatives on departmental level • Rewarding initiatives for innovative and excellent teaching • Reinventing program directors and coordinators • Collaborating with staff development and controlling units as well as teachers and researchers throughout the university Evaluierung & Qualitätsmanagement in der Lehre

13. … and some final caveats Organisational beliefs and values are nothomogeneous, not even within a stakeholder group Quality cannotbe decreed Permanent control or standardisation are no guarantors for success and transparency External and internal requirements cannot be satisfied by one and the same strategy Evaluierung & Qualitätsmanagement in der Lehre

14. Part II The students‘ role Evaluierung & Qualitätsmanagement in der Lehre

15. Student Contributions take responsibility give feedback participate in surveys and selfevaluations The students‘ role tutors and mentors self-assessment and self-evaluation take part in decision-making and design processes Evaluierung & Qualitätsmanagement in der Lehre

16. student contributions - examples course evaluations, sounding boards, feedback boxes, fast feedback … introductory tutors, learning coaches, buddy networks, teaching assistants… members of self-assessment teams, members of external review teams, graduate surveys … curriculum development boards, advisory boards, teaching & learning forums etc. feedback tutoring evaluating designing & deciding Evaluierung & Qualitätsmanagement in der Lehre

17. Student self-assessment (1) What did I learn today? What were my contributions in class today? What do I need help with? What do I want to know more about? What would you do differently, if you took this class Again? Evaluierung & Qualitätsmanagement in der Lehre

18. Student self-assessment • self-evaluation forms • learning inventories • reflection logs and learning diaries • expectation exercise • discussions • peer assessment Evaluierung & Qualitätsmanagement in der Lehre

19. Part III The administration‘s role Evaluierung & Qualitätsmanagement in der Lehre

20. Administrative contributions establishing supportive structures informationand communication providing quality services The administration‘s role avoiding quality bureaucracy Engaging in self- Assessment and self-reflection supporting feedback loops Evaluierung & Qualitätsmanagement in der Lehre

21. feedback systems • develop suitable quantitative and qualitative quality indicators that hold relevance for all the actors involved • maintain a reasonable balance between the input (time, efforts) and outcome of evaluations • support the other actors to make sense of their findings • integrate evaluations into a broader system of quality assurance and development (follow ups) • keep the feedback processes as transparent as possible Evaluierung & Qualitätsmanagement in der Lehre

22. necessary information • student progression and success rates • employability of graduates • students’ satisfaction with their programmes • effectiveness of teachers • profile of the student population • learning resources and their costs • the institution’s own key performance indicators • ENQA Standards & Guidelines Evaluierung & Qualitätsmanagement in der Lehre

23. Criteria for self-evaluation (cost) effectiveness efficiency information quality celerity transparency service-orientation learning-orientation … Evaluierung & Qualitätsmanagement in der Lehre

24. Part IV The teachers‘ role Evaluierung & Qualitätsmanagement in der Lehre

25. Administrative contributions take the role of learners learning coaches creating suitable learning environments The teacher‘s role implementing a feedback culture engaging in self- Assessment and self-reflection peer teaching & peer reviews Evaluierung & Qualitätsmanagement in der Lehre

26. Goals of evaluations Feedback assessment improvement development control Quality of content and didactics Quality of teaching and learning processes Staff satisfaction/student satisfaction Evaluierung & Qualitätsmanagement in der Lehre

27. Course Evaluations Evaluierung & Qualitätsmanagement in der Lehre

28. Student questionnaires assessing the consistency of teaching goals, contents and methods regular student feedback (questionnaires) multifaceted item pool, flexibilised questionnaires Constant documentation/observance of results and developments objective method tools monitoring Evaluierung & Qualitätsmanagement in der Lehre

29. Additional Fast Feedback early diagnosis of potential problems, assessing standards of knowledge/understanding etc. flexible and individualised use: • dimensions/aspects • time • extent counselling, support objective method Support Evaluierung & Qualitätsmanagement in der Lehre

30. Fast Feedback: Beispiel 1 Course: + ? What I liked about the course.. What could be improved… My further comments Evaluierung & Qualitätsmanagement in der Lehre

31. Fast Feedback: Beispiel 2 Course Date Minute Paper Please answer the two questions below: What is the most important thing you What were the “muddy points”? learned today? Evaluierung & Qualitätsmanagement in der Lehre