Giving All Students a Voice – Introducing Online Evaluation of Teaching and Learning within QUT

1. Giving All Students a Voice – Introducing Online Evaluation ofTeaching and Learning within QUT Presented at Association for Tertiary Education Management Queensland Branch Conference “Taking it to the Streets” By Margot Duncan Jude Smith Sam Nielsen 1 June 2007

2. Aims of Presentation • To overview QUT’s new approach to student evaluation • To highlight challenges related to implementing the new approach to surveying • To discuss lessons about implementation of a major reform learned from the process

3. The Need for Change QUT Senior Executive responded to recommendations from a review ofQuality Teaching that: • Teaching evaluation should be universal for all teachers every semester in every unit; • Teaching and unit data should be provided routinely and form part of corporate reporting internally; • Individual teaching results should be reported (minimally) to the teacher, supervisor and relevant managers; • Data should form part of performance management and development discussions; and • Advice and support should be provided to enable teaching data to be used more consistently, systematically and appropriately in both performance management and in responding to students.

4. The New Evaluation Scheme at QUT QUT has been improving student evaluations processes and policies during 2007: • a new suite of internal surveys encompassing teaching, units, courses and experience has been introduced: • Learning Experience Survey • First Year Experience Survey • Exit Year Experience Survey • Mid Year Experience Survey • The surveys are fully administered and reported online • The surveys allow QUT students unprecedented opportunity to provide feedback about individual teachers and units

5. Taking it to the Streets… • The new systematic, online approach to student evaluation has raised concerns and involved challenges for students, professional staff, academics, and QUT’s senior executive. To meet these challenges the project team had to “Take it to the streets”

6. Perspective This presentation focuses upon the implementation from the perspective that the outcomes are not yet fully known.

7. Challenges and Issues STOP

8. Challenges: Students QUT has almost 40,000 students. • In the streets: • students don’t think surveys are ever used to improve teaching • students feel they are being over-surveyed • students don’t have time to complete lengthy surveys • students doubt the confidentiality of their survey comments 40,000 Students

9. Challenges: Academic Staff QUT has over 3,000 Academic Staff. • In the streets: • staff doubt the validity and accuracy of survey data • staff are concerned about promotional implications of survey data • staff have limited support in innovating their teaching practice based on survey data • staff have issues with previous online surveying efforts 3,000 Academic Staff

10. Challenges: Professional Staff Professional Staff from 9 faculties and 2 Divisions are involved. • In the streets: • professional staff have been frustrated with previous complex systems & the demands of managing mandatory evaluations • each faculty and division is structured differently & has different evaluation requirements • evaluation communication channels between faculties and divisions have not been established Professional staff: 2 divisions 3 campuses 9 faculties

11. Challenges : QUT Senior Executive • In the streets: • Senior Executive feels pressure from AUQA and LTPF. • Policy changes have union implications.

12. Challenges : Technical Staff 3 Technical Systems Technical staff working with 3 university-wide systems are involved. • In the streets: • key data systems were missing from university practice • existing technical systems were disconnected

13. Challenges: Project Management Many limitations and conflicting issues • In the streets: • Demanding deadlines • Limited funding • Politics • Uncertainty about roles • Aware of challenges for others and need for solution satisfying all Project Challenges

14. Moving ahead… Approach to and lessons learned from implementing change

15. 1. Keep the project goal in mind. …to get all stakeholders working together effectively to ensure continuous improvement of teaching through listening to the student voice.

16. 2. Consider All Perspectives Knowledge from other implementations Use of available research Constructive feedback

17. 3. Manage Project Scope … do not seek to achieve more than is possible … do not over-promise … be willing to enhance the product in the future … effectively use project committees and project sponsorship

18. 4. Bring stakeholders on the journey

19. 5. Enable Communication Undergraduate Students Human Resources Staff International Students Faculty Admin Staff Post Graduate Students IT Help desk staff The Project Team Academic Staff Senior Staff Sessional Staff T&L Support Staff Course Coordinators Corporate Systems Managers Off-shore Coordinators Learning & Teaching Consultants Corporate Performance Staff “Always be prepared to inform and be informed by stakeholders”

20. Forums Focus Groups Emails ReferenceGroups OnlineResources 6. Take every opportunity and use every method to communicate

21. Forums Focus Groups Emails ReferenceGroups OnlineResources • We used: • HR workshops for Academic PPR • Teaching Portfolio Workshops • Faculty-based Workshops • Campus based Staff Forums • IT Helpdesk training • T&L Support Staff training • We used: • Steering groups • Reference groups • Working Parties • Committees • We used: • DVC & Senior Staff email communications to Academic staff • Staff information Packs • Staff Information Web Site • Staff & Student Information Web Site • Staff Teaching Resource Web site • We used: • Student Focus groups for survey questions & Interface design • Staff Focus groups for survey questions • Staff & students in survey trials • We used: • Student advertising campaign through postcards, posters, monitor displays, unit website messages.

22. 7. Confront stakeholders’ concerns.

23. Student Confidentiality Response Rates PPR Processes Seamless experience for students & staff T&L Support Closing the Loop Sustainability of systems & processes

24. 8. Find the shortcuts … Cross organisational boundaries … Include and utilise expertise

25. 9. Trust your Intuition. • Many choices did not have ‘the’ obvious answer • Many discussions had an unspoken political aspect • Intuition as an indicator that there is a need for further ‘unpacking’ of a problem

26. 10. Think outside the box.

27. 11. Keep going.