Getting Student Feedback

1. Getting Student Feedback Dr. Michael Parkinson, School of Biotechnology.

2. Costs/Benefits • We have have a statutory requirement under the Universities Act to assess Quality. Student feedback is an integral and mandatory part of this. We should be doing it anyway. • To develop programmes, we need feedback from the stakeholders. • We have to balance staff time against the quality and quantity of feedback and its usefulness.

3. Entry Module SSOT ‘Quick feedback’ Year SSOAP Structured Discussions* Alumni CEQ* Structured Discussions Feedback mechanism

4. The Student Survey of Academic Programme (SSOAP) • A series of graded questions put to the students by Education & Management Analysis (EMA) unit in the Registry. • You don’t have to do anything. • Produces useful feedback which we have found to be consistent with other ways of getting feedback.

5. ‘Typical’ response to SSOAP

6. “Structured Discussions” • The SSOAP is a useful instrument in itself but is limited in a number of ways: • Students respond to the questions that you ask! • The responses are the untested opinions of individual students. • Focuses on the academic programme. • Can highlight what is wrong but not how to fix it. • For these reasons we piloted Structured Discussions.

7. Why Structured Discussions? • Gives an overview of a whole academic year, or a whole academic programme. • The whole point is to produce an agenda for positive change. • Highlights what is right and wrong, and also provides positive suggestions to improve quality. • Involves students deeply in the Quality Review.

8. How to set up Structured Discussions • To promote ‘frank & full’ discussions it should be facilitated by an ‘external’ Facilitator (someone not involved in teaching that group of students). • It needs a ‘Moderator’ to set it up and act as a link between staff and student. • I chose to use an external consultant as Facilitator, and a member of staff (myself) as Moderator.

9. C.a. 90 min facilitated discussion with the students. Facilitator writes a report. Report checked by the Moderator and clarified with the students. Students and Moderator produce an agenda for positive change. The report is taken to staff, and then discussed between staff and students. Overview of Structured Discussion process

10. Starts with a focused question: ‘We are interested in anything that affects the quality of your learning’. ‘Brainstorming’: Students are asked to write down as many points as they can and prioritise them. Students form into groups and discuss them. The facilitator goes around the room taking ideas from each group in turn and checking that all groups agree. Facilitator writes up & organises the points. A structured Discussion session

11. Results of structured discussions • A very high quality and quantity of feedback resulted with many suggestions for positive change. • We were able to respond immediately to some student concerns. • Some points were discussed at Programme Board level, and introduced. • Structured Discussions formed an important part of the Quality Review Document.

12. If anyone is interested in setting up Structured Discussions with their students please contact:michael.parkinson@dcu.ieI shall be organising a short training workshop, external facilitation of discussions, an opportunity to see the process in action, and support.

13. The Course Experience Questionnaire • Developed in 1975 by Ramsden. • Consists of 23-31 questions answered on a Likert Scale. • From this, 5-6 scales generated. • Administered to all alumni in Australian Universities since 1980 and results are freely and readily available! http://www.avcc.edu.au/students/gradlink/ GCCA/

14. Uses of the CEQ • Identification of areas that need to be addressed. • Internal year on year ‘Benchmarking’ of quality. • External Benchmarking.*

15. How the CEQ is being used!

16. Questions

17. Responses • From 45 questionnaires sent to 45 Alumni, 2 were returned. • Of 9 sent out to postgraduates, 6 were returned. • The EMA unit set up a web form, and I sent the URL embedded in a text message to the Alumni. We got 36 replies (about 1/4 of Alumni from previous 2 years).

18. Web Form

19. Yvonne, I would be very grateful if you could include the following in your next bulletin to Alumni. Graduates of Biotechnology only - Biotechnology Quality Review In the School of Biotechnology, we are currently conducting a review of all aspects of quality. We would like to get feedback from as many graduates as possible, and for this are using the Course Experience Questionnaire, basically a list of 24 questions, which you can access at the URL below: http://www.dcu.ie/ro/forms/survey/mibio.html It should take only 2-3 minutes to complete and will give us important feedback to help us to improve the quality of our course for future students. Mike Parkinson

20. Raw scores on a Likert scale can be automatically dumped to a comma delimited text file that can be read straight into XL. Raw scores

21. Rather than scoring on a scale of 1-5, these are converted to -100 to +100. Minus is bad, plus is good, 0 is neutral. Some of the question scores have to be reversed. All done automatically in XL. Re-calibration to -100 to +100

22. Done automatically in XL. 5 scales plus an Overall Satisfaction Index Good Teaching Generic Skills Clear Goals and Standards Appropriate Assessment Appropriate Workload Overall satisfaction index For BT for 1997-2002, all scales are on the increase. Scores converted to Scales

23. Sources • For the web-based CEQ contact Margaret Barron, Education & Management Analysis unit in the Registry. • All materials can be accessed at: http://odtl.dcu.ie/projects/Structured_discussions/Structured_discussions.html • If you are interested in setting up structured discussions contact Michael.Parkinson@DCU.ie

24. Acknowledgments • ODTL/ HEA for funding the 3 year pilot into the Structured Discussions. • EMA for setting up the web version of the CEQ. • Yvonne McLoughlin for providing email addresses, and distributing the CEQ to the Alumni with their newsletter.