EVS: The Good, the Bad and the Ugly

1. EVS: The Good, the Bad and the Ugly Michaela Cottee & Fiona Robins

2. Context • EVS = Electronic Voting System(s) • Used with TurningPoint software, which links with Powerpoint presentations. • University-wide, centrally funded, assessment project during 2010/11. • UHBS given 1000 EVS handsets for use in Semester B. • 500+ EVS handsets were issued to Level 4 students. • Also used with Level 5 students. • Additional handsets were used for one-off test and revision sessions (various levels).

3. EVS: Four advantages • Promotion of active learning [through anonymity] • Generation of [prompt] student feedback • Generation of staff feedback • Encourages attendance and engagement Iain Cross, Life Sciences “Blended Learning in Practice” May 2011

4. EVS: Good practice Seven principles for good practice in UG education: • EVS supports active learning through anonymity • EVS facilitates prompt student feedback • EVS encourages contact between students and staff • EVS develops reciprocity and cooperation among students • EVS emphasises time on task • EVS communicates high expectations • EVS respects diverse talents and ways of learning Chickering & Gamson (1987)

6. Overview Fiona Robins • 4BUS0154 – Decision Making Tools for Business • N = 82 students • EVS used in LECTURES to provide both formative and summative feedback Michaela Cottee • 4BUS1017 – Quantitative Analysis for Accounting and Finance • N = 146 students • EVS used in TUTORIALS to monitor attendance and to provide summative feedback

7. Fiona RobinsUsing EVS in Lectures

8. Presentation

9. Using EVS in Lectures • EVS used in a level 4 quantitative module • EVS accounted for 5% of the marks for the module • Used for both formative and summative feedback

10. Why did I use EVS To improve attendance Final Grade (%) Number of Lectures missed

11. Why did I use EVS To check understanding Basil ... “there is too much butter on those trays” Manuel ...“no no, senor, not on those tres” ....” no sir “un, dos, tres

12. Why did I use EVS To inject some fun

13. The good • Feedback to the lecturer on students learning • Instant feedback to student on their understanding • Correct their understanding • Cement learning – three times to go into brain • Records results

14. The Bad Time taken from lecture Some students too embarrassed to vote Stressful to manage lecture and EVS

15. The Ugly Managing the Technology

16. The Ugly Attendance didn’t improve Final Grade (%) Number of Lectures missed

17. Michaela CotteeUsing EVS in Tutorials

18. Assessment diet • 20% EVS tests (8 weekly in-class tests – best 4 results) • 20% StudyNet quizzes (9 weekly on-line quizzes) • 20% Mid-term test (written) • 40% End-of-module test (on-line multiple-choice)

19. Module Handbook info There will be 8 weekly in-class tests requiring the use of the EVS handsets. The best 4 results will be used to determine the mark for this assessment. The tests will be in the form of multiple choice questions which must be answered using the EVS handset. Please note: • You should bring your EVS handset (and calculator) with you to every tutorial. • You must attempt a test in order to be recorded as present at that tutorial. (In accordance with Business School policy, students who miss more than three tutorials may fail the module through non-attendance.) • No allowance will be made for forgetting your EVS handset or if the battery runs out. • Lending your EVS handset to another student will be classed as cheating. • As the best 4 results out of 8 will be used, if you miss one week (e.g. through illness) we will take the best 4 results out of 7. • You will be loaned an EVS handset for the duration of the module. Failure to return it at the end of the Semester will result in a debt to the University.

20. Solve the following equation for x:x – 4 = –2 • –2 • 2 • 4 • –4 • None of the above 45

21. If a Normal distribution has mean 10 and standard deviation 2, what is P(X < 12)? • 0.1587 • 0.4602 • 0.5398 • 0.8413 • None of the above 60

22. Test Instructions • You will need a pen, paper and calculator. • You will need a copy of the standard Normal distribution table. • You will need to use the following formula – write it down now:

23. Performance (excluding zeros) End-of-module test average: 54% Mid-term test average: 59% StudyNet quiz average: 69% EVS test average: 79% Overall average: 62% Pass rate (before resits): 88%

24. Tutorial Attendance * Test 8 took place on the last Friday afternoon of term!

25. Attendance & EVS Test Performance N = 146

26. The Good... • The system lends itself to multiple choice testing – as does Quants. • Quick and easy to create and incorporate questions into Powerpoint using TurningPoint. • The students had very few problems learning to use the EVS handsets. • Taking the “best X out of Y” results gave me a very easy way of coping with missed tutorials / forgotten handsets / faulty handsets / etc. • Improved attendance at tutorials – and easier monitoring of attendance. • Good student engagement.

27. The Good... • Students had EVS tests on all but two topics and so had to revise as they went along. • Students met each topic up to FIVE times – first of all in the lecture, then in the tutorial, then in the StudyNet Quiz, then in the EVS test and then finally in either the mid-term test or the end-of-module test. • Good marks – both for the EVS tests and overall. • Weekly feedback via StudyNet – League table (using SRNs).

28. The Bad... • Getting started – one more thing to learn incorporate into a new module in a very busy Semester! • I didn’t seek feedback from students about the use of EVS in the module. • I didn’t use the results to inform my own teaching or to chase or support weak students. • The testing format suited some types / styles of question better than others. NB. The above points reflect badly on me, but not on the use of EVS!

29. The Ugly... • It took several weeks for all students to collect handsets from SSG. • The tutors had to be trained in the use of EVS – including new tutors who took over towards the end of the module. • The tutors felt that the EVS tests ate into the tutorial time and distracted from the tutorial activity. • There was signal interference between the tutorial rooms - but coping with the consequences of this was thought to be easier than trying to get the students to change channel on the EVS handsets!

30. The Ugly (continued)... • Reports from TurningPoint were only partially helpful (but not necessarily TurningPoint’s fault!) • Results had to be entered manually into a spreadsheet. • Good support available within UHBS (Glenn Alie, Karen Robins, Richard Balawender, etc.) – but not always when it was needed! • Nothing went seriously wrong – the responses were successfully saved each week. Phew! • Allowance could not be made for students who forgot their handsets or whose handset wasn’t working. • Inclusive practice: How to cope with “extra time” students?

32. A Third Perspective Karen Robins • 5BUS0133 – OR Models • N = 64 students • EVS used in lectures to provide summative feedback – best 5 out of 7 (worth 10%) Student Feedback – Qualitative & Quantitative N= 44 respondents

33. Student Feedback “EVS is easy to use” – 98% agree “I would recommend using EVS for this module next year” –95% agree “I would recommend use of EVS technology in other Business School modules” – 90% agree “Using EVS helped my understanding of the module” – 74% agree “I was happy to be provided with an EVS handset, knowing that if I did not return it I would have to pay £35 to replace it” –40% agree; 60% disagree “Weekly EVS questions were the only reason for attending lectures – 28% agree; 72% disagree “To use EVS technology the user must be computer literate” – 21% agree; 79% disagree

34. Negative Comments • “The software would not always work” • “It’s annoying because lectures are not meant to be compulsory but I have to attend” • “If lost the handset it is too expensive to pay for it” [£35] • “Sometimes gets confusing to operate (changing channels)” • “Usually forget to take it to the class” • “When there were problems with the pad it either took too long to get fixed or trouble to get help and a new one” • “It keeps me attending lectures and puts me under pressure when I am late, plus it’s just one lecture a week which is not friendly!!!!”

35. Positive Comments • “Simple and straightforward to use the handsets” • “A good way to enhance understanding by ensuring revision on a weekly basis” • “Is fun at the same time as it is when working hard” • “Helps the teacher to have a better understanding of which students struggle more than others” • “Using EVS makes individuals come to the lecture as it counts towards the final grade” • “It’s something new and innovative” • “Save paper” “Save the environment” • “Quick answers/results”