Developing on-line learning aids for M150 Students

1. Michael Isherwood Developing on-line learning aids for M150 Students Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT)Centre for Excellence in Teaching and Learning The Open University is incorporated by Royal Charter (RC 000391), an exempt charity in England & Wales and a charity registered in Scotland (SC 038302).

2. Presentation Summary • Project Aims • Background • The Quizzes • Initial Analyses • Student Views • 2nd Pilot and Beyond

3. Project Aims - summary • Support students in their understanding of new and difficult parts of M150 (in effect, Block 2) • Confirm understanding prior to tma feedback • Investigate extension to other computing courses • Through better understanding, aid retention

4. Background • Students find getting to grips with programming difficult • This leads to frustration and dropping out of M150 (and therefore further computing courses)

5. Background – 2006J TMAs

6. Background - TMA02/03 Marks

7. On-Line Quizzes • 4 Quizzes in First Pilot in April/May 2008 • Structured English • Conditions, truth and trace tables • if constructs in JavaScript • Syntax in JavaScript • Question Types • Selecting tick boxes, radio buttons • Free text and expression entry • Entering values • Drag and drop

8. 67% correct at 1st try • further 21% at 2nd attempt • 3% correct at 3rd attempt • 10% remained wrong if constructs – Q1 • Most errors were • < > instead of != • = rather than == feedback for this Q is to indicate wrong answers. As always, correct answer is shown at the end

9. 83% correct at 1st attempt • Only 4% wrong after 3rd attempt if constructs – Q2 Note Feedback after 1st attempt After 2nd attempt the correct answers remain, so student concentrates on incorrect answer(s)

10. if construct – Q2 cont

11. 75% of responses correct at 1st attempt • 3% wrong after 3rd attempt. if constructs – Q3 The majority of the errors were at the boundary, as here, though several students reversed After a 2nd attempt, the correct answers were retained, as shown, for the final attempt.

12. if constructs – Q6 On 1st attempt the number of wrong answers is given along with a reminder of the OR (||) operator's effect. The 2nd attempt has feedback as above and the 3rd attempt starts with all the correct responses retained.

13. Conclusions on Questions • Most students complete quizzes in 10 – 15 minutes • Most students get questions right first time and few fail to be correct after third attempt • Most students find the quizzes valuable and the feedback helpful • Only one question “didn’t work” • Variants of a question generally seemed equivalent

14. Caveats on Conclusions • Sample of students small (<=80) • Sample is self selecting

15. Student Questionnaire on Quiz

16. Student Responses • 96% of students found the quiz enjoyable • 91% found the quiz useful • The majority of students didn’t suggest improvements or make any other comment, however some useful suggestions and observations were made and one or two errors were identified

17. Outstanding Analysis • Review the tma performance and overall result of students who participated in the quizzes • Analyse remaining quizzes

18. 2nd Pilot • 2008J December 2008 / January 2009 • Existing Quizzes honed • Further 2 or 3 quizzes • Important that more students participate – a wider range, including weaker students – how can this be achieved?

19. The Future • Similar Open Mark Quizzes form an integral part of the new Science level 1 course, S104 • Should such quizzes form part of M150’s replacement along with podcasts and the videos and powerpoint presentations that Ian Cooke has developed for M150? • A major benefit of such quizzes could be that a tutor can monitor a student’s performance and intervene with struggling students long before the tma becomes due, or a quiz hasn’t been attempted. To be effective, this would require an element of compulsion.

20. Studies Section Does Quiz Next Section A Vision “Pod” + Ppt (IC) Special Session Tutor Advice