Online Learning Environments and the Pedagogy of Computer Programming

1. Online Learning Environments and the Pedagogy of Computer Programming Paul Neve, David Livingstone, Gordon Hunter, James Orwell Learning Technology Research Group Faculty of Science, Engineering and Computing Kingston University

2. Computer programming is painful for everyone…! • Students • "Programming is boring…" • "Programming is difficult…" • "I don't understand programming…" • "I'm no good at programming…" • "Can I choose a combination of modules for my computing course that doesn't involve programming…?" • Lecturers • "They just don't get programming no matter what we teach or how we teach it" • "Some students just aren't equipped to be programmers" • "They might have passed the programming modules but they've no idea how to tackle the programming component of their final year project"

3. Practical Programming • A first year second semester programming module at Kingston University • Follows a first semester module that uses* Java to teach fundamental programming concepts to beginners • Sets out three aims: • To develop students' enthusiasm for practical programming • To enhance students' experience with programming environments • To develop students' confidence in their ability to write programs • Two flavours, one for Computer Science students, one for Information Systems students • In 2011 almost half the IS cohort failed even after resits had been taken into account  * or rather, attempts to use…

4. Enter NoobLab • NoobLab: an online learning environment designed specifically for teaching programming • Several "angles": • An immersive, engaging and flexibly available learning experience • Deliver some some of the feedback that a tutor might give in a programming workshop • Monitor progress, learn about how they learn, and inform the pedagogy of programming

5. NoobLab and Practical Programming • A holistic approach: • Module redesign – content, activities and assessment used NoobLab as the primary delivery tool • All summative assessment involved a practical programming activity within the environment: • Four "Small Tests": simple programming tasks • Three coursework assignments: Hangman, Tic Tac Toe and Connect 4 • One "Big Test": making live changes to the games • New features added to the environment as content dictates • Tail and dog in perfect harmony!

6. NoobLab demo If you want to follow along… http://nooblab.kingston.ac.uk Username: anything you like! Password: altc2012

7. The results… • 78% of students who completed a majority* of summative assessment tasks achieved a passing grade • Significant correlation between time spent in the NoobLab environment and final grade (R = 0.749, p < 0.001) • Students' end of module survey • An average 90% satisfaction factor+aboutthe NoobLab environment • "I did better on this module than I expected" - 83%+ • NoobLabenvironment frequently complimented in free-text feedback, e.g. "what was the best thing about the module?" * 4 or more of the 8 assessment tasks + A "satisfaction factor" was determined by giving the student a positive statement e.g. "I did better on this module than I expected to". Students assigned a mark where 1 indicated strong disagreement with the positive statement, and 5 strong agreement – i.e. higher numbers representing a better response. A maximum possible score for each based on number of responses was calculated, and the final score actually achieved is expressed as a percentage of this maximum possible.

8. Building on the experience… • Teaching the skill of "thinking like a programmer is key…" • …make the learning activities more visual • …avoid abstractions and examples that don't speak students' own language (i.e. maths!) • Carol the Shopping Trolley Wielding Robot • Inspired by Logo and Karel the Robot [1, 2] • New materials coming up for Practical Programming and other modules • [1] R. E. Pattis, Karel the Robot: A Gentle Introduction to the Art of Programming, 1st ed. John Wiley & Sons, Inc. New York, NY, USA, 1981. • [2] B. W. Becker, “Teaching CS1 with Karel the Robot in Java,” SIGCSE Bulletin, vol. 33, no. 1, pp. 50–54, ACM, New York, USA, 2001.

9. Patterns in the chaos… • Big BrotherNoobLab is watching you; logs… • Navigation through learning material • Interactions with learning material, e.g. responses to quiz questions • Any code executed – and any errors that result • Exercises attempted – and whether the criteria for the exercise was met • Common patterns between students are already evident in this data… • The "SOS" pattern • The "Rosetta Stone" pattern • We expect to find more given further analysis • Patterns can be used as triggers for feedback

10. The "Rosetta Stone" Pattern • The "Rosetta Stone" pattern happens when a student who is struggling with an exercise succeeds after examining a certain piece of content • If several students exhibit the same pattern, it might be used as a possible feedback trigger… • "You seem to be having problems with this exercise… perhaps you should check out Lesson X…" • If the "Rosetta Stone" pattern is unexpected, it might also be a trigger for a content author to re-examine the material #5 EUREKA! #2 Student goes back to lesson 2 and lesson 3 material #3 Still can't crack it…  #4 Student advances to lesson 4… particularly hovering around part 4 of this lesson… #1 Student fails first attempt at Small Test 2 #3 Student goes all the way back to the lesson 1 material

11. For more information • Email Paul Neve • paul@kingston.ac.uk • Source code (no documentation yet, though…!) • svn://wlab.paulneve.com/NoobLabPP • KU Learning Technology Research Group blog • http://ltrg.kingston.ac.uk • Papers • HEA STEM Conference 2012 – Developing Virtual Programming Laboratories to Inform the Pedagogy of Programming • http://www.heacademy.ac.uk/assets/documents/stem-conference/Computing1/Paul_Neve_David_Livingstone.pdf • IC12: International Symposium on the Intelligent Campus – Nooblab: An Intelligent Learning Environment for Teaching Programming (forthcoming, December 2012)

12. Questions…?