What Novice Programmers Don’t Know

1. What Novice Programmers Don’t Know Gary Lewandowski Alicia Gutschow Robert McCartney Kate Sanders Dermot Shinners-Kennedy

2. Which programming concepts have little or no meaning for students? • How frequently do students indicate a concept has less meaning? • Does “don’t know” differ from “not applicable”? • Does classification vary based upon course performance? • Is it the concept or the vocabulary? ICER 2005

3. Outline • Bootstrapping • Card Sorts and Ragbags • Methodology • Results • Conclusions ICER 2005

4. Bootstrapping • NSF sponsored workshop: Sally, Marian and Josh • Multi-national, multi-institutional study • 22 researchers from four continents, six countries • Data from 247 students • How do first-competency students organize programming concepts? ICER 2005

5. Card Sorting The way things work in programs Things you can do Functions Things about variables On its own Elements in data flow What modifies the data flow Data itself Not applicable Don’t know Things contained in a program Things that are Things that are not ICER 2005

6. Ragbags • Don’t Know (DK): never knows it • Not Sure (NS): doesn’t know in this sort • Not Applicable (NA): not applicable in this sort • Boolean sorts may or may not have an NA ICER 2005

7. Methodology: Tagging the data categories NS NS On its own M Agreed on 4280 of 4410 categories NA leftovers NA NA M Things that are not NA Concepts associated with programming M M NS NS Can’t fit them in NA Sometimes you need a lawyer to settle your disputes… ICER 2005

8. Mid High Low [3,4) [4,5] [1,3) 50 students 106 students 63 students Methodology: Performance Students rated in each course 1 = failing … 5 = Picasso 28 students not rated Average Performance Rating ICER 2005

9. Dependency Decomposition Abstraction Encapsulation State Tree S S S S S S S S S S S S Methodology: Abstract Concepts 11 Scholars categorized concepts as Abstract or Concrete 7 of 11 described as abstract ICER 2005

10. boolean Methodology: Meaningfulness How meaningful is a concept to a student? Levels of how specific each one is Didn’t really know Smallest items loops If-then-else is always Meaningful (A) Boolean is usually meaningful (U) Type is limited meaningful (L) Scope is never meaningful (N) type If-then-else scope How things can be operated with by using other things Things that act on other things Things I don’t know Things that get acted on type boolean scope If-then-else Breakdown of Classes Largest components methods/functions Not applicable type scope boolean If-then-else ICER 2005

11. Data: Frequency of Use ICER 2005

12. Data: Which Concepts? ICER 2005

13. Data: Not Applicables ICER 2005

14. Data: Meaningful or Familiar? Variable is always or usually known to 238 of 247 students Object is always or usually known to 234 students State is always or usually known to 165 students. Recursion is always or usually known to 223 of 247 students Iteration: 217 Procedure: 212 Expression: 204 Deomposition, the least well known concept, is always or usually known by 126 of the 247 students. ICER 2005

15. Life, the universe, and ragbags • Ragbags used frequently, dominated by abstract concepts. • Ragbag categories are used for different purposes. • Low performers less likely to place concepts into don’t know. (Murphy/Tenenberg, do computer science students know what they know?) How do we help them see their confusion? • Dichotomous pairs: a problem and opportunity? • Yet again, programming language and paradigm are not the issue. ICER 2005