The Mystery of “b := (b = false)”

1. The Mystery of “b := (b = false)” Stuart Reges University of Washington http://www.cs.washington.edu/homes/reges/mystery

2. Data overview • 1988 AP Computer Science Exam (Pascal) • Divided into A and B (roughly CS1 and CS2) • 35 A multiple choice • 15 B multiple choice • 3 A free response • 2 B free response • 7,374 AB tests; 3,344 A-only tests

3. Analysis overview • Correlations between multiple choice questions • Any correlation below 0.2 ignored • Correlations between multiple choice and free response questions • Didn't find much of interest in correlating free response with free response

4. The mystery? • In general, not much correlated at the level of 0.2 or higher • But, five multiple choice questions kept popping up over and over • nickname: the powerhouse questions • One question in particular (question 23) stood out

5. A multiple choice correlations

6. Minus powerhouse questions

7. B multiple choice correlations

8. Minus powerhouse questions

9. Free response correlations

10. Free response vs powerhouse

11. Greatest AB vs A-only deltas

12. Mysterious question 23 If b is a Boolean variable, then the statementb := (b = false) has what effect? • It causes a compile-time error message. • It causes a run-time error message. • It causes b to have value false regardless of its value just before the statement was executed. • It always changes the value of b. • It changes the value of b if and only if b had value true just before the statement was executed.

13. And question 20 procedure Wow(n : integer); begin if n > 1 then Wow(n div 2); write(n, ' ') end; The procedure call Wow(16) will yield as output which of the following sequences of numbers? • 10 8 6 4 2 • 16 8 4 2 1 • 1 2 4 8 16 • 32 16 8 4 2 • 2 4 8 16 32

14. What does this mean? • Bob Floyd: These questions seem to test whether a student has a model of computation; whether they can play computer in their head • Don Knuth: "I conclude that roughly 2% of all people ‘think algorithmically,’ in the sense that they can reason rapidly about algorithmic processes."

15. More Knuth “The other missing concept that seems to separate mathematicians from computer scientists is related to the ‘assignment operation’ :=, which changes values of quantities. More precisely, I would say that the missing concept is the dynamic notion of the state of a process. ‘How did I get here? What is true now? What should happen next if I’m going to get to the end?’ Changing states of affairs, or snapshots of a computation, seem to be intimately related to algorithms and algorithmic thinking.”

16. Snapshot example public static int mystery(int n) { int x = 0; while (n % 2 == 0) { // Point A n = n / 2; x++; // Point B } // Point C return x; } Is (n % 2 == 0) always true, never true, or sometimes true/sometimes false at points A, B and C?

17. What next? • More exploration, more studies • Powerhouse questions suggest certain kinds of questions to explore • Is this measuring some innate aptitude? Some acquired ability? Or is it a fluke? • Join the discussion: http://www.cs.washington.edu/homes/reges/mystery