Programming Aptitude Testing Materials, Data analysis

1. Programming Aptitude Testing Materials, Data analysis Saeed Dehnadi Middlesex University

2. I talk about: • Test materials • Questions • Mental Models • Answer Sheet • Mark Sheet • Interpretation Algorithms • Results • Middlesex University (initial test) - 2005 • University of Newcastle (Australia) - 2006 • York - 2006 • Strathclyde - 2006 • Middlesex University (second test) - 2006 • Collaborators • Summary

3. Questions 1 (single assignment)

4. (M1) Mental model “move value” a = b; • The value of b is given to a and b changes its value to zero. a  b // b  0 Ans ( a = 20 , b = 0 ) 8th Answer

5. (M2) Mental model“copy value” a = b; • The value of b is given to a and b keeps its original value. a  b // b unchanged Correct conception of LHS  RHS Ans ( a = 20 , b = 20 ) 4th Answer

6. (M3) Mental model “move value” a = b; • The value of a is given to b and a changes its value to zero. b  a // a  0 Ans ( a = 0 , b = 10 ) 3rd Answer

7. (M4) Mental model“copy value” a = b; • The value of a is given to b and a keeps its original value. b  a // a unchanged Ans ( a = 10 , b = 10 ) 1st Answer

8. Mental models (M5) (copy + add) value a = b; • The sum of a and b is given to a, and b keeps its original value. a  (a + b) // b unchanged Ans ( a = 30 , b = 20 ) 2nd Answer

9. Mental models (M6) (move + add) value a = b; • The sum of a and b is given to a, and b changes its value to zero. a  (a + b) // b  0 Ans ( a = 30 , b = 0 ) 10th Answer

10. Mental models (M7) (copy + add) value a = b; • The sum of a and b is given to b, and a keeps its original value. b  (a + b) // a unchanged Ans ( a = 10 , b = 30 ) 9th Answer

11. Mental models (M8) (move + add) value a = b; • The sum of a and b is given to b, and a changes its value to zero. b  (a + b) // a  0 Ans ( a = 0 , b = 30 ) 5th Answer

12. Mental models (M9) “no changes” a = b; • a and b keep their original values. a unchanged // b unchanged Ans ( a = 10 , b = 20 ) 6th Answer

13. Mental models (M10) Equality a = b; • Assignment is a simple equation, and then all equal values of a and b are acceptable. Ans ( a = 10 , b = 10 ) and Ans ( a = 20 , b = 20 ) both 1st and 4th answers

14. Mental models (M11) “Swap values” a = b; • a and b swap their values simultaneously. a  b a gets b’s value b  a b gets a’s value Ans ( a = 20 , b = 10 ) 7th Answer

15. Models for a single assignmentint a = 10; int b = 20;a = b;

16. Answer Sheet (Question1)

17. Questions 2 (single assignment)

18. Questions 3 (single assignment)

19. Question 4 (two assignments)

20. Mental model“M1 + Sequence” a = b; b = a; Model is M1 that applies sequentially through both statements: • L1) a  b and b  0 then a = 20 and b = 0 • L2) b  a and a  0 then b = 20 and a = 0 Single answer: a = 0 , b = 20

21. Mental model“M1 + Independent (M1 I)” a = b; b = a; • Model is M1 that applies independently for each individual line. • L1) a  b and b  0 • L2) b  a and a  0 Multiple answers: a = 20 and b = 0 a = 0 and b = 10

22. Mental model“M1 + Simultaneous-single (M1 Ss)” a = b; b = a; • M1 Ss Same as (M1 I), subjects only interested on Left-hand-side values of statements and ignores the right-hand-side values. • L1) a  b and b  ignores • L2) b  a and a  ignores Single answer: a = 20 , b = 10

23. Answer Sheet (question4)

24. Question 5 (two assignments)

25. Question 6 (two assignments)

26. Answer Sheet (question6)

27. Question 7 (three assignments)

28. Answer Sheet (question7)

29. Mark Sheet

30. Mark Sheet Interpretation(Tree Structure)

31. Mark Sheet Interpretation (Mode Structure)

32. Data sample

33. Middlesex University Barnet College First Experiment 2005 Data provided by: Saeed DehnadiMiddlesex University

34. Experiment • Subjects: • 30 students from Middlesex University Introductory to programming (Java) • 31 students from Barnet College Further Education Programming Course • Time: • Week 0 of the course • Questions • 12 questions

35. CategoriesWeek 0 • Consistent (45%) • Inconsistent (39%) • Blank (16%)

36. CategoriesWeek 3 • Consistent (67%) • Inconsistent (33%) • Blank (0%)

37. Stable group membership(Week3) Data provided by: Saeed Dehnadi Middlesex University

38. Pass/Fail statistics(Initial Result) P < = 0.01 Data provided by: Saeed Dehnadi Middlesex University

39. Two populations(Initial Result)

40. Australian Data University of Newcastle 2006 Data provided by: SimonSchool of DCIT (Design, Communication, and Information Technology)

41. Pass/Fail statistics(First glance) P < = 0.01 Data provided by: Simon The University of Newcastle

42. Pass/Fail statisticsExclude m2 P < = 0.025 Data provided by: Simon The University of Newcastle

43. Pass/Fail statisticsExclude m2 P < = 0.001 Data provided by: Simon The University of Newcastle

44. Strathclyde Data The University of Strathclyde 2006 Data provided by: Linxiao Mu

45. Mental models in week 20 Value assignment Reference assignment Data provided by: Linxiao Mu The University of Strathclyde

46. Value assignment mental models in final exam P < = 0.05 Data provided by: Linxiao Mu The University of Strathclyde

47. Value assignment mental models in 4 in-course tests P < = 0.01 Data provided by: Linxiao Mu The University of Strathclyde

48. Reference assignment mental models in final exam P < = 0.025 Data provided by: Linxiao Mu The University of Strathclyde

49. Reference assignment mental models in 4 in-course tests P < = 0.05 Data provided by: Linxiao Mu The University of Strathclyde

50. York The University of York 2006 Data provided by: Dimitar Kazakov