QA on Anderson et al. 1995 Intro to CTAT

1. QA on Anderson et al. 1995Intro to CTAT CPI 494 & 598 Jan 27, 2009 Kurt VanLehn

2. Initial Framework NEXT • Step loop • User interface • Interpreting student actions • Suggesting good actions • Feedback and hints • Task selection • Assessment • Authoring and the software development • Evaluations • Dissemination

3. Describe the user interface of the Lisp Tutor • What do students type? What are the <..> things? Steps?

4. Describe the user interface of the Geometry tutor • What does student do? Steps?

5. Initial Framework • Step loop • User interface • Interpreting student actions • Suggesting good actions • Feedback and hints • Task selection • Assessment • Authoring and the software development • Evaluations • Dissemination NEXT

6. For the Lisp tutor • What counts as a correct step? • How did the tutor determine if the student’s step is correct? • Would the tutor accept both • (plus X 5) and (plus 5 X) • (times X 0.5) and (divide X 2) • How does it know? • Can students choose their own variable names?

7. For the geometry tutor • What steps count as correct? • True but useless propositions? • Top down vs. bottom up? • How does the tutor determine if the student’s step is correct?

8. Initial Framework • Step loop • User interface • Interpreting student actions • Suggesting good actions • Feedback and hints • Task selection • Assessment • Authoring and the software development • Evaluations • Dissemination NEXT

9. For the Lisp tutor • When the student asks for a hint, how does the tutor determine what step to hint? • If the tutor allowed students to keep their incorrect steps on the screen, what step would the tutor suggest when asked? • What if the tutor did not give immediate feedback and were asked for a hint?

10. For the Geometry tutor • How does it decide what step to suggest? • What does ACT theory say about this policy? • Would it be better to recognize the student’s plan and help them along by suggesting its next step?

11. Initial Framework • Step loop • User interface • Interpreting student actions • Suggesting good actions • Feedback and hints • Task selection • Assessment • Authoring and the software development • Evaluations • Dissemination NEXT

12. For the Lisp tutor • Does the “normal” version of the tutor give immediate feedback, delayed…? • What other kinds of feedback policies did they try? What were the results? • How was hinting usually done? • Did the tutor ever ask a question when asked for a hint?

13. For the Geometry tutor • Does the “normal” version of the tutor give immediate feedback, delayed…? • What other kinds of feedback policies did they try? What were the results? • How was hinting usually done? • Did the tutor ever ask a question when asked for a hint?

14. Initial Framework • Step loop • User interface • Interpreting student actions • Suggesting good actions • Feedback and hints • Assessment • Task selection • Authoring and the software development • Evaluations • Dissemination NEXT

15. Lisp tutor: Assessment • What is assessed? Motivation? Knowledge? What kind of …? • What granularity? • Give examples of what is assessed? • What’s the name of the technique? • When students enter correct step, incorrect step or hint-request, what happens to the assessment?

16. IF the goal is to get the second element of the list • THEN code car and set as a subgoal to pass to car an argument that is the tail of the list

17. Geometry tutor: Assessment • What is assessed? • Give examples • How is the assessed stuff different from Lisp’s?

18. IF the goal is to prove two triangles are congruent • THEN set as a subgoal to prove that corresponding parts are congruent.

19. Initial Framework • Step loop • User interface • Interpreting student actions • Suggesting good actions • Feedback and hints • Assessment • Task selection • Authoring and the software development • Evaluations • Dissemination NEXT

20. Lisp tutor: Task selection • What are the tasks to be selected from? • Are they authored or generated dynamically? • Who does the selection? Tutor or student? • What is the procedure for selection? • What information does the procedure need? • How does this procedure fit with ACT theory?

21. Geometry tutor: task selection • What are the tasks to be selected from? • Are they authored or generated dynamically? • Who does the selection? Tutor or student? • What is the procedure for selection? • What information does the procedure need? • How does this procedure fit with ACT theory

22. Initial Framework • Step loop • User interface • Interpreting student actions • Suggesting good actions • Feedback and hints • Assessment • Task selection • Authoring and the software development • Evaluations • Dissemination NEXT

23. Lisp stutor: Evaluations • List the evaluations done • For each, is it: Formative? Summative? Hypothesis testing? • What are the results of each?

24. a fixed set of programming exercises either with the tutor or in a standard LISP environment. Students using the tutor completed the exercises 30% faster and performed 43% better on a posttest. • using the tutor completed the exercises 64% faster and scored 30% higher on posttests than did students using a standard LISP environment.

25. What did this graph show?

26. What did this graph show?

27. Geometry tutor: Evaluations • List the evaluations done • For each, is it: Formative? Summative? Hypothesis testing? • What are the results of each?

28. In Peabody HS, one standard deviation • An ethnography • Again, but only got positive result with teacher.