Cristina Conati Department of Computer Science University of British Columbia

1. Plan Recognition for User-Adaptive Interaction Cristina Conati Department of Computer Science University of British Columbia

2. Research Context • User-Adaptive Interaction (UAI): interaction that can better support individual users by adapting to their specific needs • User Modeling: how to infer, represent and reason about user features relevant for adaptivity. Adaptation Knowledge/Skills Beliefs/Preferences Goals/Plans User Model Activities Emotions Meta-cognitive skills ………

3. Overview • Brief examples of our plan/goal/activity recognition work in the context of UAI • Two research directions • Using eye-tracking information to facilitate plan recognition • Explaining to the user the reasoning underlying the adaptive intervensions

4. Adaptive Support To Problem Solving • A tutoring agent monitors the student’s solution and intervenes when the student needs help. • Example: Andes, tutoring system for Newtonian physics(Conati et al UMUAI 2002) N Fw = mc*g Think about the direction of N…

5. Several sources of uncertainty • Same action can belong to different solutions, or different parts of the same solution • Solutions steps can be skipped - reasoning behind the student’s actions can be hidden hidden from the tutor • Correct answers can be achieved through guessing. Errors can be due to slips • There can be flexible solution step order

6. Probabilistic Student Model • Bayesian network (automatically generated) • represents how solution steps derive from physic rules and previous steps • Captures student interface actions to perform • on-line knowledge assessment, • plan recognition • prediction of students’ actions • Performs plan recognition by integrating information about the available solutions and the student’s knowledge

7. Example 2 Solution • Find the velocity by applying the kinematics equation Vtx2 = V0x2 + 2dx*ax • Find the acceleration of the car by applying Newton's 2nd law Fx = Wx + Nx = m*ax If the student draws the axes and then gets stuck, is she • trying to write the kinematics equations to find V? • trying to find the car acceleration by applying Newton’s laws

8. 0.5 0.68 0.95 0.9 0.4 0.5 / 0.6 0.83 0.8 / 0.9 0.5 0.7 0.72 0.9 0.2 1.0 0.9 R Rule 0.5 F/G Fact/Goal Rule Application RA R -try-Newton-2law G_goal_car-acceleration? R -try-kinematics-for-velocity G_goal_car-velocity? R- choose-axis-for-Newton G-try-Newton-2law G-try-kinematics R- choose-axis-for-kinematics R-find-all-forces-on-body R-find-kinematics quantities G-find-axis-for-newton G-find-axis-for-kinematics F-N-is-normal-force-on-car F-axis-is 20 F-D-is-car-displacement F-A-is-car-acceleration CPTs

9. Evaluation • Several studies showed effectiveness of Andes tutoring • Could not evaluate the plan recognition component directly, because of lack of ground truth values (Conati et al UMUAI 2002)

10. Adaptive Support To Learning From Educational Games Tricky problem • Help students learn • While maintaining fun And engagement Model of User Knowledge Model of User Affect

11. Agent Action Outcome Goals Goals Satisfied Emotion toward Game state Emotions Towards Agent ti+1 Goal recognitoonfor Modeling User Affect (via Cognitive Appraisal) (ConatiMaclaren 2009) Personality User Action Outcome Personality Goals Goals Satisfied Interaction Patterns Interaction Patterns Emotion toward Game state Emotions Towards Self ti

12. Agreeableness Neuroticism Extraversion Conscientiousness Succeed by Myself Have Fun Learn Math Avoid Falling Beat Partner Use Mag. Glass Often Ask Advice Often Move Quickly Follow Advice Fall Often Subnetwork for Goal Assessment Personality [Costa and McCrae, 1991] Goals Interaction Patterns Links and probabilities derived from data (Zhou and Conati 2003)

13. Evaluation • DDN with goal assessment performs better than variation with goals initialized with population priors • Pretty good results on emotions recognition (~70%), but could be improved if we modeled goals as dynamic (changing priorities) (Conati et al UMUAI 2002, Conati 2010)

14. Current work • See if we can improve goal recognition by including information on user attention • In previous research, we showed that including gaze information can improve a system’s prediction of user reflection/learning (Conati and Merten, Intelligent User Interfaces 2007) • We are now looking at whether eye-tracking can help recognize user goals and intentions (hints, no hints)

15. Adaptive Support To Interface Customization MICA: Mixed-initiative support in creating a “personal interface” with tailored toolbar and menu entries (Bunt ConatiMacgrenere IUI 2007)

16. Example: Adding Features

17. Example: Adding Features

18. Interface Layout Suggestions Generation Personal interface with best expected performance Expertise User Performance with a given Personal interface Expected Usages

19. Overview • Brief examples of our plan/goal/activity recognition work in the context of UAI • Two research directions • Using eye-tracking information to facilitate plan recognition • Explaining to the user the reasoning underlying the adaptive interventions

20. One Challenge of UAI • How to provide effective adaptivity without violating the basic principles of HCI • Predictability, Controllability, Unobtrusiveness, Transparency • One possible approach: • Enable the system to explain to the user the rationale underlying its suggested adaptive interventions

21. Example: Adding Features

22. Rationale: How

23. Rationale: How

24. Formal Evaluation of Mica’s rationale • Compared versions of MICA with and without rationale [Bunt , Mcgrenere and Conati UM 2007] • Within subject laboratory study. • Participants performed guided tasks with MSWord, designed to motivate customization • User Model initialized with accurate information • Expected usages frequencies obtained from guided tasks • Expertise obtained via detailed questionnaire

25. Study 2 (Rationale vs. No Rationale): Main Findings • No performance differences • 94.2% (Rationale) vs 93.3% (No Rationale) recommendations followed

26. Preference Results • Majority of users prefer to have the rational present, but non-significant number don’t need or want it. • Identified aspects of this context that may make rationale unnecessary for some • Found the rational to be common sense • Unnecessary in a mixed-initiative interaction or productivity application • Inherent trust • Design implications: rationale should be available but not intrusive rationale no rationale

27. Open Questions • When is it important to provide the rationale? • How much information should be given? • How to handle user feedback?

28. Conclusions • Plan/Goal/Activity recognition crucial in user-adaptive interaction • Important to explore new sources of information for accurate user modeling • E.g. eye tracking • Important to increase UAI acceptance via mixed-initiative approaches, that possibly include explanations of system behavior

29. Understanding user goals and limitations in interactive with information visualizations • How can gaze information help? • video